Perception of first- and second-order motion: Separable neurological mechanisms?

Unraveling nonlinear electrophysiologic processes in the human visual system with full dimension spectral analysis

Natural sensory signals have nonlinear structures dynamically composed of the carrier frequencies and the variation of the amplitude (i.e., envelope). How the human brain processes the envelope information is still poorly understood, largely due to the conventional analysis failing to quantify it directly. Here, we used a recently developed method, Holo-Hilbert spectral analysis, and steady-state visually evoked potential collected using electroencephalography (EEG) recordings to investigate how the human visual system processes the envelope of amplitude-modulated signals, in this case with a 14 Hz carrier and a 2 Hz envelope. The EEG results demonstrated that in addition to the fundamental stimulus frequencies, 4 Hz amplitude modulation residing in 14 Hz carrier and a broad range of carrier frequencies covering from 8 to 32 Hz modulated by 2 Hz amplitude modulation are also found in the two-dimensional frequency spectrum, which have not yet been recognized before. The envelope of the stimulus is also found to dominantly modulate the response to the incoming signal. The findings thus reveal that the electrophysiological response to amplitude-modulated stimuli is more complex than could be revealed by, for example, Fourier analysis. This highlights the dynamics of neural processes in the visual system.

Visual motion serves but is not under the purview of the dorsal pathway

Visual motion processing is often attributed to the dorsal visual pathway despite visual motion's involvement in almost all visual functions. Furthermore, some visual motion tasks critically depend on the structural integrity of regions outside the dorsal pathway. Here, based on numerous studies, I propose that visual motion signals are swiftly transmitted via multiple non-hierarchical routes to primary motion-dedicated processing regions (MT/V5 and MST) that are not part of the dorsal pathway, and then propagated to a multiplicity of brain areas according to task demands, reaching these regions earlier than the dorsal/ventral hierarchical flow. This not only places MT/V5 at the same or even earlier visual processing stage as that of V1, but can also elucidate many findings with implications to visual awareness. While the integrity of the non-hierarchical motion pathway is necessary for all visual motion perception, it is insufficient on its own, and the transfer of visual motion signals to additional brain areas is crucial to allow the different motion perception tasks (e.g. optic flow, visuo-vestibular balance, movement observation, dynamic form detection and perception, and even reading). I argue that this lateral visual motion pathway can be distinguished from the dorsal pathway not only based on faster response latencies and distinct anatomical connections, but also based on its full field representation. I also distinguish between this primary lateral visual motion pathway sensitive to all motion in the visual field, and a much less investigated optic flow sensitive medial processing pathway (from V1 to V6 and V6A) that appears to be part of the dorsal pathway. Multiple additional predictions are provided that allow testing this proposal and distinguishing between the visual pathways.

Orientation-cue invariant population responses to contrast-modulated and phase-reversed contour stimuli in macaque V1 and V2

Visual scenes can be readily decomposed into a variety of oriented components, the processing of which is vital for object segregation and recognition. In primate V1 and V2, most neurons have small spatio-temporal receptive fields responding selectively to oriented luminance contours (first order), while only a subgroup of neurons signal non-luminance defined contours (second order). So how is the orientation of second-order contours represented at the population level in macaque V1 and V2? Here we compared the population responses in macaque V1 and V2 to two types of second-order contour stimuli generated either by modulation of contrast or phase reversal with those to first-order contour stimuli. Using intrinsic signal optical imaging, we found that the orientation of second-order contour stimuli was represented invariantly in the orientation columns of both macaque V1 and V2. A physiologically constrained spatio-temporal energy model of V1 and V2 neuronal populations could reproduce all the recorded population responses. These findings suggest that, at the population level, the primate early visual system processes the orientation of second-order contours initially through a linear spatio-temporal filter mechanism. Our results of population responses to different second-order contour stimuli support the idea that the orientation maps in primate V1 and V2 can be described as a spatial-temporal energy map.

Weaker signals induce more precise temporal-integration

It is well known that our perceptual system integrates local information to generate global percepts. Although events in the real world occur over time, it is still unclear how we integrate temporally dispersed information. Here, I systematically explore the relation between the intensity of local signals and the resultant of temporal-integration. Presented sequentially with two directions of variably coherent random dot patterns (40 degrees apart), participants were instructed to report a perceived motion direction by using a mouse driven cursor. As the strength of the motion signal was varied, the behavioral results and qualitative analyses indicate existence of two aspects of temporal-integration, subthreshold integration and suprathreshold integration. In subthreshold integration (when the local signals are subthreshold), stronger signals enhance integration. On the other hand, in suprathreshold integration (when the signals are above threshold), stronger signals decrease integration. As a result, relatively weaker, subthreshold or near threshold signals produce more precise temporal-integration than suprathreshold signals.

Direction-selective patterns of activity in human visual cortex suggest common neural substrates for different types of motion

A sense of motion can be elicited by the movement of both luminance- and texture-defined patterns, what is commonly referred to as first- and second-order, respectively. Although there are differences in the perception of these two classes of motion stimuli, including differences in temporal and spatial sensitivity, it is debated whether common or separate direction-selective mechanisms are responsible for processing these two types of motion. Here, we measured direction-selective responses to luminance- and texture-defined motion in the human visual cortex by using functional MRI (fMRI) in conjunction with multivariate pattern analysis (MVPA). We found evidence of direction selectivity for both types of motion in all early visual areas (V1, V2, V3, V3A, V4, and MT+), implying that none of these early visual areas is specialized for processing a specific type of motion. More importantly, linear classifiers trained with cortical activity patterns to one type of motion (e.g., first-order motion) could reliably classify the direction of motion defined by the other type (e.g., second-order motion). Our results suggest that the direction-selective mechanisms that respond to these two types of motion share similar spatial distributions in the early visual cortex, consistent with the possibility that common mechanisms are responsible for processing both types of motion.

Visual deficits in amblyopia constrain normal models of second-order motion processing

It is well established that amblyopes exhibit deficits in processing first-order (luminance-defined) patterns. This is readily manifest by measuring spatiotemporal sensitivity (i.e. the "window of visibility") to moving luminance gratings. However the window of visibility to moving second-order (texture-defined) patterns has not been systematically studied in amblyopia. To address this issue monocular modulation sensitivity (1/threshold) to first-order motion and four different varieties of second-order motion (modulations of either the contrast, flicker, size or orientation of visual noise) was measured over a five-octave range of spatial and temporal frequencies. Compared to normals amblyopes are not only impaired in the processing of first-order motion, but overall they exhibit both higher thresholds and a much narrower window of visibility to second-order images. However amblyopia can differentially impair the perception of some types of second-order motion much more than others and crucially the precise pattern of deficits varies markedly between individuals (even for those with the same conventional visual acuity measures). For the most severely impaired amblyopes certain second-order (texture) cues to movement in the environment are effectively invisible. These results place important constraints on the possible architecture of models of second-order motion perception in human vision.

Neuropsychological evidence for three distinct motion mechanisms

We describe psychophysical performance of two stroke patients with lesions in distinct cortical regions in the left hemisphere. Both patients were selectively impaired on direction discrimination in several local and global second-order but not first-order motion tasks. However, only patient FD was impaired on a specific bi-stable motion task where the direction of motion is biased by object similarity. We suggest that this bi-stable motion task may be mediated by a high-level attention or position based mechanism indicating a separate neurological substrate for a high-level attention or position-based mechanism. Therefore, these results provide evidence for the existence of at least three motion mechanisms in the human visual system: a low-level first- and second-order motion mechanism and a high-level attention or position-based mechanism.

No interaction of first- and second-order signals in the extraction of global-motion and optic-flow

Edwards and Badcock (Vision Research 35, 2589, 1995) argued for independent first-order (FO) and second-order (SO) motion systems up to and including the global-motion level. That study used luminance (which they called FO) and contrast (SO) modulated dots. They found that SO noise dots did not mask signal extraction with luminance increment dots while luminance increment dots did mask SO signal extraction. However, they argued this asymmetry was not due to a combined FO-SO pathway, but rather due to the fact that the luminance-modulated dots, being also local variations in contrast, are both FO and SO stimuli. We test their claim of FO and SO independence by using a stimulus that can generate pure FO and SO signals, specifically one consisting of multiple Gabors (the global-Gabor stimulus) in which the Gaussian envelopes are static and the carriers drift. The carrier can either be luminance-modulated (FO) or contrast-modulated (SO) and motion signals from the randomly-oriented local Gabors must be combined to detect the global-motion vector. Results show no cross-masking of FO and SO signals, thus supporting the hypothesis of independent FO and SO systems up to and including the level extracting optic-flow.

Distinct position assignment mechanisms revealed by cross-order motion

Motion perception influences perceived position. It has been shown that first-order (luminance defined) motion shifts perceived position across a wide range of spatial and temporal frequencies. On the other hand, second-order (contrast defined) motion shifts perceived position over a narrow range of temporal frequencies, regardless of spatial frequency [Bressler, D. W., & Whitney, D. (2006). Second-order motion shifts perceived position. Vision Research, 46(6-7), 1120-1128]. These results suggest the presence of distinct position assignment mechanisms for first- and second-order motion. We investigated whether the first- and second-order systems independently encode and assign the position of a moving stimulus. To measure motion induced position shift we presented two horizontally offset Gabors placed above and below a central fixation point, with sine wave carriers drifting in opposite directions. Subjects judged the position of the top Gabor relative to the bottom one. We used both first-order Gabors (sinusoidal luminance modulation of a dynamic noise carrier enveloped by a static Gaussian) and second-order Gabors (sinusoidal contrast modulation of a dynamic noise carrier enveloped by a static Gaussian). Results showed a strong position shift in the direction of the carrier motion when both Gabors were first-order, a weak position shift when both Gabors were second-order, and no appreciable position shift when one Gabor was first-order and the other was second-order (cross-order motion). The absence of a position shift using cross-order motion supports the hypothesis that the two motion systems independently encode and assign the position of a moving object. These results are consistent with those of experiments investigating global spatial interactions between static first-order and second-order Gabor patches, indicating a commonality in the underlying spatial integration processes.

Inversion of Perceived Direction of Motion Caused by Spatial Undersampling in Two Children with Periventricular Leukomalacia

We report here two cases of two young diplegic patients with cystic periventricular leukomalacia who systematically, and with high sensitivity, perceive translational motion of a random-dot display in the opposite direction. The apparent inversion was specific for translation motion: Rotation and expansion motion were perceived correctly, with normal sensitivity. It was also specific for random-dot patterns, not occurring with gratings. For the one patient that we were able to test extensively, contrast sensitivity for static stimuli was normal, but was very low for direction discrimination at high spatial frequencies and all temporal frequencies. His optokinetic nystagmus movements were normal but he was unable to track a single translating target, indicating a perceptual origin of the tracking deficit. The severe deficit for motion perception was also evident in the seminatural situation of a driving simulation video game. The perceptual deficit for translational motion was reinforced by functional magnetic resonance imaging studies. Translational motion elicited no response in the MT complex, although it did produce a strong response in many visual areas when contrasted with blank stimuli. However, radial and rotational motion produced a normal pattern of activation in a subregion of the MT complex. These data reinforce the existent evidence for independent cortical processing for translational, and circular or radial flow motion, and further suggest that the two systems have different vulnerability and plasticity to prenatal damage. They also highlight the complexity of visual motion perception, and how the delicate balance of neural activity can lead to paradoxical effects such as consistent misperception of the direction of motion. We advance a possible explanation of a reduced spatial sampling of the motion stimuli and report a simple model that simulates well the experimental results.

Eye Movements in Response to Dichoptic Motion: Evidence for a Parallel-Hierarchical Structure of Visual Motion Processing in Primates

Brief movements of a large-field visual stimulus elicit short-latency tracking eye movements termed “ocular following responses” (OFRs). To address the question of whether OFRs can be elicited by purely binocular motion signals in the absence of monocular motion cues, we measured OFRs from monkeys using dichoptic motion stimuli, the monocular inputs of which were flickering gratings in spatiotemporal quadrature, and compared them with OFRs to standard motion stimuli including monocular motion cues. Dichoptic motion did elicit OFRs, although with longer latencies and smaller amplitudes. In contrast to these findings, we observed that other types of motion stimuli categorized as non-first-order motion, which is undetectable by detectors for standard luminance-defined (first-order) motion, did not elicit OFRs, although they did evoke the sensation of motion. These results indicate that OFRs can be driven solely by cortical visual motion processing after binocular integration, which is distinct from the process incorporating non-first-order motion for elaborated motion perception. To explore the nature of dichoptic motion processing in terms of interaction with monocular motion processing, we further recorded OFRs from both humans and monkeys using our novel motion stimuli, the monocular and dichoptic motion signals of which move in opposite directions with a variable motion intensity ratio. We found that monocular and dichoptic motion signals are processed in parallel to elicit OFRs, rather than suppressing each other in a winner-take-all fashion, and the results were consistent across the species.

First and second-order motion perception after focal human brain lesions

Perception of visual motion includes a first-order mechanism sensitive to luminance changes and a second-order motion mechanism sensitive to contrast changes. We studied neural substrates for these motion types in 142 subjects with visual cortex lesions, 68 normal controls and 28 brain lesion controls. On first-order motion, the visual cortex lesion group performed significantly worse than normal controls overall and in each hemifield, but second-order motion did not differ. Only one individual showed a selective second-order motion deficit. Motion deficits were seen with lesions outside the small occipito-temporal region thought to contain a human homolog of motion processing area MT (V5), suggesting that many areas of human brain process visual motion.

Priming of first- and second-order motion: Mechanisms and neural substrates

Priming for luminance-modulated (first-order) motion has been shown to rely on the functional integrity of visual area V5/MT [Campana, G., Cowey, A., & Walsh, V. (2002). Priming of motion direction and area V5/MT: A test of perceptual memory. Cerebral Cortex, 12, 663-669; Campana, G., Cowey, A., & Walsh, V. (2006). Visual area V5/MT remembers "what" but not "where". Cerebral Cortex, 16, 1766-1770]. The high retinotopical organization of this area would predict that direction priming is sensitive to spatial position. In order to test this hypothesis, and to see whether a similar priming mechanism also exists with second-order motion, we tested motion direction priming and its interaction with spatial position with both first- and second-order motion. Indeed, whereas a number of studies have pinpointed the specific mechanisms and neural substrates for these two kinds of motion perception that appear to be (partially) non-overlapping (i.e., Lu, Z. L., & Sperling, G. (2001). Three-systems theory of human visual motion perception: Review and update. Journal of the Optical Society of America A, 18, 2331-2370; Vaina, L. M., & Soloviev, S. (2004). First-order and second-order motion: Neurological evidence for neuroanatomically distinct systems. Progress in Brain Research, 144, 197-212), the mechanisms and neural substrates mediating implicit memory for first- and second-order motion are still unknown. Our results indicate that priming for motion direction occurs not only with first-order but also with second-order motion. Priming for motion direction is position-sensitive both with first- and second-order motion, suggesting for both processes a locus of representation where retinotopicity is still maintained, that is within the V5/MT complex but earlier than MST. Cross-order motion priming also exists but is not sensitive to spatial position, suggesting that the locus where processing of first- and second-order motion converge is situated in MST or beyond.

Initial ocular following in humans: a response to first-order motion energy

Visual motion is sensed by low-level (energy-based) and high-level (feature-based) mechanisms. Ocular following responses (OFR) were elicited in humans by applying horizontal motion to vertical square-wave gratings lacking the fundamental ("missing fundamental stimulus"). Motion consisted of successive 1/4-wavelength steps, so the features and 4n+1 harmonics (where n=integer) shifted forwards, whereas the 4n-1 harmonics--including the strongest Fourier component (the 3rd harmonic)--shifted backwards (spatial aliasing). Initial OFR, recorded with the electromagnetic search coil technique, were always in the direction of the 3rd harmonic, e.g., leftward steps resulted in rightward OFR. Thus, the earliest OFR were strongly dependent on the motion of the major Fourier component, consistent with early spatio-temporal filtering prior to motion detection, as in the well-known energy model of motion analysis.

The influence of spatial and temporal noise on the detection of first-order and second-order orientation and motion direction

Thresholds for identifying the direction of second-order motion (contrast-modulated dynamic noise) are consistently higher than those for identifying spatial orientation, unlike first-order gratings for which the two thresholds are typically the same. Two explanations of this phenomenon have been proposed: either first-order and second-order patterns are encoded by separate mechanisms with different properties, or dynamic noise selectively impairs ("masks") sensitivity to second-order motion direction but not orientation. The former predicts the two thresholds should remain distinct for second-order patterns, irrespective of the temporal structure (static vs. dynamic) of the noise carrier. The latter predicts direction thresholds should be higher than orientation thresholds, for both second-order and first-order motion patterns, when dynamic (but not static) noise is present. To resolve this issue we measured direction and orientation thresholds for first-order (luminance) and second-order (contrast or polarity) modulations of static or dynamic noise. Results were decisive: The two thresholds were invariably the same for first-order stimuli but markedly different (direction thresholds approximately 50% higher) for second-order stimuli, regardless of the temporal properties (static or dynamic) and the overall contrast of the noise, or the drift temporal frequency of the envelope. This suggests that first-order and second-order motion are encoded separately and that the mechanisms encoding second-order stimuli cannot determine direction at the absolute threshold for spatial form.

The Ferrier Lecture 2004 what can transcranial magnetic stimulation tell us about how the brain works?

Transcranial magnetic stimulation (TMS) is a technique whereby parts of the cerebral cortex and underlying white matter can be excited by a brief electrical current induced by a similarly brief, rapidly fluctuating magnetic field which is itself produced by rapidly discharging a current through an insulated coil held against the scalp. When combined with magnetic resonance structural and functional images of the subject's brain, the stimulation can be directed at specific cortical areas. Over a period of only 15 years, TMS has revealed hitherto unsuspected aspects of brain function, such as the role of distant parts of the brain in recovery from stroke, and has helped to resolve several previously intractable disputes, such as the neuronal basis of conscious awareness. This article describes and discusses the origins and nature of TMS, its applications and limitations, and its especial usefulness in conjunction with other techniques of evaluating or imaging brain activity.

Deficits of motion integration and segregation in patients with unilateral extrastriate lesions

Functional neuroimaging in human subjects and single cell recordings in monkeys show that several extra-striate visual areas are activated by visual motion. However, the extent to which different types of motion are processed in different regions remains unclear, although neuropsychological studies of patients with circumscribed lesions hint at regional specialization. We, therefore, studied four patients with unilateral damage to different regions of extrastriate visual cortex on a series of visual discrimination tasks that required them, to a different extent, to integrate local motion signals in order to correctly perceive the direction of global motion. Performance was assessed psychophysically and compared with that of control subjects and with the patients' performance with stimuli presented in the visual field ipsilateral to the lesion. The results indicate considerable regional specialization in extra-striate regions for different aspects of motion processing, namely the largest displacement from frame to frame (D-max) that can sustain perception of coherent motion; perception of relative speed; the amount of coherent motion needed to sustain a percept of global motion in a particular direction; the detection of discontinuities within a moving display; the extraction of form from motion. It was also clear that a defect in local motion, i.e. D-max, can be overcome by integrating local motion signals over a longer period of time. Although no patient suffered from only one defect, the overall pattern of results strongly supports the notion of regional specialization for different aspects of motion processing.

Sensitivity to spatial and temporal modulations of first-order and second-order motion

This study characterises the spatiotemporal "window of visibility" for first-order motion (luminance-modulated noise) and three varieties of second-order motion (contrast-modulated, polarity-modulated and spatial length-modulated noise). Direction-identification thresholds (minimum modulation depth producing 79.4% correct) were measured for each motion pattern (acuity permitting) over a five octave range of spatial and temporal frequencies (0.5-16 c/deg and 0.5-16 Hz respectively). Thresholds were converted into modulation sensitivity (1/threshold). For first-order motion patterns, sensitivity functions were generally bandpass. However, for second-order motion patterns, functions were predominantly lowpass in nature. In particular, the functions corresponding to contrast-modulated and polarity-modulated noise were virtually identical in terms of shape and sensitivity. However, sensitivity to modulations of spatial length was extremely poor and more lowpass, suggesting that additional strategies, perhaps a feature-based system, may be required for encoding motion of images of this type.

Human cortical responses to contrast modulations of visual noise

We studied visual evoked potentials (VEPs) elicited by second-order contrast modulations of binary dynamic noise and first-order luminance modulations. Using a 3-point Laplacian operator centred on Oz, we found that contrast modulations of both low and higher spatial frequencies elicited a negative component whose latency was about 200 ms. The latency of this component was significantly longer than that of the early Laplacian components to first-order luminance modulations. These findings could be due to slower first-stage linear filters and additional processing stages of the second-order pathway. The topographical analysis of scalp recorded VEPs to central and half-field stimulation has suggested that the responses to second-order patterns are likely to be generated by neuronal structures within the primary visual cortex which may have inputs from extrastriate neurons via feedback connections.

Role of the superior temporal region in human visual motion perception

While moving objects are usually seen using luminance (first-order) cues, humans can perceive the motion of objects via non-luminance (second-order) cues. Contrary to previous case reports, no physiological studies have elucidated distinct differences in the cortical regions involved in first- and second-order motion processes. We investigated brain responses related to these two types of motion perception in human subjects using 3 T functional magnetic resonance imaging and strictly controlled apparent motion stimulus pairs. Comparison of brain activation to moving versus static states of each motion stimulus isolated cortical activity related to each type of motion perception. We found a selective neural response to second-order motion stimulus in the anterior part of the superior temporal sulcus (STS) contralateral to stimulus presentation and cue-invariant activation of MT/V5+. No significant activation in the STS was observed by the first-order motion, even when its visibility was reduced to levels comparable to that of second-order motion. Furthermore, the STS demonstrated significant activation for highly visible motion stimulus with both first- and second-order attributes. The STS represents the cardinal structure for perception of second-order motions, although further studies are needed to elucidate the exact neural process occurring in this area.

The effect of disrupting the human magnocellular pathway on global motion perception

The purpose of this study was to demonstrate the effect of human magnocellular (M)-pathway disruption on global motion perception. Coherence thresholds for global motion direction discrimination in random dot patterns were determined at slow and moderate dot speeds: (1) after adaptation to full-field sinusoidal flicker or a steady gray field, and (2) on a red or a gray background. Adaptation to flicker and a red background increased motion coherence thresholds relative to the gray baseline conditions at both dot speeds. Physiological studies have shown that M cells in the retina and LGN are inhibited by red light and are a main contributor to flicker perception in monkeys. Therefore, our results suggest that interference with processing in the subcortical M pathway disrupts higher-level motion integration.

Deficits in the processing of local and global motion in very low birthweight children

This study evaluated the impact of premature birth on the development of local and global motion processing in a group of very low birthweight (<1500 g), 5- to 8-year-old children. Sensitivity to first- and second-order local motion stimuli and coherence thresholds for global motion in random dot kinematograms were measured. Relative to full-term controls, premature children showed deficits on all three aspects of motion processing. These problems could not be accounted for by stereo deficits, amblyopia, or attentional problems. A history of mild retinopathy of prematurity and/or intraventricular hemorrhage increased risk, but deficits were observed in some children with no apparent ocular or cerebral pathology. It is important to note that, despite the observed group differences, individual profiles of performance did vary; the results suggest that these three forms of motion processing may involve separate neural mechanisms. These findings serve to increase our understanding of the organization and functional development of motion-processing subsystems in humans, and of the impact of prematurity and associated complications on visual development.

Spatial frequency selective masking of first-order and second-order motion in the absence of off-frequency 'looking'

Converging evidence suggests that, at least initially, first-order (luminance defined) and second-order (e.g. contrast defined) motion are processed independently in human vision. However, adaptation studies suggest that second-order motion, like first-order motion, may be encoded by spatial frequency selective mechanisms each operating over a limited range of scales. Nonetheless, the precise properties of these mechanisms are indeterminate since the spatial frequency selectivity of adaptation aftereffects may not necessarily represent the frequency tuning of the underlying units [Vision Research 37 (1997) 2685]. To address this issue we used visual masking to investigate the spatial-frequency tuning of the mechanisms that encode motion. A dual-masking paradigm was employed to derive estimates of the spatial tuning of motion sensors, in the absence of off-frequency 'looking'. Modulation-depth thresholds for identifying the direction of a sinusoidal test pattern were measured over a 4-octave range (0.125-2 c/deg) in both the absence and presence of two counterphasing masks, simultaneously positioned above and below the test frequency. For second-order motion, the resulting masking functions were spatially bandpass in character and remained relatively invariant with changes in test spatial frequency, masking pattern modulation depth and the temporal properties of the noise carrier. As expected, bandpass spatial frequency tuning was also found for first-order motion. This provides compelling evidence that the mechanisms responsible for encoding each variety of motion exhibit spatial frequency selectivity. Thus, although first-order and second-order motion may be encoded independently, they must utilise similar computational principles.

First- and second-order information in natural images: a filter-based approach to image statistics

Previous analyses of natural image statistics have dealt mainly with their Fourier power spectra. Here we explore image statistics by examining responses to biologically motivated filters that are spatially localized and respond to first-order (luminance-defined) and second-order (contrast- or texture-defined) characteristics. We compare the distribution of natural image responses across filter parameters for first- and second-order information. We find that second-order information in natural scenes shows the same self-similarity previously described for first-order information but has substantially less orientational anisotropy. The magnitudes of the two kinds of information, as well as their mutual unsigned correlation, are much stronger for particular combinations of filter parameters in natural images but not in unstructured fractal images having the same power spectra.

Is precise discrimination of low level motion needed for heading discrimination?

Normal observers judge heading well both when moving in a straight line and when moving along a curved path. Judgments of curved path motion require depth variations in the scene while judgments of straight line heading (pure translation) do not. Here we show that a stroke patient who is impaired in low level 2D motion discrimination tasks and cannot accurately judge 3D structure from motion can accurately judge heading for straight line self-motion. This patient is impaired in judgments of curved path self-motion. This suggests that accurate heading judgments for observer translation do not require accurate 2D motion perception or 3D reconstruction of the scene. Judgments of curved path motion appear more dependent on accurate 2D motion perception.

A higher order motion region in human inferior parietal lobule: evidence from fMRI

The proposal that motion is processed by multiple mechanisms in the human brain has received little anatomical support so far. Here, we compared higher- and lower-level motion processing in the human brain using functional magnetic resonance imaging. We observed activation of an inferior parietal lobule (IPL) motion region by isoluminant red-green gratings when saliency of one color was increased and by long-range apparent motion at 7 Hz but not 2 Hz. This higher order motion region represents the entire visual field, while traditional motion regions predominantly process contralateral motion. Our results suggest that there are two motion-processing systems in the human brain: a contralateral lower-level luminance-based system, extending from hMT/V5+ into dorsal IPS and STS, and a bilateral higher-level saliency-based system in IPL.

Normal and anomalous development of visual motion processing: motion coherence and 'dorsal-stream vulnerability'

Directional motion processing is a pervasive and functionally important feature of the visual system. Behavioural and VEP studies indicate that it appears as a cortical function after about 7 weeks of age, with global processing, motion based segmentation, and the use of motion in complex perceptual tasks emerging shortly afterwards. A distinct, subcortical motion system controls optokinetic nystagmus (OKN) from birth, showing characteristic monocular asymmetries which disappear as binocular cortical function takes over in normal development. Asymmetries in cortical responses are linked to this interaction in a way that is not yet fully understood. Beyond infancy, a range of developmental disorders show a deficit of global motion compared to global form processing which we argue reflects a general 'dorsal-stream vulnerability'.

Neuroimaging of direction-selective mechanisms for second-order motion

Psychophysical findings have revealed a functional segregation of processing for 1st-order motion (movement of luminance modulation) and 2nd-order motion (e.g., movement of contrast modulation). However neural correlates of this psychophysical distinction remain controversial. To test for a corresponding anatomical segregation, we conducted a new functional magnetic resonance imaging (fMRI) study to localize direction-selective cortical mechanisms for 1st- and 2nd-order motion stimuli, by measuring direction-contingent response changes induced by motion adaptation, with deliberate control of attention. The 2nd-order motion stimulus generated direction-selective adaptation in a wide range of visual cortical areas, including areas V1, V2, V3, VP, V3A, V4v, and MT+. Moreover, the pattern of activity was similar to that obtained with 1st-order motion stimuli. Contrary to expectations from psychophysics, these results suggest that in the human visual cortex, the direction of 2nd-order motion is represented as early as V1. In addition, we found no obvious anatomical segregation in the neural substrates for 1st- and 2nd-order motion processing that can be resolved using standard fMRI.

Physiological evidence of interaction of first- and second-order motion processes in the human visual system: a magnetoencephalographic study

Humans have several mechanisms for the visual perception of motion, including one that is luminance-based (first-order) and another that is luminance-independent (second-order). Recent psychophysical studies have suggested that significant interaction occurs between these two neural processes. We investigated whether such interactions are represented as neural activity measured by magnetoencephalography (MEG). The second-order motion of a drifting sinusoidal grating, which is defined by the speed of the dot motion, did not generate a response. Apparent motion (AM) of the square area, defined by the speed of randomly moving dots, evoked a magnetic response whose latency and amplitude changed with the distance that the area moved (a second-order characteristic), though the response properties were significantly different from those for the first-order AM. AM, defined by both first- and second-order attributes, evoked an MEG response and the latencies and the amplitudes were distributed between those for the first- and second-order motions. The cortical source of the response was estimated to be around MT+. The results show a distinct difference in the neural processing of the second-order motion that cannot be explained by the difference in visibility, and they indicate that the interaction of the neural processes underlying first- and second-order motion detection occurs before the MEG response. Our study provides the first physiological evidence of a neural interaction between the two types of early motion detection.

Cerebral regions processing first- and higher-order motion in an opposed-direction discrimination task

Using PET, we studied the processing of different types of motion in an opposed-direction discrimination task. We used first-order motion and two types of higher-order motion (presented as moving gratings with stripes defined by flickering texture and kinetic boundaries, respectively). In these experiments, we found that all types of motion activate a common set of cortical regions when comparing a direction discrimination task to a detection of the dimming of the fixation point. This set includes left hV3A, bilateral hMT/V5+ and regions in the middle occipital gyrus, bilateral activations in the posterior and anterior parts of the intraparietal sulcus, bilateral precentral gyrus, medial frontal cortex and regions in the cerebellum. No significant differences were observed between different types of motion, even at low statistical thresholds. From this we conclude that, under our experimental conditions, the same cerebral regions are involved in the processing of first-order and higher-order motion in an opposed-direction discrimination task.

Sensitivity to contrast modulation: the spatial frequency dependence of second-order vision

We consider the overall shape of the second-order modulation sensitivity function (MSF). Because second-order modulations of local contrast or orientation require a carrier signal, it is necessary to evaluate modulation sensitivity against a variety of carriers before reaching a general conclusion about second-order sensitivity. Here we present second-order sensitivity functions for new carrier types (low pass (1/f) noise, and high pass noise) and demonstrate that, when first-order artefacts have been accounted for, the shape of the resulting MSFs are similar to one another and to those for white and broad band noise. They are all low pass with a likely upper frequency limit in the range 10-20 c/deg, suggesting that detection of second-order stimuli is relatively insensitive to the structure of the carrier signal. This result contrasts strongly with that found for (first-order) luminance modulations of the same noise types. Here the noise acts as mask and each noise type masks most those frequencies that are dominant in its spectrum. Thus the shape of second-order MSFs are largely independent of the spectrum of their noise carrier, but first-order CSFs depend on the spectrum of an additive noise mask. This provides further evidence for the separation of first- and second-order vision and characterises second-order vision as a low pass mechanism.

Processing of second-order stimuli in the visual cortex

Naturally occurring visual stimuli are rich in examples of objects delineated from their backgrounds simply by differences in luminance, so-called first-order stimuli, as well as those defined by differences of contrast or texture, referred to as second-order stimuli. Here we provide a brief overview of visual cortical processing of second-order stimuli, as well as some comparative background on first-order processing, concentrating on single-unit neurophysiology, but also discussing relationships to human psychophysics and to neuroimaging. The selectivity of visual cortical neurons to orientation, spatial frequency, and direction of movement of first-order, luminance-defined stimuli is conventionally understood in terms of simple linear filter models, albeit with some minor nonlinearities such as thresholding and gain control. However, these kinds of models fail entirely to account for responses of neurons to second-order stimuli such as contrast envelopes, illusory contours, or texture borders. Second-order stimuli constructed from sinusoidal components have been used to analyze the neurophysiological mechanisms of such responses; these experiments demonstrate that the same neuron can exhibit three distinct kinds of tuning to spatial frequency, and also to orientation. These results can be understood in terms of a type of nonlinear 'filter-->rectify-->filter' model, which has been widely used in human psychophysics. Finally, several general issues will be discussed, including potential artifacts in experiments with second-order stimuli, and strategies for avoiding or controlling for them; caveats about definitions of first- vs. second-order mechanisms and stimuli; the concept of form-cue invariance; and the functional significance of second-order processing.

A new approach to analysing texture-defined motion

It has been widely accepted that standard low-level computational approaches to motion processing cannot extract texture-defined motion without applying some pre-processing nonlinearity. This has motivated accounts of motion perception in which luminance- and texture-defined motion are processed by separate mechanisms. Here, we introduce a novel method of image description where motion sequences may be described in terms of their local spatial and temporal gradients. This allows us to assess the local velocity information available to standard low-level motion mechanisms. Our analysis of several texture-motion stimuli shows that the information indicating correct texture-motion velocity and/or direction is present in the raw luminance measures. This raises the possibility that luminance-motion and texture-motion may be processed by the same cortical mechanisms. Our analysis offers a way of looking at texture-motion processing that is, to our knowledge, new and original.

Neuronal basis of the motion aftereffect reconsidered

Several fMRI studies have reported MT+ response increases correlated with perception of the motion aftereffect (MAE). However, attention can strongly affect MT+ responses, and subjects may naturally attend more to the MAE than control trials without MAE. We found that requiring subjects to attend to motion on both MAE and control trials produced equal levels of MT+ response, suggesting that attention may have confounded the interpretation of previous experiments; in our data, attention accounts for the entire effect. After eliminating this confound, we observed that direction-selective motion adaptation produced a direction-selective imbalance in MT+ responses (and earlier visual areas), and yielded a corresponding asymmetry in speed discrimination thresholds. These findings provide physiological evidence that population level response imbalances underlie the MAE, and quantify the relative proportions of direction-selective neurons across human visual areas.

Centrifugal bias for second-order but not first-order motion

Limited-lifetime Gabor stimuli were used to assess both first- and second-order motion in peripheral vision. Both first- and second-order motion mechanisms were present at a 20-deg eccentricity. Second-order motion, unlike first-order, exhibits a bias for centrifugal motion, suggesting a role for the second-order mechanism in optic flow processing.

Three-systems theory of human visual motion perception: review and update

Lu and Sperling [Vision Res. 35, 2697 (1995)] proposed that human visual motion perception is served by three separate motion systems: a first-order system that responds to moving luminance patterns, a second-order system that responds to moving modulations of feature types-stimuli in which the expected luminance is the same everywhere but an area of higher contrast or of flicker moves, and a third-order system that computes the motion of marked locations in a "salience map," that is, a neural representation of visual space in which the locations of important visual features ("figure") are marked and "ground" is unmarked. Subsequently, there have been some strongly confirmatory reports: different gain-control mechanisms for first- and second-order motion, selective impairment of first- versus second- and/or third-order motion by different brain injuries, and the classification of new third-order motions, e.g., isoluminant chromatic motion. Various procedures have successfully discriminated between second- and third-order motion (when first-order motion is excluded): dual tasks, second-order reversed phi, motion competition, and selective adaptation. Meanwhile, eight apparent contradictions to the three-systems theory have been proposed. A review and reanalysis here of the new evidence, pro and con, resolves the challenges and yields a more clearly defined and significantly strengthened theory.

Processing of second-order motion stimuli in primate middle temporal area and medial superior temporal area

Two rhesus monkeys were subjects in a direction-discrimination task involving moving stimuli defined by either first- or second-order motion. Two different second-order motion stimuli were used: drift-balanced motion consisting of a rectangular field of stationary dots and theta motion consisting of the same rectangular field with dots moving in the direction opposite to that of the object. The two types of stimuli involved different segmentation cues between the moving object and the background: temporal structure of the luminance (flicker) in the case of drift-balanced motion and opposed motion in the case of the theta-motion stimulus. Our monkeys were able to correctly report the direction of each stimulus. Single-unit recordings from the middle temporal (MT) and medial superior temporal (MST) areas revealed that 16 out of 38 neurons (41%) from area MT and 34 out of 68 neurons (50%) from area MST responded in a directionally selective manner to the drift-balanced stimulus. The movement of an object defined by theta motion is not explicitly encoded in the neuronal activity in areas MT or MST. Our results do not support the hypothesis that the neuronal activity in these areas codes for the direction of stimulus movement independent of specific stimulus parameters. Furthermore, our results emphasize the relevance of different segmentation cues between figure and background. Therefore the notion that there are multiple sites responsible for the processing of second-order motion is strongly supported.

A transient deficit of motion perception in human

We studied the motion perception abilities in a young adult, SF, who had her right occipito-temporal cortices resected to treat epilepsy. Following resection, SF showed transient deficits of both first- and second-order motion perception that recovered to normal within weeks. Previous human studies have shown either first- or second n order motion deficits that have lasted months or years after cerebral damage. SF also showed a transient defect in processing of shape-from-motion with normal perception of shape from non-motion cues. Furthermore, she showed greatly increased reaction times for a mental rotation task, but not for a lexical decision task. The nature and quick recovery of the deficits in SF resembles the transient motion perception deficit observed in monkey following ibotenic acid lesions, and provides additional evidence that humans possess specialized cortical areas subserving similar motion perception functions.

Computational modelling of interleaved first- and second-order motion sequences and translating 3f+4f beat patterns

Despite detailed psychophysical, neurophysiological and electrophysiological investigation, the number and nature of independent and parallel motion processing mechanisms in the visual cortex remains controversial. Here we use computational modelling to evaluate evidence from two psychophysical studies collectively thought to demonstrate the existence of three separate and independent motion processing channels. We show that the pattern of psychophysical results can largely be accounted for by a single mechanism. The results demonstrate that a low-level luminance based approach can potentially provide a wider account of human motion processing than generally thought possible.

A lesion of cortical area V2 selectively impairs the perception of the direction of first-order visual motion

Lesions of area MT/V5 in monkeys and its presumed homologue, the motion area, in humans impair motion perception, including the discrimination of the direction of global motion in random dot kinematograms. Here we report the results of similar tests on patient TF, who has a discrete and very small, unilateral infarct in the medial superior part of the right occipital cortex. Structural MRI, co-registered in software with a standardized human brain atlas, reveals that the lesion involves area V2. The patient was impaired in his retinotopically corresponding left lower quadrant on several motion tasks including discrimination in random dot kinematograms of direction, speed and motion-defined discontinuity. He was also impaired on tasks selectively involving first-order motion based on luminance contrast but not on second-order motion based on texture contrast. The results show that even though area MT/V5 is intact, motion perception is abnormal and, in particular, his perception of first-order motion is impaired.