Motion opponency in visual cortex

Comparing the Effects of Auditory Deprivation and Sign Language within the Auditory and Visual Cortex

To investigate neural plasticity resulting from early auditory deprivation and use of American Sign Language, we measured responses to visual stimuli in deaf signers, hearing signers, and hearing nonsigners using functional magnetic resonance imaging. We examined “compensatory hypertrophy” (changes in the responsivity/size of visual cortical areas) and “cross-modal plasticity” (changes in auditory cortex responses to visual stimuli). We measured the volume of early visual areas (V1, V2, V3, V4, and MT+). We also measured the amplitude of responses within these areas, and within the auditory cortex, to a peripheral visual motion stimulus that was attended or ignored. We found no major differences between deaf and hearing subjects in the size or responsivity of early visual areas. In contrast, within the auditory cortex, motion stimuli evoked significant responses in deaf subjects, but not in hearing subjects, in a region of the right auditory cortex corresponding to Brodmann's areas 41, 42, and 22. This hemispheric selectivity may be due to a predisposition for the right auditory cortex to process motion; earlier studies report a right hemisphere bias for auditory motion in hearing subjects. Visual responses within the auditory cortex of deaf subjects were stronger for attended than ignored stimuli, suggesting top-down processes. Hearing signers did not show visual responses in the auditory cortex, indicating that cross-modal plasticity can be attributed to auditory deprivation rather than sign language experience. The largest effects of auditory deprivation occurred within the auditory cortex rather than the visual cortex, suggesting that the absence of normal input is necessary for large-scale cortical reorganization to occur.

Chromatic sensitivity of neurones in area MT of the anaesthetised macaque monkey compared to human motion perception

We recorded activity from neurones in cortical motion-processing areas, middle temporal area (MT) and middle posterior superior temporal sulcus (MST), of anaesthetised and paralysed macaque monkeys in response to moving sinewave gratings modulated in luminance and chrominance. The activity of MT and MST neurones was highly dependent on luminance contrast. In three of four animals isoluminant chromatic modulations failed to activate MT/MST neurones significantly. At low luminance contrast a systematic dependence on chromaticity was revealed, attributable mostly to residual activity of the magnocellular pathway. Additionally, we found indications for a weak S-cone input, but rod intrusion could also have made a contribution. In contrast to the activity of MT and MST neurones, speed judgments and onset amplitude of evoked optokinetic eye movements in human subjects confronted with equivalent visual stimuli were largely independent of luminance modulation. Motion of every grating (including isoluminant) was readily visible for all but one observer. Similarity with the activity of MT/MST cells was found only for motion-nulling equivalent luminance contrast judgments at isoluminance. Our results suggest that areas MT and MST may not be involved in the processing of chromatic motion, but effects of central anaesthesia and/or the existence of intra- and inter-species differences must also be considered.

Methodological considerations in rat brain BOLD contrast pharmacological MRI

RATIONALE AND OBJECTIVES

Blood oxygen level dependent (BOLD) contrast pharmacological magnetic resonance imaging (phMRI) is an increasingly popular technique that allows the non-invasive investigation of spatial and temporal changes in rat brain function in response to pharmacological stimulation in vivo. Rat brain BOLD contrast phMRI is, at present, established in few neuropharmacological laboratories, and various issues associated with the technique require attention. The present review is primarily aimed at psychopharmacologists with no previous experience of phMRI, who are interested in the practical aspects that phMRI studies entail.

RESULTS AND DISCUSSION

Experimental and analytical considerations, including anaesthesia, physiological monitoring, drug dose and delivery, scanning protocols, statistical approaches and the interpretation of phMRI data, are discussed.

Relationship between steady-state and induced gamma activity to motion

When a moving stimulus is presented at a specific temporal frequency, both steady-state responses and induced gamma activity may be elicited in the electroencephalogram. The electroencephalogram was recorded when study participants viewed random dot kinematograms under three conditions: coherent motion, incoherent motion and stationary. Dot position was changed at a rate of 9.3 Hz in the coherent and incoherent conditions. Induced power at 40 Hz was increased during coherent motion compared with the other conditions. In contrast, the steady-state response at 9.3 Hz showed a trend for increased power during the incoherent condition. These results suggest that steady-state responses to moving stimuli reflect sensory activation, while the induced gamma activity indexes perceptual processes.

Opponent-motion mechanisms are self-normalizing

In the ultimate stage of the Adelson-Bergen motion energy model [Adelson, E. H., & Bergen, J. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America, 2, 284-299], motion is derived from the difference between directionally opponent energies E(L) and E(R). However, Georgeson and Scott-Samuel [Georgeson, M. A., & Scott-Samuel, N. E. (1999). Motion contrast: A new metric for direction discrimination. Vision Research, 39, 4393-4402] demonstrated that motion contrast-a metric that normalizes opponent motion energy (E(L)-E(R)) by flicker energy (E(L)+E(R))-is a better descriptor of human direction discrimination. In a previous study [Rainville, S. J. M., Makous, W. L., & Scott-Samuel, N. E. (2002). The spatial properties of opponent-motion normalization. Vision Research, 42, 1727-1738], we used a lateral masking paradigm to show that opponent-motion normalization is selective for flicker position, orientation, and spatial-frequency. In the present study, we used a superposition masking paradigm and compared results to lateral masking data, as the two masking types activate local and remote normalization mechanisms differentially. Although selectivity for flicker orientation and spatial frequency varied across observers, bandwidths were similar across lateral and superimposed masking conditions. Additional experiments demonstrated that normalization signals are pooled over a spatial region whose aspect ratio and size are consistent with those of local motion detectors. Together, results show no evidence of remote normalization signals predicted by broadband inhibitory models [(e.g.) Heeger, D. J. (1992). Normalization of cell responses in cat striate cortex. Visual Neuroscience, 9, 181-197; Foley, J. M. (1994). Human luminance pattern-vision mechanisms: Masking experiments require a new model. Journal of the Optical Society of America A-Optics and Image Science, 11, 1710-1719] but support a local normalization process whose spatial properties are inherited from low-level motion detectors.

Binding of motion and colour is early and automatic

At what stages of the human visual hierarchy different features are bound together, and whether this binding requires attention, is still highly debated. We used a colour-contingent motion after-effect (CCMAE) to study the binding of colour and motion signals. The logic of our approach was as follows: if CCMAEs can be evoked by targeted adaptation of early motion processing stages, without allowing for feedback from higher motion integration stages, then this would support our hypothesis that colour and motion are bound automatically on the basis of spatiotemporally local information. Our results show for the first time that CCMAE's can be evoked by adaptation to a locally paired opposite-motion dot display, a stimulus that, importantly, is known to trigger direction-specific responses in the primary visual cortex yet results in strong inhibition of the directional responses in area MT of macaques as well as in area MT+ in humans and, indeed, is perceived only as motionless flicker. The magnitude of the CCMAE in the locally paired condition was not significantly different from control conditions where the different directions were spatiotemporally separated (i.e. not locally paired) and therefore perceived as two moving fields. These findings provide evidence that adaptation at an early, local motion stage, and only adaptation at this stage, underlies this CCMAE, which in turn implies that spatiotemporally coincident colour and motion signals are bound automatically, most probably as early as cortical area V1, even when the association between colour and motion is perceptually inaccessible.

Adaptation to spiral motion: global but not local motion detectors are modulated by attention

In this study, we investigated the effect of attention on local motion detectors. For this purpose we used logarithmic spirals previously used by Cavanagh and Favreau [Perception, 1980, 9(2), 175-182]. While the adapting stimulus was a rotating logarithmic spiral, the test stimulus was either the same spiral or its mirror image. When superimposed, all contours of the spiral stimulus and its mirror image are 90 degrees apart. Presenting the same spiral during the test period shows adaptation of both local motion detectors and global rotation detectors, whereas showing the mirror-spiral stimulates another set of local motion detectors, and therefore illustrates adaptation at only the global motion level. To manipulate the attentional state of observers, a secondary task was presented during the adaptation phase and observers either performed the task or ignored it. Motion aftereffect (MAE) duration was measured afterwards. While the effects of attention and test stimulus type on MAE duration were both significant, the difference in the MAE strength between the attention-distracted and attention-not-distracted conditions was equal when the test stimulus was the same-spiral or the mirror-spiral, suggesting that attention to spiral motion modulates only global rotation units and does not affect local motion detectors located at V1. Our results are in accord with those reported by Watanabe et al. [Proceedings of the National Academy of Sciences of the USA, 1998, 95(19), 11489-11492] which showed differential modulation of motion processing areas depending on the type of motion being attended. Therefore our data are supportive of the notion that attentional modulation of V1 is highly task-dependent.

Visual aftereffects: cortical neurons change their tune

Recent studies of areas V1 and MT in the visual cortex show that exposure to a stimulus can change the contrast sensitivity of cells and shift their peak sensitivity to a new orientation or movement direction. In MT, these shifts can correctly predict illusory changes - visual aftereffects - in movement direction, but in V1, they are more difficult to interpret.

The motion aftereffect of transparent motion: Two temporal channels account for perceived direction

Adaptation to orthogonal transparent patterns drifting at the same speed produces a unidirectional motion aftereffect (MAE) whose direction is opposite the average adaptation direction. If the patterns move at different speeds, MAE direction can be predicted by an inverse vector average, using the observer's motion sensitivity to each individual pattern as vector magnitudes. These weights are well approximated by the duration of each pattern's MAE, as measured with static test patterns. However, previous efforts to use the inverse-vector-average rule with dynamic test patterns have failed. Generally, these studies have used spatially and temporally broadband test stimuli. Here, in order to gain insight into the possible contribution of temporal channels, we filtered our test pattern in the temporal domain to produce five ideal, octave-width pass-bands. MAE durations were measured for single-component stimuli drifting at various adaptation speeds and tested at a range of temporal frequencies. Then, two components with orthogonal directions and different speeds were combined and the direction of the resulting MAE was measured. The key findings are that: (i) for a given adaptation speed, the duration of a single component's MAE is dependent on test temporal frequency; (ii) the direction of MAEs produced by transparent motion (i.e., bivectorial adaptation) also varies strongly as a function test temporal frequency (by up to 90 degrees for some speed pairings); and (iii) the inverse-vector-average rule predicts the direction of the transparent MAE provided the MAE durations used to weight the vector combination were obtained from stimuli matched in adaptation speed and test temporal frequency. These results are discussed in terms of the number and shape of temporal channels in our visual system.

Two neural correlates of consciousness

Neuroscientists continue to search for "the" neural correlate of consciousness (NCC). In this article, I argue that a framework in which there are at least two distinct NCCs is increasingly making more sense of empirical results than one in which there is a single NCC. I outline the distinction between phenomenal NCC and access NCC, and show how they can be distinguished by experimental approaches, in particular signal-detection theory approaches. Recent findings in cognitive neuroscience provide an empirical case for two different NCCs.

Activation in the MT-complex during visual perception of apparent motion and temporal succession

Previous studies have shown that MT (i.e., the MT-complex) is activated during visual perception of apparent motion. To further explore the function of MT, we measured activation in MT by positron emission tomography (PET) using a broad range of stroboscopic stimulus events in which (a) the frame rate was so fast that observers perceived stimulus frames as simultaneous, (b) the frame rate was slower and generated compelling impressions of apparent motion, or (c) the frame rate was so slow that observers perceived temporal succession (successive views of the same objects at different locations) instead of motion. As expected, the simultaneity condition showed no activation (reliable increase in regional cerebral blood flow, rCBF) in MT whereas the motion condition showed activation in both left and right MT. However, the succession condition showed even stronger activation in left and right MT than did the motion condition. MT seems implicated in perception of retinal stimuli as successive views of the same object at different locations whether or not the views are connected by apparent motion.

Motion mechanisms in macaque MT

The macaque middle temporal area (MT) is exquisitely sensitive to visual motion and there is a large amount of evidence that neural activity in MT is tightly correlated with the perception of motion. The mechanisms by which MT neurons achieve their directional selectivity, however, have received considerably less attention. We investigated the motion-energy model as a description of motion mechanisms in macaque MT. We first confirmed one of the predictions of the motion-energy model; macaques-just like humans-perceive a reversed direction of motion when a stimulus reverses contrast with every displacement (reverse-phi). This reversal of perceived direction had a clear correlate in the neural responses of MT cells, which were predictive of the monkey's behavioral decisions. Second, we investigated how multiple motion-energy components are combined. Psychophysical data have been used to argue that motion-energy components representing opposite directions are subtracted from each other. Our data show, however, that the interactions among motion-energy components are more complex. In particular, we found that the influence of a given component on the response to a stimulus consisting of multiple components depends on factors other than the response to that component alone. This suggests that there are subthreshold nonlinear interactions among multiple motion-energy components; these could take place within MT or in earlier stages of the motion network such as V1. We propose a model that captures the complexity of these component interactions by means of a competitive interaction among the components. This provides a better description of the MT responses than the subtractive motion opponency envisaged in the motion-energy model, even when the latter is combined with a gain-control mechanism. The competitive interaction extends the dynamic range of the cells and allows them to provide information on more subtle changes in motion patterns, including changes that are not purely directional.

Effect of adaptation direction on the motion VEP and perceived speed of drifting gratings

The N200 amplitude of the motion-onset VEP evoked by a parafoveal grating of variable contrast (0.5-64%), constant speed (2 degrees/s), direction (horizontally rightward), and spatial frequency (2 cpd) was studied before and after adaptation to a stationary or drifting grating (1, 2, or 4 degrees/s rightward or leftward). These results are compared to those for the pattern-appearance VEP. Psychophysical measurements were made simultaneously of the perceived speed. While iso-directional (rightward) adaptation leads to a mean amplitude reduction of 39%, the decrease after counter-directional adaptation has a size of 20%. The post-adaptation matches of perceived speed differ in dependence on the iso-directional adapting speed and decrease on average to 98%, 85%, and 69% of the pre-adapt perceived speed after 1, 2, and 4 degrees/s adapting speeds, respectively. The perceived speed is moderately reduced (83% of the pre-adapt value) after counter-directional adaptation nearly independently of the adapting speed. A model of velocity processing is presented, which enables us to predict the trends of the experimental motion VEP and perceived speed data.

The temporal frequency tuning of human visual cortex investigated using synthetic aperture magnetometry

Using synthetic aperture magnetometry (SAM) analyses of magnetoencephalographic (MEG) data, we investigated the variation in cortical response magnitude and frequency as a function of stimulus temporal frequency. In two separate experiments, a reversing checkerboard stimulus was used in the right or left lower visual field at frequencies from 0 to 21 Hz. Average temporal frequency tuning curves were constructed for regions-of-interest located within medial visual cortex and V5/MT. In medial visual cortex, it was found that both the frequency and magnitude of the steady-state response varied as a function of the stimulus frequency, with multiple harmonics of the stimulus frequency being found in the response. The maximum fundamental response was found at a stimulus frequency of 8 Hz, whilst the maximum broadband response occurred at 4 Hz. In contrast, the magnitude and frequency content of the evoked onset response showed no dependency on stimulus frequency. Whilst medial visual cortex showed a power increase during stimulation, extra-striate areas such as V5/MT exhibited a bilateral event-related desynchronisation (ERD). The frequency content of this ERD did not depend on the stimulus frequency but was a broadband power reduction across the 5-20 Hz frequency range. The magnitude of this ERD within V5/MT was strongly low-pass tuned for stimulus frequency, and showed only a moderate preference for stimuli in the contralateral visual field.

Object-based cross-feature attentional modulation from color to motion

Object-based theories of visual attention predict that attempting to direct attention to a particular attribute of a visual object will result in an automatic selection of the whole object, including all of its features. It has been assumed, but not critically tested, that the spreading of attention from one feature to another in this manner, i.e. cross-feature attentional (CFA) effects, takes place at object-level stages of processing as opposed to early, local stages. In the present study we disambiguated these options for color-to-motion CFA by contrasting attention's effect on bivectorial transparent versus bivectorial locally paired motion displays. We found that association between features at the global, but not at the local, stage of motion processing leads to cross-feature attentional effects. These findings provide strong psychophysical evidence that such effects are indeed object-based.

Psilocybin impairs high-level but not low-level motion perception

The hallucinogenic serotonin(1A&2A) agonist psilocybin is known for its ability to induce illusions of motion in otherwise stationary objects or textured surfaces. This study investigated the effect of psilocybin on local and global motion processing in nine human volunteers. Using a forced choice direction of motion discrimination task we show that psilocybin selectively impairs coherence sensitivity for random dot patterns, likely mediated by high-level global motion detectors, but not contrast sensitivity for drifting gratings, believed to be mediated by low-level detectors. These results are in line with those observed within schizophrenic populations and are discussed in respect to the proposition that psilocybin may provide a model to investigate clinical psychosis and the pharmacological underpinnings of visual perception in normal populations.

Learning motion discrimination with suppressed MT

We studied perceptual learning in motion discrimination when the brain's middle temporal area (MT/V5) was functionally suppressed. This was achieved by using the "paired-dots" motion stimulus where the two dots in a pair always move in counter-phase over a short distance [J. Neurosci. 14 (1994) 7357]. The motion directional signal of the stimulus is therefore always 0 on average. As a result, this stimulus activates MT in Rhesus monkeys no more than flicker noise does [J. Neurosci. 14 (1994) 7367]. We added a new manipulation to eliminate the Glass pattern in the original stimulus that would have otherwise provided a static orientation cue. Two such new motion stimuli were presented sequentially, in a 2AFC task. Subjects decided if the global motion-axis of the stimuli changed clockwise or counter-clockwise. When the task difficulty was set at 60% correct, none of the subjects could learn with feedback, even though their performance was well above chance. However, when the task difficulty was set instead at 70% correct, a new group of subjects was able to learn. Hence, learning motion discrimination was possible when MT was presumably eliminated.

Stereoscopic processing of absolute and relative disparity in human visual cortex

Stereoscopic vision relies mainly on relative depth differences between objects rather than on their absolute distance in depth from where the eyes fixate. However, relative disparities are computed from absolute disparities, and it is not known where these two stages are represented in the human brain. Using functional MRI (fMRI), we assessed absolute and relative disparity selectivity with stereoscopic stimuli consisting of pairs of transparent planes in depth in which the absolute and relative disparity signals could be independently manipulated (at a local spatial scale). In experiment 1, relative disparity was kept constant, while absolute disparity was varied in one-half the blocks of trials ("mixed" blocks) and kept constant in the remaining one-half ("same" blocks), alternating between blocks. Because neuronal responses undergo adaptation and reduce their firing rate following repeated presentation of an effective stimulus, the fMRI signal reflecting activity of units selective for absolute disparity is expected to be smaller during "same" blocks as compared with "mixed" ones. Experiment 2 similarly manipulated relative disparity rather than absolute disparity. The results from both experiments were consistent with adaptation with differential effects across visual areas such that 1) dorsal areas (V3A, MT+/V5, V7) showed more adaptation to absolute than to relative disparity; 2) ventral areas (hV4, V8/V4alpha) showed an equal adaptation to both; and 3) early visual areas (V1, V2, V3) showed a small effect in both experiments. These results indicate that processing in dorsal areas may rely mostly on information about absolute disparities, while ventral areas split neural resources between the two types of stereoscopic information so as to maintain an important representation of relative disparity.

Noninvasive measurement of neuronal activity with near-infrared optical imaging

Diffuse optical imaging (DOI) alone offers the possibility of simultaneously and noninvasively measuring neuronal and vascular signals in the brain with temporal resolution of up to 1 ms. However, while optical measurement of hemodynamic signals is well established, optical measurement of neuronal activation (the so-called fast signal) is just emerging and requires further optimization and validation. In this work, we present preliminary studies in which we measured the fast signal in 10 healthy volunteers during finger-tapping, tactile stimulation, and electrical median nerve stimulation. We used an instrument (CW4) with 8 source (690 and 830 nm) and 16 detector positions-more optodes than the instruments in previously reported studies. This allowed us to record the ipsilateral and contralateral sensorimotor cortex simultaneously, while at the same time measuring the evoked hemodynamic response. We used an acquisition time of 25 ms per image; after averaging approximately 1000 events, the signal-to-noise ratio was approximately 10(4). Since the expected relative intensity changes due to the fast signal (approximately 10(-3)) are smaller than the relative intensity changes due to physiological effects (approximately 10(-1)), we enhanced the suppression of competing signals such as the heartbeat-associated intensity changes, and established five criteria with which to assess the robustness of the fast signal. We detected the fast signal in 43% of the measurements during finger-tapping, 60% of those during tactile stimulation, and 23% of those during electrical median nerve stimulation. The relative changes in intensity associated with the fast signal were approximately 0.07% and the latency of the signal was approximately 100 ms.

The efficiency of speed discrimination for coherent and transparent motion

Transparent motion involves the integration and segmentation of local motion signals. Previous research found a cost for processing transparent random dot motions relative to single coherent motions. However, this cost can be the result of the increased complexity of the transparent stimuli. We investigated this possibility by measuring the efficiency of transparent and coherent motions. Since efficiency normalises human performance to that of an ideal observer in the same task, performance can be compared fairly across tasks. Our task, identical in both transparent and coherent conditions, was to discriminate the fastest speed between two opposite motion directions. In two experiments where we varied dot density and speed, we confirmed the cost in human sensitivity for transparent motion but also found a cost for the ideal observer. The outcome was a consistent residual cost in efficiency for transparent motion. This result points to a processing limitation for transparent motion analogous to previously suggested inhibitory mechanisms between opposite directions of motion. Furthermore, we found that both transparent and coherent motion efficiencies decreased as dot density increased. This latter result stresses the importance of the correspondence problem and suggests that local motion signals are integrated over large areas.

Neuroimaging of direction-selective mechanisms for second-order motion

Psychophysical findings have revealed a functional segregation of processing for 1st-order motion (movement of luminance modulation) and 2nd-order motion (e.g., movement of contrast modulation). However neural correlates of this psychophysical distinction remain controversial. To test for a corresponding anatomical segregation, we conducted a new functional magnetic resonance imaging (fMRI) study to localize direction-selective cortical mechanisms for 1st- and 2nd-order motion stimuli, by measuring direction-contingent response changes induced by motion adaptation, with deliberate control of attention. The 2nd-order motion stimulus generated direction-selective adaptation in a wide range of visual cortical areas, including areas V1, V2, V3, VP, V3A, V4v, and MT+. Moreover, the pattern of activity was similar to that obtained with 1st-order motion stimuli. Contrary to expectations from psychophysics, these results suggest that in the human visual cortex, the direction of 2nd-order motion is represented as early as V1. In addition, we found no obvious anatomical segregation in the neural substrates for 1st- and 2nd-order motion processing that can be resolved using standard fMRI.

GABA uptake into astrocytes is not associated with significant metabolic cost: implications for brain imaging of inhibitory transmission

Synaptically released glutamate has been identified as a signal coupling excitatory neuronal activity to increased glucose utilization. The proposed mechanism of this coupling involves glutamate uptake into astrocytes resulting in increased intracellular Na+ (Nai+) and activation of the Na+/K+-ATPase. Increased metabolic demand linked to disruption of Nai+ homeostasis activates glucose uptake and glycolysis in astrocytes. Here, we have examined whether a similar neurometabolic coupling could operate for the inhibitory neurotransmitter gamma-aminobutyric acid (GABA), also taken up by Na+-dependent transporters into astrocytes. Thus, we have compared the Nai+ response to GABA and glutamate in mouse astrocytes by microspectrofluorimetry. The Nai+ response to GABA consisted of a rapid rise of 4-6 mM followed by a plateau that did not, however, significantly activate the pump. Indeed, the GABA transporter-evoked Na+ influxes are transient in nature, almost totally shutting off within approximately 30 sec of GABA application. The metabolic consequences of the GABA-induced Nai+ response were evaluated by monitoring cellular ATP changes indirectly in single cells and measuring 2-deoxyglucose uptake in astrocyte populations. Both approaches showed that, whereas glutamate induced a robust metabolic response in astrocytes (decreased ATP levels and glucose uptake stimulation), GABA did not cause any measurable metabolic response, consistent with the Nai+ measurements. Results indicate that GABA does not couple inhibitory neuronal activity with glucose utilization, as does glutamate for excitatory neurotransmission, and suggest that GABA-mediated synaptic transmission does not contribute directly to brain imaging signals based on deoxyglucose.

Human cortical response to incoherent motion on a background of coherent motion

To investigate whether humans achieve a high sensitivity to coherent motion by excluding the response to incoherent motion, we measured the magnetoencephalographic response to the motion of randomly located dots one half of which moved coherently while the other half moved incoherently. The response was related to the faster motion of either coherent or incoherent motion though the observers saw both. All the estimated response sources were within the extrastriate area. The results indicate that incoherent motion is represented in the neural activity of the human extrastriate area even when the coherent motion is perceived at the same time. The fact that the neural activity for the slower coherent motion is not represented in the magnetic response suggests the existence of interaction between the neural activities for the two motions.

Neuronal adaptation to visual motion in area MT of the macaque

The responsivity of primary sensory cortical neurons is reduced following prolonged adaptation, but such adaptation has been little studied in higher sensory areas. Adaptation to visual motion has strong perceptual effects, so we studied the effect of prolonged stimulation on neuronal responsivity in the macaque's area MT, a cortical area whose importance to visual motion perception is well established. We adapted MT neurons with sinusoidal gratings drifting in the preferred or null direction. Preferred adaptation reduced the responsiveness of MT cells, primarily by changing their contrast gain, and this effect was spatially specific within the receptive field. Null adaptation reduced the ability of null gratings to inhibit the response to a simultaneously presented preferred stimulus. While both preferred and null adaptation alter MT responses, these effects probably do not occur in MT neurons but are likely to reflect adaptation-induced changes in contrast gain earlier in the visual pathway.

Direction-selective motion blindness after unilateral posterior brain damage

Motion blindness (MB) is defined as the selective disturbance of visual motion perception despite intact perception of other features of the visual scene. MB is characterized by a pandirectional deficit of motion direction discrimination and is assumed to result from damage to the visual motion pathway, especially area MT/V5. However, the most characteristic feature of primate MT/V5 neurons is not their motion selectivity but their preference for one direction of motion (direction selectivity), which changes incrementally at neighbouring columns. In addition to this microscopic directional organization, studies in nonhuman and human primates suggest that single directions of motion are also coded at a more macroscopic level. We thus hypothesized that if MB in humans results from damage to direction-selective neurons in the visual motion pathway, posterior brain damage might cause MB which is direction selective, not pandirectional. The present study investigated motion direction discrimination in patients with posterior unilateral brain damage and determined separate psychophysical thresholds for the four cardinal directions. In addition, we analysed whether the direction of erroneous motion perception (i.e. the perception of right motion for upward motion) was random or showed a directional bias. We report three principal findings. First, motion direction discrimination was severely impaired in one or two directions while it was normal in the other directions. This constituted direction-selective MB. Second, MB was characterized not only by a quantitative direction-selective increase in psychophysical thresholds but also by a qualitative impairment of perceiving motion direction systematically in wrong directions. Both findings suggest that the cortical modules specialized for the perception of a single direction of motion might be larger than previously thought. Third, lesion analysis showed that unilateral damage, not only the human homologue of MT/V5 but also to parieto-occipital cortex, leads to MB.

Microstimulation of visual cortex affects the speed of perceptual decisions

Direction-selective neurons in the middle temporal visual area (MT) are crucially involved in motion perception, although it is not known exactly how the activity of these neurons is interpreted by the rest of the brain. Here we report that in a two-alternative task, the activity of MT neurons is interpreted as evidence for one direction and against the other. We measured the speed and accuracy of decisions as rhesus monkeys performed a direction-discrimination task. On half of the trials, we stimulated direction-selective neurons in area MT, thereby causing the monkeys to choose the neurons' preferred direction more often. Microstimulation quickened decisions in favor of the preferred direction and slowed decisions in favor of the opposite direction. Even on trials in which microstimulation did not induce a preferred direction choice, it still affected response times. Our findings suggest that during the formation of a decision, sensory evidence for competing propositions is compared and accumulates to a decision-making threshold.

Adaptation of response transients in fly motion vision. I: Experiments

Two types of transient responses have been investigated in fly motion-sensitive neurons in the past: the impulse and the step response. In response to a brief motion pulse, cells show a sudden rise in activity followed by an exponential decay ('impulse response'). In response to the onset of a constant velocity stimulus, cells exhibit transient oscillations before settling to a steady-state value ('step response'). Since the impulse response has been shown to shorten when tested after presentation of an adapting motion stimulus, we investigated whether adaptation also occurs during the step response. We tested this hypothesis by recording extracellularly the response of the H1-cell in the lobula plate of the blowfly Calliphora vicina to gratings of varying pattern contrasts and drift velocity. We found that the transient oscillations of the step response strongly depend on the pattern contrast: at low contrasts, oscillations lasted for several seconds, whereas at high contrasts, they settled within fractions of a second. This suggests that motion adaptation occurs during the initial period of the stimulus presentation and is dependent on the contrast of the motion stimulus. Using identical stimulus parameters (contrast and temporal frequency) for the adapting stimulus and testing the impulse response afterwards, we found that the impulse response and the transient period in the step response shortened in a similar way. We then analyzed the dynamic of the transients oscillations produced by ongoing motion of a square wave pattern in the anti-preferred direction (null direction) of H1. As observed for preferred direction motion, we found that the duration and amplitude of those transients shortened as the contrast and the velocity of the pattern increased, and that the oscillations disappeared when a blank screen instead of a pattern was presented before the onset of motion. Under both stimulus conditions, i.e. grating and blank screen before motion onset, the steady-state response level showed the same dependence on the contrast and temporal frequency of the pattern. When we analyzed the responses of the cell to pattern of various sizes and contrasts moving in the preferred direction of the cell, we found that increments in the size affected the overall amplitude of both the transient oscillations and the steady-state response level, whereas the duration of the oscillations only depended on the local pattern contrast. We also tested the impulse response before and after the presentation of an adapting stimulus presented in either the same or a different location of the visual field. The response shortened only when both the adapting and the test stimuli were presented at the same location. These last experiments demonstrate a strictly local mechanism of adaptation affecting the response transients of both the impulse and the step response.

Functional imaging of the visual pathways

Functional neuroimaging has provided a new view of activity in human visual cortex. There have been a series of interesting developments in understanding the relationship between the functional signals, particularly functional MRI, and basic measurements of action potentials and local field potentials. The new human neuro-imaging measurements have clarified some of the similarities and differences between the general organization of visual areas in human and macaque visual cortex, and there have been some interesting new results concerning cortical visual plasticity and dysfunction. The new fMRI focus on measurements of the human brain will drive new relationships between neurology and visual neuroscience that should help us learn much more about the neural basis of perception.

Greater plasticity in lower-level than higher-level visual motion processing in a passive perceptual learning task

Simple exposure is sufficient to sensitize the human visual system to a particular direction of motion, but the underlying mechanisms of this process are unclear. Here, in a passive perceptual learning task, we found that exposure to task-irrelevant motion improved sensitivity to the local motion directions within the stimulus, which are processed at low levels of the visual system. In contrast, task-irrelevant motion had no effect on sensitivity to the global motion direction, which is processed at higher levels. The improvement persisted for at least several months. These results indicate that when attentional influence is limited, lower-level motion processing is more receptive to long-term modification than higher-level motion processing in the visual cortex.

Heschl's gyrus is more sensitive to tone level than non-primary auditory cortex

Previous neuroimaging studies generally demonstrate a growth in the cortical response with an increase in sound level. However, the details of the shape and topographic location of such growth remain largely unknown. One limiting methodological factor has been the relatively sparse sampling of sound intensities. Additionally, most studies have either analysed the entire auditory cortex without differentiating primary and non-primary regions or have limited their analyses to Heschl's gyrus (HG). Here, we characterise the pattern of responses to a 300-Hz tone presented in 6-dB steps from 42 to 96 dB sound pressure level as a function of its sound level, within three anatomically defined auditory areas; the primary area, on HG, and two non-primary areas, consisting of a small area lateral to the axis of HG (the anterior lateral area, ALA) and the posterior part of auditory cortex (the planum temporale, PT). Extent and magnitude of auditory activation increased non-linearly with sound level. In HG, the extent and magnitude were more sensitive to increasing level than in ALA and PT. Thus, HG appears to have a larger involvement in sound-level processing than does ALA or PT.

Multiple uses of visual motion. The case for stability in sensory cortex

The problem of 'readout' from sensory maps has received considerable attention recently. Specifically, many experiments in different systems have suggested that the routing of sensory signals from cortical maps can be impressively flexible. In this review, we discuss many of the experiments addressing readout of motion signals from the middle temporal area (also known as V5) in the macaque monkey. We focus on two different types of output: perceptual reports (categorical decisions, usually) and motion-guided eye movements. We specifically consider situations in which multiple-motion vectors present in the stimulus are combined, as well as those in which one or more of the vectors in the stimulus is selected for output. The results of these studies suggest that in some situations multiple motions are vector averaged, while in others multiple vectors can be maintained. Interestingly, in most of the experiments producing a single (often average) vector, the output is a movement. However, many perceptual experiments involve the simultaneous processing of multiple-stimulus motions. One prosaic explanation for this pattern of apparently discrepant results is that different downstream structures impose different rules, in parallel, on the output from sensory maps such as the one in the middle temporal area. We also specifically discuss the case of motion opponency, a specific readout rule that has been posited to explain perceptual phenomena such as the waterfall illusion (motion aftereffect). We present evidence from a recent experiment showing that an opponent step must occur downstream from the middle temporal area itself. This observation is consistent with our proposal that significant processing need occur downstream from sensory structures. If a single output is to be used for multiple purposes, often at once, this necessitates a degree of task invariance on the sensory information present even at a relatively high level of cortical processing.

Global effects of feature-based attention in human visual cortex

The content of visual experience depends on how selective attention is distributed in the visual field. We used functional magnetic resonance imaging (fMRI) in humans to test whether feature-based attention can globally influence visual cortical responses to stimuli outside the attended location. Attention to a stimulus feature (color or direction of motion) increased the response of cortical visual areas to a spatially distant, ignored stimulus that shared the same feature.

Contrast sensitivity in human visual areas and its relationship to object recognition

An important characteristic of visual perception is the fact that object recognition is largely immune to changes in viewing conditions. This invariance is obtained within a sequence of ventral stream visual areas beginning in area V1 and ending in high order occipito-temporal object areas (the lateral occipital complex, LOC). Here we studied whether this transformation could be observed in the contrast response of these areas. Subjects were presented with line drawings of common objects and faces in five different contrast levels (0, 4, 6, 10, and 100%). Our results show that indeed there was a gradual trend of increasing contrast invariance moving from area V1, which manifested high sensitivity to contrast changes, to the LOC, which showed a significantly higher degree of invariance at suprathreshold contrasts (from 10 to 100%). The trend toward increased invariance could be observed for both face and object images; however, it was more complete for the face images, while object images still manifested substantial sensitivity to contrast changes. Control experiments ruled out the involvement of attention effects or hemodynamic "ceiling" in producing the contrast invariance. The transition from V1 to LOC was gradual with areas along the ventral stream becoming increasingly contrast-invariant. These results further stress the hierarchical and gradual nature of the transition from early retinotopic areas to high order ones, in the build-up of abstract object representations.

The spatial properties of opponent-motion normalization

The final stage of the Adelson-Bergen model [J. Opt. Soc. Am. A 2 (1985) 284] computes net motion as the difference between directionally opposite energies E(L) and E(R). However, Georgeson and Scott-Samuel [Vis. Res. 39 (1999) 4393] found that human direction discrimination is better described by motion contrast (C(m))--a metric where opponent energy (E(L)-E(R)) is divided by flicker energy (E(L)+E(R)). In the present paper, we used a lateral masking paradigm to investigate the spatial properties of flicker energy involved in the normalization of opponent energy. Observers discriminated between left and right motion while viewing a checkerboard in which half of the checks contained a drifting sinusoid and the other half contained flicker (i.e. a counterphasing sinusoid). The relative luminance contrasts of flicker and motion checks determined the checkerboard's overall motion contrast C(m). We obtained selectivity functions for opponent-motion normalization by measuring C(m) thresholds whilst varying the orientation, spatial frequency, or size of flicker checks. In all conditions, performance (percent correct) decayed lawfully as we decreased motion contrast, validating the C(m) metric for our stimuli. Thresholds decreased with check size and also improved as we increased either the orientation or spatial-frequency difference between motion and flicker checks. Our data are inconsistent with Heeger-type normalization models [Vis. Neurosci. 9 (1992) 181] in which excitatory inputs are normalized by a non-selective pooling of inhibitory inputs, but data are consistent with the implicit assumption in Georgeson and Scott-Samuel's model that flicker normalization is localized in orientation, scale, and space. However, our lateral masking paradigm leaves open the possibility that the spatial properties of flicker normalization would be different if opponent and flicker energies spatially overlapped. Further characterization of motion contrast will require models of the spatial, temporal, and joint space-time properties of mechanisms mediating opponent-motion and flicker normalization.

Motion integration during motion aftereffects

The perceived global motion of a stimulus depends on how its different local motion-direction vectors are distributed in space and time. When they are explicitly co-localized, as in the case of locally paired motion, competitive motion integration mechanisms produce a unitary global motion direction determined by their vector average. During motion aftereffects induced by simultaneous adaptation to multiple motion directions, just as in the case of locally paired motion, different directional signals originate simultaneously from exactly the same position in space. Therefore, the perceived global motion direction during motion aftereffects results from local vector averaging of the co-localized motion-direction signals induced by adaptation.

Neural correlates of spontaneous direction reversals in ambiguous apparent visual motion

Looking at bistable visual stimuli, the observer experiences striking transitions between two competing percepts while the physical stimulus remains the same. Using functional imaging techniques, it is therefore possible to isolate neural correlates of perceptual changes that are independent of the low-level aspects of the stimulus. Previous experiments have demonstrated distributed activations in human extrastriate visual cortex related to switches between competing percepts. Here we asked where extrastriate responses still occur with a bistable stimulus that minimizes the cognitive difference between the two percepts. We used the "spinning wheel illusion," a bistable apparent motion stimulus of which both possible percepts correspond to the same object, share the same center, and are perceived as identically patterned stimuli moving at the same speed and changing only in direction. Using functional magnetic resonance imaging, we analyzed the spatial distribution of event-related activations occurring during spontaneous reversals of perceived direction of motion. In accordance with earlier neuroimaging findings for bistable percepts, we observed event-related activations in several frontal and parietal areas, including the superior parietal cortex bilaterally, the right inferior parietal cortex, and the premotor and inferior frontal cortex of both hemispheres. Furthermore, we found bilateral activations in the occipitotemporal junction (hMT+/V5) and in the lateral occipital sulcus ("KO") posterior to hMT+/V5, but not in areas of the "ventral stream" of cortical visual processing. Our data suggest that, while a frontoparietal network subserves more general aspects in bistable visual perception, the activations in functionally specialized extrastriate visual cortex are highly category- or attribute-specific.

Neural correlates of perceptual priming of visual motion

In two experiments, the temporal dynamics of neural activity underlying perceptual priming of visual motion was examined using event-related potentials (ERPs) during directional judgments of the apparent motion of two-dimensional sine-wave gratings. Compared to perceptually ambiguous motion, unambiguous left- or rightward motion was associated with enhanced ERP activity about 300 ms after the onset of apparent motion. In the second experiment, ERPs were recorded to two successive motion jumps in which an unambiguous motion jump served as a prime for a subsequent target motion that was ambiguous. The prime-target time interval was varied between 200, 400, and 1000 ms. In a control (motion reversal) condition, the two motion jumps were both unambiguous but in opposite directions. Compared to the motion reversal condition, motion priming was associated with an enhancement of ERP amplitudes at 100 ms and 350 ms following target stimulus onset. ERP enhancement was greatest at a short prime-target interval of 200 ms, which was also associated behaviorally with the strongest priming. The ERP enhancement and behavioral priming were both eliminated at the long 1000 ms prime-target interval. Functional magnetic resonance imaging (fMRI) data from a subset of subjects supported the view that motion priming involves modulation of neural responses both in early visual cortex and in later stages of visual processing.

Pattern-motion responses in human visual cortex

Physiological models of visual motion processing posit that 'pattern-motion cells' represent the direction of moving objects independent of their particular spatial pattern. We performed fMRI experiments to identify neuronal activity in the human brain selective for pattern motion. A protocol using adaptation to moving 'plaid' stimuli allowed us to separate pattern-motion responses from other types of motion-related activity within the same brain structures, and revealed strong pattern-motion selectivity in human visual area MT+. Reducing the perceptual coherence of the plaids yielded a corresponding decrease in pattern-motion responsivity, providing evidence that percepts of coherent motion are closely linked to the activity of pattern-motion cells in human MT+.

Sound-level-dependent representation of frequency modulations in human auditory cortex: a low-noise fMRI study

Recognition of sound patterns must be largely independent of level and of masking or jamming background sounds. Auditory patterns of relevance in numerous environmental sounds, species-specific vocalizations and speech are frequency modulations (FM). Level-dependent activation of the human auditory cortex (AC) in response to a large set of upward and downward FM tones was studied with low-noise (48 dB) functional magnetic resonance imaging at 3 Tesla. Separate analysis in four territories of AC was performed in each individual brain using a combination of anatomical landmarks and spatial activation criteria for their distinction. Activation of territory T1b (including primary AC) showed the most robust level dependence over the large range of 48-102 dB in terms of activated volume and blood oxygen level dependent contrast (BOLD) signal intensity. The left nonprimary territory T2 also showed a good correlation of level with activated volume but, in contrast to T1b, not with BOLD signal intensity. These findings are compatible with level coding mechanisms observed in animal AC. A systematic increase of activation with level was not observed for T1a (anterior of Heschl's gyrus) and T3 (on the planum temporale). Thus these areas might not be specifically involved in processing of the overall intensity of FM. The rostral territory T1a of the left hemisphere exhibited highest activation when the FM sound level fell 12 dB below scanner noise. This supports the previously suggested special involvement of this territory in foreground-background decomposition tasks. Overall, AC of the left hemisphere showed a stronger level-dependence of signal intensity and activated volume than the right hemisphere. But any side differences of signal intensity at given levels were lateralized to right AC. This might point to an involvement of the right hemisphere in more specific aspects of FM processing than level coding.

Functional magnetic resonance imaging of the visual system

Functional magnetic resonance imaging (fMRI), which is a technique useful for non-invasive mapping of brain function, is well suited for studying the visual system. This review highlights current clinical applications and research studies involving patients with visual deficits. Relevant reports regarding the investigation of the brain's role in visual processing and some newer fMRI techniques are also reviewed. Functional magnetic resonance imaging has been used for presurgical mapping of visual cortex in patients with brain lesions and for studying patients with amblyopia, optic neuritis, and residual vision in homonymous hemianopia. Retinotopic borders, motion processing, and visual attention have been the topics of several fMRI studies. These reports suggest that fMRI can be useful in clinical and research studies in patients with visual deficits.

Shifts in the population response in the middle temporal visual area parallel perceptual and motor illusions produced by apparent motion

We recorded behavioral, perceptual, and neural responses to targets that provided apparent visual motion consisting of a sequence of stationary flashes. Increasing the flash separation degrades the quality of motion, but for some separations evoked larger smooth pursuit responses from both humans and monkeys than did smooth motion. The same flash separations also produced an increase in perceived speed in humans. Recordings from single neurons in the middle temporal visual area (MT) of awake monkeys revealed a potential basis for the illusion in the population response. Apparent motion produced diminished neural responses relative to smooth motion. However, neurons with slow preferred speeds were more affected than were those with fast preferred speeds. Increasing the flash separation thus caused the population response to become diminished in amplitude and to shift so that the most active neurons had higher preferred speeds. The entire constellation of effects of apparent motion on the magnitude and latency of the initial pursuit response was accounted for if the MT population response was decoded by (1) creating an opponent motion signal for each neuron by treating its preferred and opposite direction responses as those of a pair of oppositely tuned neurons and (2) computing the vector average of these opponent motion signals. Other ways of decoding the population response recorded in MT failed to account for one or more aspects of behavior. We conclude that the effects of apparent motion on both pursuit and perception can be accounted for if target speed is estimated from the MT population response by a neural computation that implements a vector average based on opponent motion.

Motion processing in the macaque: revisited with functional magnetic resonance imaging

A great deal is known about the response properties of single neurons processing sensory information. In contrast, less is understood about the collective characteristics of networks of neurons that may underlie sensory capacities of animals. We used functional magnetic resonance imaging to study the emergent properties of populations of neurons processing motion across different brain areas. Using a visual adaptation paradigm, we localized a distributed network of visual areas that process information about the direction of motion as expected from single-cell recording studies. However, we found an apparent discrepancy between the directional signals in certain visual areas as measured with blood oxygenation level-dependent imaging compared with an estimate based on the spiking of single neurons. We propose a hypothesis that may account for this difference based on the postulate that neuronal selectivity is a function of the state of adaptation. Consequently, neurons classically thought to lack information about certain attributes of the visual scene may nevertheless receive and process this information. We further hypothesize that this adaptation-dependent selectivity may arise from intra- or inter-area cellular connections, such as feedback from higher areas. This network property may be a universal principle the computational goal of which is to enhance the ability of neurons in earlier visual areas to adapt to statistical regularities of the input and therefore increase their sensitivity to detect changes along these stimulus dimensions.

Neuronal basis of the motion aftereffect reconsidered

Several fMRI studies have reported MT+ response increases correlated with perception of the motion aftereffect (MAE). However, attention can strongly affect MT+ responses, and subjects may naturally attend more to the MAE than control trials without MAE. We found that requiring subjects to attend to motion on both MAE and control trials produced equal levels of MT+ response, suggesting that attention may have confounded the interpretation of previous experiments; in our data, attention accounts for the entire effect. After eliminating this confound, we observed that direction-selective motion adaptation produced a direction-selective imbalance in MT+ responses (and earlier visual areas), and yielded a corresponding asymmetry in speed discrimination thresholds. These findings provide physiological evidence that population level response imbalances underlie the MAE, and quantify the relative proportions of direction-selective neurons across human visual areas.

Human cortical activity correlates with stereoscopic depth perception

Stereoscopic depth perception is based on binocular disparities. Although neurons in primary visual cortex (V1) are selective for binocular disparity, their responses do not explicitly code perceived depth. The stereoscopic pathway must therefore include additional processing beyond V1. We used functional magnetic resonance imaging (fMRI) to examine stereo processing in V1 and other areas of visual cortex. We created stereoscopic stimuli that portrayed two planes of dots in depth, placed symmetrically about the plane of fixation, or else asymmetrically with both planes either nearer or farther than fixation. The interplane disparity was varied parametrically to determine the stereoacuity threshold (the smallest detectable disparity) and the upper depth limit (largest detectable disparity). fMRI was then used to quantify cortical activity across the entire range of detectable interplane disparities. Measured cortical activity covaried with psychophysical measures of stereoscopic depth perception. Activity increased as the interplane disparity increased above the stereoacuity threshold and dropped as interplane disparity approached the upper depth limit. From the fMRI data and an assumption that V1 encodes absolute retinal disparity, we predicted that the mean response of V1 neurons should be a bimodal function of disparity. A post hoc analysis of electrophysiological recordings of single neurons in macaques revealed that, although the average firing rate was a bimodal function of disparity (as predicted), the precise shape of the function cannot fully explain the fMRI data. Although there was widespread activity within the extrastriate cortex (consistent with electrophysiological recordings of single neurons), area V3A showed remarkable sensitivity to stereoscopic stimuli, suggesting that neurons in V3A may play a special role in the stereo pathway.

Early discrimination of coherent versus incoherent motion by multiunit and synaptic activity in human putative MT+

A laminar probe was chronically implanted in human putative MT+. The area was specifically responsive to globally coherent visual motion, a crucial aspect of the perception of movement through space. The probe contained 23 microcontacts spaced every 175 microm in a linear array roughly perpendicular to the cortical surface. Current-source density (CSD) and multiunit activity (MUA) were recorded while viewing initially stationary random dot patterns that either moved incoherently or dilated from the central fixation. Onset of visual motion evoked large MUA/CSD activity, with coherent motion evoking earlier and faster-rising MUA/CSD activity than incoherent, in both superficial and deep pyramidal layers. The selective response, peaking at approximately 115 ms, was especially large in deep pyramids, providing evidence that information necessary for visual flow calculations is projected from MT+ at an early latency to distant structures. The early onset of differential MUA/CSD implies that the selectivity of this area does not depend on recurrent inhibition or other intrinsic circuitry to detect coherent motion. The initially greater increase of MUA to coherent stimuli was followed by a greater decrease beginning at approximately 133 ms, apparently because of recurrent inhibition. This resulted in the total MUA being greater to incoherent than coherent stimuli, whereas total rectified CSD was overall greater to coherent than to incoherent stimuli. However, MUA distinguished stationary from moving stimuli more strongly than did CSD. Thus, while estimates of total cell firing (MUA), and of total synaptic activity (CSD) generally correspond to previously reported BOLD results, they may differ in important details.

Neural dynamics of motion integration and segmentation within and across apertures

A neural model is developed of how motion integration and segmentation processes, both within and across apertures, compute global motion percepts. Figure-ground properties, such as occlusion, influence which motion signals determine the percept. For visible apertures, a line's terminators do not specify true line motion. For invisible apertures, a line's intrinsic terminators create veridical feature-tracking signals. Sparse feature-tracking signals can be amplified before they propagate across position and are integrated with ambiguous motion signals within line interiors. This integration process determines the global percept. It is the result of several processing stages: directional transient cells respond to image transients and input to a directional short-range filter that selectively boosts feature-tracking signals with the help of competitive signals. Then, a long-range filter inputs to directional cells that pool signals over multiple orientations, opposite contrast polarities, and depths. This all happens no later than cortical area MT. The directional cells activate a directional grouping network, proposed to occur within cortical area MST, within which directions compete to determine a local winner. Enhanced feature-tracking signals typically win over ambiguous motion signals. Model MST cells that encode the winning direction feed back to model MT cells, where they boost directionally consistent cell activities and suppress inconsistent activities over the spatial region to which they project. This feedback accomplishes directional and depthful motion capture within that region. Model simulations include the barberpole illusion, motion capture, the spotted barberpole, the triple barberpole, the occluded translating square illusion, motion transparency and the chopsticks illusion. Qualitative explanations of illusory contours from translating terminators and plaid adaptation are also given.

Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect

Electrophysiological studies in nonhuman primates and other mammals have shown that sensory cues from different modalities that appear at the same time and in the same location can increase the firing rate of multisensory cells in the superior colliculus to a level exceeding that predicted by summing the responses to the unimodal inputs. In contrast, spatially disparate multisensory cues can induce a profound response depression. We have previously demonstrated using functional magnetic resonance imaging (fMRI) that similar indices of crossmodal facilitation and inhibition are detectable in human cortex when subjects listen to speech while viewing visually congruent and incongruent lip and mouth movements. Here, we have used fMRI to investigate whether similar BOLD signal changes are observable during the crossmodal integration of nonspeech auditory and visual stimuli, matched or mismatched solely on the basis of their temporal synchrony, and if so, whether these crossmodal effects occur in similar brain areas as those identified during the integration of audio-visual speech. Subjects were exposed to synchronous and asynchronous auditory (white noise bursts) and visual (B/W alternating checkerboard) stimuli and to each modality in isolation. Synchronous and asynchronous bimodal inputs produced superadditive BOLD response enhancement and response depression across a large network of polysensory areas. The most highly significant of these crossmodal gains and decrements were observed in the superior colliculi. Other regions exhibiting these crossmodal interactions included cortex within the superior temporal sulcus, intraparietal sulcus, insula, and several foci in the frontal lobe, including within the superior and ventromedial frontal gyri. These data demonstrate the efficacy of using an analytic approach informed by electrophysiology to identify multisensory integration sites in humans and suggest that the particular network of brain areas implicated in these crossmodal integrative processes are dependent on the nature of the correspondence between the different sensory inputs (e.g. space, time, and/or form).