Effect of spatial attention on the responses of area MT neurons

Shared attentional resources for global and local motion processing

One of the most important aspects of visual attention is its flexibility; our attentional "window" can be tuned to different spatial scales, allowing us to perceive large-scale global patterns and local features effortlessly. We investigated whether the perception of global and local motion competes for a common attentional resource. Subjects viewed arrays of individual moving Gabors that group to produce a global motion percept when subjects attended globally. When subjects attended locally, on the other hand, they could identify the direction of individual uncrowded Gabors. Subjects were required to devote their attention toward either scale of motion or divide it between global and local scales. We measured direction discrimination as a function of the validity of a precue, which was varied in opposite directions for global and local motion such that when the precue was valid for global motion, it was invalid for local motion and vice versa. There was a trade-off between global and local motion thresholds, such that increasing the validity of precues at one spatial scale simultaneously reduced thresholds at that spatial scale but increased thresholds at the other spatial scale. In a second experiment, we found a similar pattern of results for static-oriented Gabors: Attending to local orientation information impaired the subjects' ability to perceive globally defined orientation and vice versa. Thresholds were higher for orientation compared to motion, however, suggesting that motion discrimination in the first experiment was not driven by orientation information alone but by motion-specific processing. The results of these experiments demonstrate that a shared attentional resource flexibly moves between different spatial scales and allows for the perception of both local and global image features, whether these features are defined by motion or orientation.

Cortical neuronal responses to optic flow are shaped by visual strategies for steering

We hypothesized that neuronal responses to virtual self-movement would be enhanced during steering tasks. We recorded the activity of medial superior temporal (MSTd) neurons in monkeys trained to steer a straight-ahead course, using optic flow. We found smaller optic flow responses during active steering than during the passive viewing of the same stimuli. Behavioral analysis showed that the monkeys had learned to steer using local motion cues. Retraining the monkeys to use the global pattern of optic flow reversed the effects of the active-steering task: active steering then evoked larger responses than passive viewing. We then compared the responses of neurons during active steering by local motion and by global patterns: Local motion trials promoted the use of local dot movement near the center of the stimulus by occluding the peripheral visual field midway through the trial. Global pattern trials promoted the use of radial pattern movement by occluding the central visual field midway through the trial. In this study, identical full-field optic-flow stimuli evoked larger responses in global-pattern trials than in local motion trials. We conclude that the selection of specific visual cues reflects strategies for active steering and alters MSTd neuronal responses to optic flow.

Finding a face in the crowd: parallel and serial neural mechanisms of visual selection

At any given moment, our visual system is confronted with more information than it can process. Thus, attention is needed to select behaviorally relevant information in a visual scene for further processing. Behavioral studies of attention during visual search have led to the distinction between serial and parallel mechanisms of selection. To find a target object in a crowded scene, for example a "face in a crowd", the visual system might turn on and off the neural representation of each object in a serial fashion, testing each representation against a template of the target object. Alternatively, it might allow the processing of all objects in parallel, but bias activity in favor of those neurons representing critical features of the target, until the target emerges from the background. Recent neurophysiological evidence shows that both serial and parallel selections take place in neurons of the ventral "object-recognition pathway" during visual search tasks in which monkeys freely scan complex displays to find a target object. Furthermore, attentional selection appears to be mediated by changes in the synchrony of responses of neuronal populations in addition to the modulation of the firing rate of individual neurons.

The contribution of the human PPC to the orienting of visuospatial attention during smooth pursuit

Smooth pursuit eye movements function to stabilize the retinal image of small moving targets. In order for those targets to be foveated, however, they must first be "captured" by an attentional mechanism which then interacts with the oculomotor system. Cortical sites involved with producing smooth pursuit overlap with areas known to be involved in directing visuospatial attention, particularly the posterior parietal cortex (PPC). The goal of the current study was to characterize the contributions made by the left and right posterior parietal cortices (lPPC and rPPC) to the interaction between visuospatial attention and the generation of smooth pursuit eye movements. Transcranial magnetic stimulation (TMS) was used to temporarily disrupt each area at different times around target motion onset in a pursuit task that explicitly manipulated the covert orienting of attention. TMS over the lPPC, rPPC and a control site (the vertex) evoked a similar pattern of results, in that the earlier TMS delivery times caused a reduced pursuit latency compared to baseline measures, while TMS immediately prior to target motion onset resulted in latencies slower than baseline. In addition, however, TMS over the lPPC and rPPC (but not the vertex) preferentially influenced the generation of contralateral pursuit, with the lPPC doing so in a relatively time-independent manner, and the rPPC doing so in a time-dependent manner. This pattern of results implies that both the left and right PPC are directly involved in the interaction between attention and smooth pursuit preparation.

Does attention alter appearance?

Abrupt onsets in the visual field can change the appearance of subsequent stimuli, according to one interpretation, by engaging an attentional mechanism that increases effective stimulus contrast. However, abrupt onsets can also engage capacity-unlimited and thus attention-independent sensory mechanisms. We conducted a series of experiments to differentiate the sensory and attentional accounts. Observers compared the contrasts of uncued low-contrast peripheral targets with simultaneous targets cued by one of three cue types with different sensory attributes: white or black peripheral abrupt onsets and central gaze direction cues devoid of sensory activity near the target locations. Each cue facilitated the perception of perithreshold targets; however, the white abrupt onsets increased the perceived contrast of suprathreshold targets, whereas the black abrupt onsets tended to reduce the perceived contrast, and the gaze direction cues had no significant effect. The effectiveness of the gaze direction cues in automatically orienting attention was demonstrated in a control experiment in which they consistently speeded response times. The results suggest that sensory interaction, and not attention, is responsible for changes in appearance.

Behavioral Influences on Cortical Neuronal Responses to Optic Flow

Optic flow selectively activates neurons in medial superior temporal (MST) cortex. We find that many MST neurons yield larger and more selective responses when the optic flow guides a subsequent eye movement. Smaller, less selective responses are seen when optic flow is preceded by a flashed precue that guides eye movements. Selectivity can decrease by a third (32%) after a flashed precue is presented at a peripheral location as a small spot specifying the target location of the eye movement. Smaller decreases in selectivity (18%) occur when the precue is presented centrally with its shape specifying the target location. Shape precues presented centrally, but not linked to specific target locations, do not appear to alter optic flow selectivity. The effects of spatial precueing can be reversed so that the precue leads to larger and more selective optic flow responses: A flashed precue presented as a distracter before behaviorally relevant optic flow is associated with larger optic flow responses and a 45% increase in selectivity. Together, these findings show that spatial precues can decrease or increase the size and selectivity of optic flow responses depending on the associated behavioral contingencies.

Effects of spatial attention on contrast response functions in macaque area V4

Previous single-unit studies of visual cortex have reported that spatial attention modulates responses to different orientations and directions proportionally, such that it does not change the width of tuning functions for these properties. Other studies have suggested that spatial attention causes a leftward shift in contrast response functions, such that its effects on responses to stimuli of different contrasts are not proportional. We have further explored the effects of attention on stimulus-response functions by measuring the responses of 131 individual V4 neurons in two monkeys while they did a task that controlled their spatial attention. Each neuron was tested with a set of stimuli that spanned complete ranges of orientation and contrast during different states of attention. Consistent with earlier reports, attention scaled responses to preferred and nonpreferred orientations proportionally. However, we did not find compelling evidence that the effects were best described by a leftward shift of the contrast response function. The modulation of neuronal responses by attention was well described by either a leftward shift or proportional scaling of the contrast response function. Consideration of differences in experimental design and analysis that may have contributed to this discrepancy suggests that it was premature to exclude a proportional scaling of responses to different contrasts by attention in favor of a leftward shift of contrast response functions. The current results reopen the possibility that the effects of attention on stimulus-response functions are well described by a single proportional increase in a neuron's response to all stimuli.

Separate attentional resources for vision and audition

Current models of attention, typically claim that vision and audition are limited by a common attentional resource which means that visual performance should be adversely affected by a concurrent auditory task and vice versa. Here, we test this implication by measuring auditory (pitch) and visual (contrast) thresholds in conjunction with cross-modal secondary tasks and find that no such interference occurs. Visual contrast discrimination thresholds were unaffected by a concurrent chord or pitch discrimination, and pitch-discrimination thresholds were virtually unaffected by a concurrent visual search or contrast discrimination task. However, if the dual tasks were presented within the same modality, thresholds were raised by a factor of between two (for visual discrimination) and four (for auditory discrimination). These results suggest that at least for low-level tasks such as discriminations of pitch and contrast, each sensory modality is under separate attentional control, rather than being limited by a supramodal attentional resource. This has implications for current theories of attention as well as for the use of multi-sensory media for efficient informational transmission.

Strategies optimize the detection of motion transients

Strategies are implicitly formed when a task is consistent and can be used to improve performance. To investigate how strategies can alter perceptual performance, I trained animals in a reaction time (RT) detection task in which the probability of a fixed duration motion pulse appearing varied over time in a consistent manner. Consistent with previous studies suggesting the implicit representation of task timing, I found that RTs were inversely related to the probability of the pulse appearing and decreased with training. I then inferred the sensory integration underlying responses using behavioral reverse correlation analysis. This analysis revealed that training and anticipation optimized detection by improving the correlation between sensory integration and the spatiotemporal extent of the motion pulse. Moreover, I found that these improvements in sensory integration could largely explain observed changes in the distribution of RT with training and anticipation. These results suggest that training can increase detection performance by optimizing sensory integration according to implicitly formed representations of the likelihood and nature of the stimulus.

Competitive stimulus interactions within single response fields of superior colliculus neurons

In addition to its role in saccade generation, the superior colliculus (SC) is involved in target selection, saccade selection, and shifting the focus of spatial attention. Here, we investigated the influence of saccade selection on sensory interactions within single response fields (RFs) of SC neurons. One or two differently shaped stimuli were presented within single RFs of SC neurons, and the shape of a centrally located cue indicated whether and where to make a saccade (Go-Go) or whether to make or withhold a saccade (Go/No-Go). We found that, when two stimuli appeared at different locations within a single RF, SC neuronal activity was reduced compared with when a single stimulus appeared in isolation within the center of the RF in both the Go-Go and Go/No-Go tasks. In both tasks, a subsequent cue indicating one stimulus as a saccade target reduced the influence of the second stimulus located within the RF. We found that the time course of the suppression resulting from the two stimuli was approximately 130 ms, a time close to that seen in cortex. Finally, we found that the influence of two stimuli within single RFs of SC neurons changed over time in both the Go-Go and the Go/No-Go tasks. Initially, the neurons averaged the influence of two stimuli. As the trial progressed, the SC neurons signaled only the saccade vector that was produced. We conclude that cues to shift gaze, like attention, modulate the influence of sensory interactions, providing additional support for the linkage between attention and saccade selection.

The relationship between task performance and functional magnetic resonance imaging response

We compared psychophysical and functional magnetic resonance imaging (fMRI) responses within areas V1-V3 and MT+ during both a speed and a contrast discrimination task. We found that fMRI responses did not depend significantly on task in any of these areas. Moreover, responses in V1-V3 were larger than those in MT+ for both the speed and the contrast discrimination tasks across a wide range of contrasts. This pattern of results demonstrates that localizing function based on finding those regions of cortex that show greater activity to a given task-stimulus combination than to other tasks and stimuli may, under certain conditions, be misleading. However, a simple ideal observer model assuming that perceptual thresholds are dependent on neuronal population responses does successfully show that V1 has neuronal properties consistent with our subjects' contrast discrimination performance, and that MT+ has neuronal properties consistent with subjects' performance on a speed discrimination task.

Active vision in parietal and extrastriate cortex

Vision is an active process. We do not see the world directly; rather, we construct a representation of it from sensory inputs in combination with internal, nonvisual signals. In the case of spatial perception, our representation of the visual scene must take into account our own movements. This allows us to perceive the world as stationary despite the constant eye movements that produce new images on the retina. How is this perceptual stability achieved? Our central hypothesis is that a corollary discharge of the eye movement command updates, or remaps, an internal representation when the eyes move. In support of this hypothesis, the authors review evidence that parietal cortex and extrastriate visual areas in both monkeys and humans participate in spatial updating. These findings shed new light on the neural circuitry involved in producing a stable and coherent perception of visual space.

fMRI evidence for individual differences in premotor modulation of extrastriatal visual-perceptual processing of redundant targets

To perceive the vast array of stimuli in the world around us, the visual system employs parallel processing mechanisms that ensure efficiency in perceiving multiple objects in a scene. A way to test this efficiency is to measure reaction time (RT) to pairs of identical stimuli, presented singly or as doublets; typically, the resulting phenomenon is the redundant targets effect (RTE), which manifests as faster RTs to paired than singly presented stimuli. It is controversial, however, whether the neural locus of the parallel processing mechanisms invoked to produce the RTE is perceptual or motor and why some studies observe a substantial RTE and others do not. To resolve these two issues, we measured the RTE in young adults while undergoing functional MRI. Regarding the question of a perceptual or motor basis for the RTE, we observed that bilateral activation of extrastriate cortex was prominent in paired vs. the sum of the two single stimulus conditions, indicating that the RTE invoked perceptual mechanisms; by contrast, the motor cortex was not disproportionately activated in this comparison. Regarding the magnitude of the RTE, we compared activation patterns in individuals with small vs. large RTEs and observed that frontal and premotor areas were activated with small RTEs. These data indicate that the primary processing level of response facilitation, observed as the RTE, is perceptual, but the modulation of the RTE magnitude is premotor and associated with basic aspects of response selection and preparation.

Area MT Neurons Respond to Visual Motion Distant From Their Receptive Fields

Neurons in cortical area MT have localized receptive fields (RF) representing the contralateral hemifield and play an important role in processing visual motion. We recorded the activity of these neurons during a behavioral task in which two monkeys were required to discriminate and remember visual motion presented in the ipsilateral hemifield. During the task, the monkeys viewed two stimuli, sample and test, separated by a brief delay and reported whether they contained motion in the same or in opposite directions. Fifty to 70% of MT neurons were activated by the motion stimuli presented in the ipsilateral hemifield at locations far removed from their classical receptive fields. These responses were in the form of excitation or suppression and were delayed relative to conventional MT responses. Both excitatory and suppressive responses were direction selective, but the nature and the time course of their directionality differed from the conventional excitatory responses recorded with stimuli in the RF. Direction selectivity of the excitatory remote response was transient and early, whereas the suppressive response developed later and persisted after stimulus offset. The presence or absence of these unusual responses on error trials, as well as their magnitude, was affected by the behavioral significance of stimuli used in the task. We hypothesize that these responses represent top-down signals from brain region(s) accessing information about stimuli in the entire visual field and about the behavioral state of the animal. The recruitment of neurons in the opposite hemisphere during processing of behaviorally relevant visual signals reveals a mechanism by which sensory processing can be affected by cognitive task demands.

Haptic feature extraction

This study examined the process by which the shape of a haptically explored object is synthesized from the geometric characteristics of simpler constituent elements, such as arcs and ellipses. Subjects traced the outlines of virtual objects by means of whole arm movements. Each object consisted of the union of a large central ellipse and two smaller circles, extending upward and outward from the top left and right sides of the base. The sizes of the two circles and the eccentricity of the elliptical base were varied. After exploring the object's contour in the absence of vision, subjects reproduced the sensed shape by means of freehand drawing. Speed and force were modulated during the exploratory phase in a manner that suggested that subjects reacted to rather than predicted changes in curvature. Also, subjects typically devoted more time to exploring the part of the contour encompassing the two smaller circles. During drawing, individual features of the explored shape were reproduced with varying degrees of fidelity. Aspects related to the size and location of the smaller circles were reproduced better than was the eccentricity of the ellipse forming the base. Since subjects spent proportionally less time exploring the base, these results suggest that subjects selectively focused attention to regions of high spatial contrast and that the exploratory strategy introduced distortions in the haptically sensed shapes.

Comparing the Effects of Auditory Deprivation and Sign Language within the Auditory and Visual Cortex

To investigate neural plasticity resulting from early auditory deprivation and use of American Sign Language, we measured responses to visual stimuli in deaf signers, hearing signers, and hearing nonsigners using functional magnetic resonance imaging. We examined “compensatory hypertrophy” (changes in the responsivity/size of visual cortical areas) and “cross-modal plasticity” (changes in auditory cortex responses to visual stimuli). We measured the volume of early visual areas (V1, V2, V3, V4, and MT+). We also measured the amplitude of responses within these areas, and within the auditory cortex, to a peripheral visual motion stimulus that was attended or ignored. We found no major differences between deaf and hearing subjects in the size or responsivity of early visual areas. In contrast, within the auditory cortex, motion stimuli evoked significant responses in deaf subjects, but not in hearing subjects, in a region of the right auditory cortex corresponding to Brodmann's areas 41, 42, and 22. This hemispheric selectivity may be due to a predisposition for the right auditory cortex to process motion; earlier studies report a right hemisphere bias for auditory motion in hearing subjects. Visual responses within the auditory cortex of deaf subjects were stronger for attended than ignored stimuli, suggesting top-down processes. Hearing signers did not show visual responses in the auditory cortex, indicating that cross-modal plasticity can be attributed to auditory deprivation rather than sign language experience. The largest effects of auditory deprivation occurred within the auditory cortex rather than the visual cortex, suggesting that the absence of normal input is necessary for large-scale cortical reorganization to occur.

Structure and function of visual area MT

The small visual area known as MT or V5 has played a major role in our understanding of the primate cerebral cortex. This area has been historically important in the concept of cortical processing streams and the idea that different visual areas constitute highly specialized representations of visual information. MT has also proven to be a fertile culture dish--full of direction- and disparity-selective neurons--exploited by many labs to study the neural circuits underlying computations of motion and depth and to examine the relationship between neural activity and perception. Here we attempt a synthetic overview of the rich literature on MT with the goal of answering the question, What does MT do?

Attention and visual perception

Somewhere between the retina and our conscious visual experience, the majority of the information impinging on the eye is lost. We are typically aware of only either the most salient parts of a visual scene or the parts that we are actively paying attention to. Recent research on visual neurons in monkeys is beginning to show how the brain both selects and discards incoming visual information. For example, what happens to the responses of visual neurons when attention is directed to one element, such as an oriented colored bar, embedded among an array of other oriented bars? Some of this research shows that attention to the oriented bar restricts the receptive field of visual neurons down to this single element. However, other research shows that attention to this single element affects the responses of neurons with receptive fields throughout the visual field. In this review, these two seemingly contradictory results are shown to actually be mutually consistent. A simple computational model is described that explains these results, and also provides a framework for predicting a variety of additional neurophysiological, neuroimaging and behavioral studies of attention.

Tuning Curve Shift by Attention Modulation in Cortical Neurons: a Computational Study of its Mechanisms

Physiological studies of visual attention have demonstrated that focusing attention near a visual cortical neuron's receptive field (RF) results in enhanced evoked activity and RF shift. In this work, we explored the mechanisms of attention induced RF shifts in cortical network models that receive an attentional 'spotlight'. Our main results are threefold. First, whereas a 'spotlight' input always produces toward-attention shift of the population activity profile, we found that toward-attention shifts in RFs of single cells requires multiplicative gain modulation. Secondly, in a feedforward two-layer model, focal attentional gain modulation in first-layer neurons induces RF shift in second-layer neurons downstream. In contrast to experimental observations, the feedforward model typically fails to produce RF shifts in second-layer neurons when attention is directed beyond RF boundaries. We then show that an additive spotlight input combined with a recurrent network mechanism can produce the observed RF shift. Inhibitory effects in a surround of the attentional focus accentuate this RF shift and induce RF shrinking. Thirdly, we considered interrelationship between visual selective attention and adaptation. Our analysis predicts that the RF size is enlarged (respectively reduced) by attentional signal directed near a cell's RF center in a recurrent network (resp. in a feedforward network); the opposite is true for visual adaptation. Therefore, a refined estimation of the RF size during attention and after adaptation would provide a probe to differentiate recurrent versus feedforward mechanisms for RF shifts.

Correlation between speed perception and neural activity in the middle temporal visual area

We conducted electrophysiological recording and microstimulation experiments to test the hypothesis that the middle temporal visual area (MT) plays a direct role in perception of the speed of moving visual stimuli. We trained rhesus monkeys on a speed discrimination task in which monkeys chose the faster speed of two moving random dot patterns presented simultaneously in spatially segregated apertures. In electrophysiological experiments, we analyzed the activity of speed-tuned MT neurons and multiunit clusters during the discrimination task. Neural activity was correlated with the monkeys' behavioral choices on a trial-to-trial basis (choice probability), and the correlation was predicted by the speed-tuning properties of each unit. In microstimulation experiments, we activated clusters of MT neurons with homogeneous speed-tuning properties during the same speed discrimination task. In one monkey, microstimulation biased speed judgments toward the preferred speed of the stimulated neurons. Together, evidence from these two experiments suggests that MT neurons play a direct role in the perception of visual speed. Comparison of psychometric and neurometric thresholds revealed that single and multineuronal signals were, on average, considerably less sensitive than were the monkeys perceptually, suggesting that signals must be pooled across neurons to account for performance.

Attentional modulation in the human visual cortex: the time-course of the BOLD response and its implications

Throughout the visual areas of the brain, the sensory response to a stimulus is enhanced by attending to the stimulus. Neurophysiological studies in primates show that such enhancement is marked in posterior parietal cortex and some anterior occipital areas, but much more modest in the earliest processing stages, such as the primary visual cortex (V1). In contrast, human fMRI studies show large and robust attentional modulation in all visual areas, including V1. We investigate the possibility that, in the case of fMRI, the BOLD (blood oxygen level dependent) response may be increased not only by local attention-related increases in neural activity, but also by local blood-flow increases caused by remote control systems that anticipate an impending need for oxygen at the attended location. Such changes could be much more rapid than the rather slow response to oxygenation change that typifies the BOLD response. We have employed a paradigm that isolates the component of the BOLD response due to attentional modulation and the component due to the mere presence of a visual stimulus. The results show that the temporal profiles of the BOLD responses in human V1 to the onset of a stimulus and to the onset of attention are extremely similar. The time-course of the attention-related BOLD response is not consistent with the action of remote, anticipatory control mechanisms and suggests that the modulatory effect of attention seen in human V1 with fMRI probably reflects genuine changes in local neural activity that are considerably larger than in non-human primates.

Subcortical modulation of attention counters change blindness

Change blindness is the failure to see large changes in a visual scene that occur simultaneously with a global visual transient. Such visual transients might be brief blanks between visual scenes or the blurs caused by rapid or saccadic eye movements between successive fixations. Shifting attention to the site of the change counters this "blindness" by improving change detection and reaction time. We developed a change blindness paradigm for visual motion and then showed that presenting an attentional cue diminished the blindness in both humans and old world monkeys. We then replaced the visual cue with weak electrical stimulation of an area in the monkey's brainstem, the superior colliculus, to see if activation at such a late stage in the eye movement control system contributes to the attentional shift that counters change blindness. With this stimulation, monkeys more easily detected changes and had shorter reaction times, both characteristics of a shift of attention.

Parallel and Serial Neural Mechanisms for Visual Search in Macaque Area V4

To find a target object in a crowded scene, a face in a crowd for example, the visual system might turn the neural representation of each object on and off in a serial fashion, testing each representation against a template of the target item. Alternatively, it might allow the processing of all objects in parallel but bias activity in favor of those neurons that represent critical features of the target, until the target emerges from the background. To test these possibilities, we recorded neurons in area V4 of monkeys freely scanning a complex array to find a target defined by color, shape, or both. Throughout the period of searching, neurons gave enhanced responses and synchronized their activity in the gamma range whenever a preferred stimulus in their receptive field matched a feature of the target, as predicted by parallel models. Neurons also gave enhanced responses to candidate targets that were selected for saccades, or foveation, reflecting a serial component of visual search. Thus, serial and parallel mechanisms of response enhancement and neural synchrony work together to identify objects in a scene. To find a target object in a crowded scene, a face in a crowd for example, the visual system might turn the neural representation of each object on and off in a serial fashion, testing each representation against a template of the target item. Alternatively, it might allow the processing of all objects in parallel but bias activity in favor of those neurons that represent critical features of the target, until the target emerges from the background. To test these possibilities, we recorded neurons in area V4 of monkeys freely scanning a complex array to find a target defined by color, shape, or both. Throughout the period of searching, neurons gave enhanced responses and synchronized their activity in the gamma range whenever a preferred stimulus in their receptive field matched a feature of the target, as predicted by parallel models. Neurons also gave enhanced responses to candidate targets that were selected for saccades, or foveation, reflecting a serial component of visual search. Thus, serial and parallel mechanisms of response enhancement and neural synchrony work together to identify objects in a scene.

Attentional modulation of motion integration of individual neurons in the middle temporal visual area

We examined how spatially directed attention affected the integration of motion in neurons of the middle temporal (MT) area of visual cortex. We recorded from single MT neurons while monkeys performed a motion detection task under two attentional states. Using 0% coherent random dot motion, we estimated the optimal linear transfer function (or kernel) between the global motion and the neuronal response. This linear kernel filtered the random dot motion across direction, speed, and time. Slightly less than one-half of the neurons produced reasonably well defined kernels that also tended to account for both the directional selectivity and responses to coherent motion of different strengths. This subpopulation of cells had faster, more transient, and more robust responses to visual stimuli than neurons with kernels that did not contain well defined regions of integration. For those neurons that had large attentional modulation and produced well defined kernels, we found attention scaled the temporal profile of the transfer function with no appreciable shift in time or change in shape. Thus, for MT neurons described by a linear transfer function, attention produced a multiplicative scaling of the temporal integration window.

Adaptation to spiral motion: global but not local motion detectors are modulated by attention

In this study, we investigated the effect of attention on local motion detectors. For this purpose we used logarithmic spirals previously used by Cavanagh and Favreau [Perception, 1980, 9(2), 175-182]. While the adapting stimulus was a rotating logarithmic spiral, the test stimulus was either the same spiral or its mirror image. When superimposed, all contours of the spiral stimulus and its mirror image are 90 degrees apart. Presenting the same spiral during the test period shows adaptation of both local motion detectors and global rotation detectors, whereas showing the mirror-spiral stimulates another set of local motion detectors, and therefore illustrates adaptation at only the global motion level. To manipulate the attentional state of observers, a secondary task was presented during the adaptation phase and observers either performed the task or ignored it. Motion aftereffect (MAE) duration was measured afterwards. While the effects of attention and test stimulus type on MAE duration were both significant, the difference in the MAE strength between the attention-distracted and attention-not-distracted conditions was equal when the test stimulus was the same-spiral or the mirror-spiral, suggesting that attention to spiral motion modulates only global rotation units and does not affect local motion detectors located at V1. Our results are in accord with those reported by Watanabe et al. [Proceedings of the National Academy of Sciences of the USA, 1998, 95(19), 11489-11492] which showed differential modulation of motion processing areas depending on the type of motion being attended. Therefore our data are supportive of the notion that attentional modulation of V1 is highly task-dependent.

Comparison of the spatial limits on direction selectivity in visual areas MT and V1

We recorded responses to apparent motion from directionally selective neurons in primary visual cortex (V1) of anesthetized monkeys and middle temporal area (MT) of awake monkeys. Apparent motion consisted of multiple stationary stimulus flashes presented in sequence, characterized by their temporal separation (delta t) and spatial separation (delta x). Stimuli were 8 degrees square patterns of 100% correlated random dots that moved at apparent speeds of 16 or 32 degrees/s. For both V1 and MT, the difference between the response to the preferred and null directions declined with increasing flash separation. For each neuron, we estimated the maximum flash separation for which directionally selective responses were observed. For the range of speeds we used, delta x provided a better description of the limitation on directional responses than did delta t. When comparing MT and V1 neurons of similar preferred speed, there was no difference in the maximum delta x between our samples from the two areas. In both V1 and MT, the great majority of neurons had maximal values of delta x in the 0.25-1 degrees range. Mean values were almost identical between the two areas. For most neurons, larger flash separations led to both weaker responses to the preferred direction and increased responses to the opposite direction. The former mechanism was slightly more dominant in MT and the latter slightly more dominant in V1. We conclude that V1 and MT neurons lose direction selectivity for similar values of delta x, supporting the hypothesis that basic direction selectivity in MT is inherited from V1, at least over the range of stimulus speeds represented by both areas.

The effects of dividing attention on smooth pursuit eye tracking

Attentional processes have traditionally been closely linked to the production of saccadic eye movements, but their role in the control of smooth pursuit eye movements remains unclear. In two experiments we used dual task paradigms to vary the attentional resources available for pursuit eye tracking. In both experiments we found that attentionally demanding secondary tasks impaired smooth pursuit performance, resulting in decreased velocity and increased position error. These findings suggest that attention is important for the maintenance of accurate smooth pursuit, and do not support the hypothesis that pursuit is a relatively automatic function that proceeds optimally in the absence of attentional control. These results add weight to the suggestion that a similar functional architecture underlies both pursuit and saccadic eye movements.

Attentional modulation of visual processing

Single-unit recording studies in the macaque have carefully documented the modulatory effects of attention on the response properties of visual cortical neurons. Attention produces qualitatively different effects on firing rate, depending on whether a stimulus appears alone or accompanied by distracters. Studies of contrast gain control in anesthetized mammals have found parallel patterns of results when the luminance contrast of a stimulus increases. This finding suggests that attention has co-opted the circuits that mediate contrast gain control and that it operates by increasing the effective contrast of the attended stimulus. Consistent with this idea, microstimulation of the frontal eye fields, one of several areas that control the allocation of spatial attention, induces spatially local increases in sensitivity both at the behavioral level and among neurons in area V4, where endogenously generated attention increases contrast sensitivity. Studies in the slice have begun to explain how modulatory signals might cause such increases in sensitivity.

A common neuronal code for perceptual processes in visual cortex? Comparing choice and attentional correlates in V5/MT

In the past two decades, sensory neuroscience has moved from describing response properties to external stimuli in cerebral cortex to establishing connections between neuronal activity and sensory perception. The seminal studies by Newsome, Movshon and colleagues in the awake behaving macaque firmly link single cells in extrastriate area V5/MT and perception of motion. A decade later, extrastriate visual cortex appears awash with neuronal correlates for many different perceptual tasks. Examples are attentional signals, choice signals for ambiguous images, correlates for binocular rivalry, stereo and shape perception, and so on. These diverse paradigms are aimed at elucidating the neuronal code for perceptual processes, but it has been little studied how they directly compare or even interact. In this paper, I explore to what degree the measured neuronal signals in V5/MT for choice and attentional paradigms might reflect a common neuronal mechanism for visual perception.

Comparing Perceptual Signals of Single V5/MT Neurons in Two Binocular Depth Tasks

Neurons in the extrastriate visual area V5/MT show perceptually relevant signals in binocular depth tasks, which can be measured as a choice probability (CP) for the neuron. The presence of a CP in a particular paradigm may be an indicator that the neuron is generally part of the substrate for the perception of binocular depth. We compared the responses of those single neurons that show CPs in one stereoscopic depth task with their responses in another stereo task. Each neuron was tested for the presence of 1) CPs during a task in which macaques responded to the sign of binocular depth in a structure-from-motion stimulus, to judge its direction of three-dimensional rotation and 2) a consistent response to the stereo disparity of binocularly anti-correlated stimuli. Previous work, confirmed here, shows that changing the disparity of these binocularly anti-correlated stimuli often fails to yield a coherent change in the depth percept. For each test alone, there are V5/MT neurons that carry signals that are congruent with the perceptual effects. However, on comparing tests, there is no fixed pool of neurons that can account for the binocular depth percept. Excitation of neurons with a measurable CP does not necessarily lead to a change in perception. The cortical circuitry must be able to make dynamic changes in the pools of neurons that underlie perceptual judgments according to the demands of the task.

Is contrast just another feature for visual selective attention?

The biased-competition theory of attention [Annual Review of Neuroscience 18 (1995) 193] suggests that attention and stimulus contrast trade off, and implies that high-contrast stimuli should be easy to attend to and hard to ignore. To test this, observers searched displays for a target digit. Observers were well able to exclude high-contrast distractors when attempting to search only among low-contrast stimuli (Experiment 1). In Experiments 2 and 3, location determined which stimuli were relevant. When contrast of relevant and irrelevant stimuli was uncertain (due to contrast varying between trials, Experiment 2), increasing the contrast of distractors impaired performance. However, when contrast was certain (due to blocking of trials, Experiment 3) and targets were of low contrast, high contrast distractors produced less interference than low contrast distractors. The ability of subjects to attend selectively to low vs. high contrast items in Experiments 1 and 3 suggests that selectivity for stimulus contrast might be similar to other types of feature selectivity (e.g., color and location). Such findings are inconsistent with the biased competition theory regarding the interplay of contrast and attention. However, results from Experiment 2 suggest that, when target contrast varies, the default tendency is to attend to high-contrast items.

Feature-Based Attention Increases the Selectivity of Population Responses in Primate Visual Cortex

BACKGROUND

Attending to the spatial location or to nonspatial features of visual stimuli can modulate neuronal responses in primate visual cortex. The modulation by spatial attention changes the gain of sensory neurons and strengthens the representation of attended locations without changing neuronal selectivities such as directionality, i.e., the ratio of responses to preferred and anti-preferred directions of motion. Whether feature-based attention acts in a similar manner is unknown.

RESULTS

To clarify this issue, we recorded the responses of 135 direction-selective neurons in the middle temporal area (MT) of two macaques to an unattended moving random dot pattern (the distractor) positioned inside a neuron's receptive field while the animals attended to a second moving pattern positioned in the opposite hemifield. Responses to different directions of the distractor were modulated by the same factor (approximately 12%) as long as the attended direction remained unchanged. On the other hand, systematically changing the attended direction from a neuron's preferred to its anti-preferred direction caused a systematic change of the attentional modulation from an enhancement to a suppression, increasing directionality by about 20%.

CONCLUSIONS

The results show that (1) feature-based attention exerts a multiplicative modulation upon neuronal responses and that the strength of this modulation depends on the similarity between the attended feature and the cell's preferred feature, in line with the feature-similarity gain model, and (2) at the level of the neuronal population, feature-based attention increases the selectivity for attended features by increasing the responses of neurons preferring this feature value while decreasing responses of neurons tuned to the opposite feature value.

Contribution of Area MT to Stereoscopic Depth Perception

Due to the diversity of tuning properties in sensory cortex, only a fraction of neurons are engaged in a particular task. Characterizing the tuning properties of neurons that are functionally linked to behavior is essential for understanding how activity is "read out" from sensory maps to guide decisions. We recorded from middle temporal (MT) neurons while monkeys performed a depth discrimination task, and we characterized the linkage between MT responses and behavioral choices. Trial-to-trial response fluctuations of MT neurons with odd-symmetric ("Near," "Far") disparity tuning were predictive of monkeys' choices, whereas responses of neurons with even-symmetric tuning were not. This result cannot be explained by neuronal sensitivity or any other response property of MT neurons that we examined but is simply explained by the task strategy that monkeys learned during training. We suggest that this approach provides a physiological means to explore how task strategies are implemented in the brain.

Looking for faces: Attention modulates early occipitotemporal object processing

Looking for somebody's face in a crowd is one of the most important examples of visual search. For this goal, attention has to be directed to a well-defined perceptual category. When this categorically selective process starts is, however, still unknown. To this end, we used magnetoencephalography (MEG) recorded over right human occipitotemporal cortex to investigate the time course of attentional modulation of perceptual processes elicited by faces and by houses. The first face-distinctive MEG response was observed at 160-170 ms (M170). Nevertheless, attention did not start to modulate face processing before 190 ms. The first house-distinctive MEG activity was also found around 160-170 ms. However, house processing was not modulated by attention before 280 ms (90 ms later than face processing). Further analysis revealed that the attentional modulation of face processing is not due to later, for example, back-propagated activation of the M170 generator. Rather, subsequent stages of occipitotemporal object processing were modulated in a category-specific manner and with preferential access to face processing.

Attentional load associated with performing and stabilizing a between-persons coordination of rhythmic limb movements

This study addressed the issue of intentional stabilization of between-persons coordination patterns (in-phase/isodirectional and anti-phase/non-isodirectional) and the attentional cost incurred by the nervous system in maintaining and further stabilizing these coordination patterns. Five pairs of participants performed in-phase and anti-phase interpersonal coordination patterns in dual-task conditions (coordination+RT task). Results showed that: (1) isodirectional pattern (in-phase) was more stable than non-isodirectional pattern (anti-phase), (2) both iso- and non-isodirectional pattern were stabilized intentionally, (3) RT was lower for the isodirectional pattern (i.e., the most stable), and (4) attentional manipulation led to a trade-off between pattern stability and RT performance. These results suggest that performing between-persons coordination patterns incurs a central cost that depends on the coupling strength between the limbs. These findings are consistent with the previous studies in intrapersonal coordination.

Emergence of neural integration in the head-direction system by visual supervision

Head-direction (HD) cells in subcortical areas of the mammalian brain are tuned to a particular head direction in space; a population of such neurons forms a neural compass that may be relevant for spatial navigation. The development of neural circuits constituting the head-direction system is poorly understood. Inspired by electrophysiological experiments about the role of recurrent synaptic connections, we investigate a learning rule that teaches neurons to amplify feed-forward inputs. We simulate random head movements of a rat, during which neurons receive both visual and vestibular (head-velocity) inputs. Remarkably, as recurrent connections learn to amplify exclusively the visual inputs, a neural network emerges that performs spatio-temporal integration. That is, during head movements in darkness, neurons resemble HD cells by maintaining a fixed tuning to head direction. The proposed learning rule exhibits similarities with known forms of anti-Hebbian synaptic plasticity. We conclude that selective amplification could serve as a general principle for the synaptic development of multimodal feedback circuits in the brain.

Task relevance enhances early transient and late slow-wave activity of distributed cortical sources

The primary purpose of these studies was to link together concepts related to attention/working memory and feedforward/feedback activity using MEG response profiles obtained in humans. Similar to recent studies of attention in monkeys, we show early "spike-like" activity (<200 ms poststimulus), most likely reflecting an early transient excitatory response mixed with feedback influences, followed by "slow-wave" activity (>200 ms poststimulus) in MEG cortical response profiles evoked by a visual working memory task. We experimentally dissociated the early transient activity from the later sustained activity (predominantly feedback) by conducting an auditory size classification task. Words, representing common objects, evoked activity in occipital cortex (presumably due to imagery) even though visual stimuli were not present in this task. The initial "spike" was absent from the response profile obtained from occipital cortex, leaving only "slow-wave" activity, thereby allowing us to characterize or profile feedback activity in this situation. Attention or task relevance enhanced the initial "spike" and "slow-wave" activity in visually responsive areas. Prefrontal activity, along the superior frontal sulcus, evoked by the working memory task, was active later in time than initial activity in visual cortex and later than the earliest effect of attention modulation in visual cortex.

Components of visual prior entry

The prior entry hypothesis contends that attention accelerates sensory processing, shortening the time to perception. Typical observations supporting the hypothesis may be explained equally well by response biases, changes in decision criteria, or sensory facilitation. In a series of experiments conducted to discriminate among the potential mechanisms, observers judged the simultaneity or temporal order of two stimuli, to one of which attention was oriented by exogenous, endogenous, gaze-directed, or multiple exogenous cues. The results suggest that prior entry effects are primarily caused by sensory facilitation and attentional modifications of the decision mechanism, with only a small part possibly due to an attention-dependent sensory acceleration.

Activity of neurons in cortical area MT during a memory for motion task

We recorded the activity of middle temporal (MT) neurons in 2 monkeys while they compared the directions of motion in 2 sequentially presented random-dot stimuli, sample and test, and reported them as the same or different by pressing one of 2 buttons. We found that MT neurons were active not only in response to the sample and test stimuli but also during the 1,500-ms delay separating them. Most neurons showed a characteristic pattern of activity consisting of a small burst of firing early in the delay, followed by a period of suppression and a subsequent increase in firing rate immediately preceding the presentation of the test stimulus. In a third of the neurons, the activity early in the delay not only reflected the direction of the sample stimulus, but was also related to the range of local directions it contained. During the middle of the delay the majority of neurons were suppressed, consistent with a gating mechanism that could be used to ignore task-irrelevant stimuli. Late in the delay, most neurons showed an increase in response, probably in anticipation of the upcoming test. Throughout most of the delay there was a directional signal in the population of MT neurons, manifested by higher firing rates following the sample moving in the antipreferred direction. Whereas some of these effects may be related to sensory adaptation, others are more likely to represent a more active task-related process. These results support the hypothesis that MT neurons actively participate in the successful execution of all aspects of the task requiring processing and remembering visual motion.

The allocation of attention during smooth pursuit eye movements

The spatial-temporal allocation of attention during smooth pursuit eye movements is poorly understood. In this chapter we review evidence showing that attention contributes to both saccades and smooth pursuit. We then discuss results from our own recent studies using a dual-task paradigm in which subjects pursued a moving stimulus and pressed a button when targets appeared in the periphery. The results from these studies are consistent with the hypothesis that the allocation of attention is biased to a location just in front of the pursuit stimulus and that this bias can be altered by pursuit velocity.

Attentional modulation in visual cortex depends on task timing

Paying attention to a stimulus selectively increases the ability to process it. For example, when subjects attend to a specific region of a visual scene, their sensitivity to changes at that location increases. A large number of studies describe the behavioural consequences and neurophysiological correlates of attending to spatial locations. There has, in contrast, been little study of the allocation of attention over time. Because subjects can anticipate predictable events with great temporal precision, it seems probable that they might dynamically shift their attention when performing a familiar perceptual task whose constraints changed over time. We trained monkeys to respond to a stimulus change where the probability of occurrence changed over time. Recording from area V4 of the visual cortex in these animals, we found that the modulation of neuronal responses changed according to the probability of the change occurring at that instant. Thus, we show that the attentional modulation of sensory neurons reflects a subject's anticipation of the timing of behaviourally relevant events.

Serial linkage of target selection for orienting and tracking eye movements

Many natural actions require the coordination of two different kinds of movements. How are targets chosen under these circumstances: do central commands instruct different movement systems in parallel, or does the execution of one movement activate a serial chain that automatically chooses targets for the other movement? We examined a natural eye tracking action that consists of orienting saccades and tracking smooth pursuit eye movements, and found strong physiological evidence for a serial strategy. Monkeys chose freely between two identical spots that appeared at different sites in the visual field and moved in orthogonal directions. If a saccade was evoked to one of the moving targets by microstimulation in either the frontal eye field (FEF) or the superior colliculus (SC), then the same target was automatically chosen for pursuit. Our results imply that the neural signals responsible for saccade execution can also act as an internal command of target choice for other movement systems.

Multiple uses of visual motion. The case for stability in sensory cortex

The problem of 'readout' from sensory maps has received considerable attention recently. Specifically, many experiments in different systems have suggested that the routing of sensory signals from cortical maps can be impressively flexible. In this review, we discuss many of the experiments addressing readout of motion signals from the middle temporal area (also known as V5) in the macaque monkey. We focus on two different types of output: perceptual reports (categorical decisions, usually) and motion-guided eye movements. We specifically consider situations in which multiple-motion vectors present in the stimulus are combined, as well as those in which one or more of the vectors in the stimulus is selected for output. The results of these studies suggest that in some situations multiple motions are vector averaged, while in others multiple vectors can be maintained. Interestingly, in most of the experiments producing a single (often average) vector, the output is a movement. However, many perceptual experiments involve the simultaneous processing of multiple-stimulus motions. One prosaic explanation for this pattern of apparently discrepant results is that different downstream structures impose different rules, in parallel, on the output from sensory maps such as the one in the middle temporal area. We also specifically discuss the case of motion opponency, a specific readout rule that has been posited to explain perceptual phenomena such as the waterfall illusion (motion aftereffect). We present evidence from a recent experiment showing that an opponent step must occur downstream from the middle temporal area itself. This observation is consistent with our proposal that significant processing need occur downstream from sensory structures. If a single output is to be used for multiple purposes, often at once, this necessitates a degree of task invariance on the sensory information present even at a relatively high level of cortical processing.

Filtering of neural signals by focused attention in the monkey prefrontal cortex

Prefrontal cortex is thought to be important in attention and awareness. Here we recorded the activity of prefrontal neurons in monkeys carrying out a focused attention task. Having directed attention to one location, monkeys monitored a stream of visual objects, awaiting a predefined target. Although neurons rarely discriminated between one non-target and another, they commonly discriminated between targets and non-targets. From the onset of the visual response, this target/non-target discrimination was effectively eliminated when the same objects appeared at an unattended location in the opposite visual hemifield. The results show that, in prefrontal cortex, filtering of ignored locations is strong, early and spatially global. Such filtering may be important in blindness to unattended signals--a conspicuous aspect of human selective attention.

Attentional modulation of behavioral performance and neuronal responses in middle temporal and ventral intraparietal areas of macaque monkey

Although many studies have demonstrated that neuronal responses are modulated by attention, the significance of this modulation for behavior is poorly understood. We recorded from neurons in the middle temporal (MT) and ventral intraparietal (VIP) areas in the visual cortex of two macaque monkeys while they performed a motion detection task under two conditions of spatial attention. The ability of the animals to detect the motion was reduced when they withdrew attention from the stimulus. Withdrawing attention also reduced neuronal responses to the motion in both the MT and VIP areas. To compare the neuronal and behavioral effects of attention, the amount of attentional modulation was expressed in units of stimulus strength. On average, attention modulated neuronal responses in MT less than needed to account for the attentional effect on behavior. The opposite was observed in VIP, where the average effect of attention on neuronal responses was greater than that needed to account for behavior. Similar results were obtained when the effects of attention on neuronal response and behavioral performance were compared using a parametric function of stimulus strength. Across neurons in both areas, attentional modulation of neuronal responses was more variable than, and uncorrelated with, attentional modulation of behavioral performance. These findings suggest that attention can alter the average relationship between neuronal activity in visual cortex and behavioral performance. Where this relationship is preserved may indicate which cortical regions are most closely associated with the behavior in a given task.

Learning increases stimulus salience in anterior inferior temporal cortex of the macaque

With experience, an object can become behaviorally relevant and thereby quickly attract our interest when presented in a visual scene. A likely site of these learning effects is anterior inferior temporal (aIT) cortex, where neurons are thought to participate in the filtering of irrelevant information out of complex visual displays. We trained monkeys to saccade consistently to one of two pictures in an array, in return for a reward. The array was constructed by pairing two stimuli, one of which elicited a good response from the cell when presented alone ("good" stimulus) and the other of which elicited a poor response ("poor" stimulus). The activity of aIT cells was recorded while monkeys learned to saccade to either the good or poor stimulus in the array. We found that neuronal responses to the array were greater (before the saccade occurred) when training reinforced a saccade to the good stimulus than when training reinforced a saccade to the poor stimulus. This difference was not present on incorrect trials, i.e., when saccades to the incorrect stimulus were made. Thus the difference in activity was correlated with performance. The response difference grew over the course of the recording session, in parallel with the improvement in performance. The response difference was not preceded by a difference in the baseline activity of the cells, unlike what was found in studies of cued visual search and working memory in aIT cortex. Furthermore, we found similar effects in a version of the task in which any of 10 possible pairs of stimuli, prelearned before the recording session, could appear on a given trial, thereby precluding a working memory strategy. The results suggest that increasing the behavioral significance of a stimulus through training alters the neural representation of that stimulus in aIT cortex. As a result, neurons responding to features of the relevant stimulus may suppress neurons responding to features of irrelevant stimuli.

Effects of set-size and selective spatial attention on motion processing

In order to investigate the effects of divided attention and selective spatial attention on motion processing, we obtained direction-of-motion thresholds using a stochastic motion display under various attentional manipulations and stimulus durations (100-600 ms). To investigate divided attention, we compared motion thresholds obtained when a single motion stimulus was presented in the visual field (set-size=1) to those obtained when the motion stimulus was presented amongst three confusable noise distractors (set-size=4). The magnitude of the observed detriment in performance with an increase in set-size from 1 to 4 could be accounted for by a simple decision model based on signal detection theory, which assumes that attentional resources are not limited in capacity. To investigate selective attention, we compared motion thresholds obtained when a valid pre-cue alerted the subject to the location of the to-be-presented motion stimulus to those obtained when no pre-cue was provided. As expected, the effect of pre-cueing was large when the visual field contained noise distractors, an effect we attribute to "noise reduction" (i.e. the pre-cue allows subjects to exclude irrelevant distractors that would otherwise impair performance). In the single motion stimulus display, we found a significant benefit of pre-cueing only at short durations (< or =150 ms), a result that can potentially be explained by a "time-to-orient" hypothesis (i.e. the pre-cue improves performance by eliminating the time it takes to orient attention to a peripheral stimulus at its onset, thereby increasing the time spent processing the stimulus). Thus, our results suggest that the visual motion system can analyze several stimuli simultaneously without limitations on sensory processing per se, and that spatial pre-cueing serves to reduce the effects of distractors and perhaps increase the effective processing time of the stimulus.