Shape selectivity in primate frontal eye field

Using Natural Scenes to Enhance our Understanding of the Cerebral Cortex's Role in Visual Search

Using natural scenes is an approach to studying the visual and eye movement systems approximating how these systems function in everyday life. This review examines the results from behavioral and neurophysiological studies using natural scene viewing in humans and monkeys. The use of natural scenes for the study of cerebral cortical activity is relatively new and presents challenges for data analysis. Methods and results from the use of natural scenes for the study of the visual and eye movement cortex are presented, with emphasis on new insights that this method provides enhancing what is known about these cortical regions from the use of conventional methods.

Frontal eye fields in macaque monkeys: prefrontal and premotor contributions to visually guided saccades

Neuronal spiking was sampled from the frontal eye field (FEF) and from the rostral part of area 6 that reaches to the superior limb of the arcuate sulcus, dorsal to the arcuate spur when present (F2vr) in macaque monkeys performing memory-guided saccades and visually guided saccades for visual search. Neuronal spiking modulation in F2vr resembled that in FEF in many but not all respects. A new consensus clustering algorithm of neuronal modulation patterns revealed that F2vr and FEF contain a greater variety of modulation patterns than previously reported. The areas differ in the proportions of visuomotor neuron types, the proportions of neurons discriminating a target from distractors during visual search, and the consistency of modulation patterns across tasks. However, between F2vr and FEF we found no difference in the magnitude of delay period activity, the timing of the peak discharge rate relative to saccades, or the time of search target selection. The observed similarities and differences between the 2 cortical regions contribute to other work establishing the organization of eye fields in the frontal lobe and may help explain why FEF in monkeys is identified within granular prefrontal area 8 but in humans is identified within agranular premotor area 6.

Dynamic and stable population coding of attentional instructions coexist in the prefrontal cortex

A large body of recent work suggests that neural representations in prefrontal cortex (PFC) are changing over time to adapt to task demands. However, it remains unclear whether and how such dynamic coding schemes depend on the encoded variable and are influenced by anatomical constraints. Using a cued attention task and multivariate classification methods, we show that neuronal ensembles in PFC encode and retain in working memory spatial and color attentional instructions in an anatomically specific manner. Spatial instructions could be decoded both from the frontal eye field (FEF) and the ventrolateral PFC (vlPFC) population, albeit more robustly from FEF, whereas color instructions were decoded more robustly from vlPFC. Decoding spatial and color information from vlPFC activity in the high-dimensional state space indicated stronger dynamics for color, across the cue presentation and memory periods. The change in the color code was largely due to rapid changes in the network state during the transition to the delay period. However, we found that dynamic vlPFC activity contained time-invariant color information within a low-dimensional subspace of neural activity that allowed for stable decoding of color across time. Furthermore, spatial attention influenced decoding of stimuli features profoundly in vlPFC, but less so in visual area V4. Overall, our results suggest that dynamic population coding of attentional instructions within PFC is shaped by anatomical constraints and can coexist with stable subspace coding that allows time-invariant decoding of information about the future target.

Visual Attention in the Prefrontal Cortex

Voluntary attention selects behaviorally relevant signals for further processing while filtering out distracter signals. Neural correlates of voluntary visual attention have been reported across multiple areas of the primate visual processing streams, with the earliest and strongest effects isolated in the prefrontal cortex. In this article, I review evidence supporting the hypothesis that signals guiding the allocation of voluntary attention emerge in areas of the prefrontal cortex and reach upstream areas to modulate the processing of incoming visual information according to its behavioral relevance. Areas located anterior and dorsal to the arcuate sulcus and the frontal eye fields produce signals that guide the allocation of spatial attention. Areas located anterior and ventral to the arcuate sulcus produce signals for feature-based attention. Prefrontal microcircuits are particularly suited to supporting voluntary attention because of their ability to generate attentional template signals and implement signal gating and their extensive connectivity with the rest of the brain. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Working Memory: From Neural Activity to the Sentient Mind

Working memory (WM) is the ability to maintain and manipulate information in the conscious mind over a timescale of seconds. This ability is thought to be maintained through the persistent discharges of neurons in a network of brain areas centered on the prefrontal cortex, as evidenced by neurophysiological recordings in nonhuman primates, though both the localization and the neural basis of WM has been a matter of debate in recent years. Neural correlates of WM are evident in species other than primates, including rodents and corvids. A specialized network of excitatory and inhibitory neurons, aided by neuromodulatory influences of dopamine, is critical for the maintenance of neuronal activity. Limitations in WM capacity and duration, as well as its enhancement during development, can be attributed to properties of neural activity and circuits. Changes in these factors can be observed through training-induced improvements and in pathological impairments. WM thus provides a prototypical cognitive function whose properties can be tied to the spiking activity of brain neurons. © 2021 American Physiological Society. Compr Physiol 11:1-41, 2021.

Comparative anatomy of the macaque and the human frontal oculomotor domain

In non-human primates, at the junction of the prefrontal with the premotor cortex, there is a sector designated as frontal eye field (FEF), involved in controlling oculomotor behavior and spatial attention. Evidence for at least two FEFs in humans is at the basis of the still open issue of the possible homologies between the macaque and the human frontal oculomotor system. In this review article we address this issue suggesting a new view solidly grounded on evidence from the last decade showing that, in macaques, the FEF is at the core of an oculomotor domain in which several distinct areas, including areas 45A and 45B, provide the substrate for parallel processing of different aspects of oculomotor behavior. Based on comparative considerations, we will propose a correspondence between some of the macaque and the human oculomotor fields, thus suggesting sharing of neural substrate for oculomotor control, gaze processing, and orienting attention in space. Accordingly, this article could contribute to settle some aspects of the so-called "enigma" of the human FEF anatomy.

Pseudosparse neural coding in the visual system of primates

When measuring sparseness in neural populations as an indicator of efficient coding, an implicit assumption is that each stimulus activates a different random set of neurons. In other words, population responses to different stimuli are, on average, uncorrelated. Here we examine neurophysiological data from four lobes of macaque monkey cortex, including V1, V2, MT, anterior inferotemporal cortex, lateral intraparietal cortex, the frontal eye fields, and perirhinal cortex, to determine how correlated population responses are. We call the mean correlation the pseudosparseness index, because high pseudosparseness can mimic statistical properties of sparseness without being authentically sparse. In every data set we find high levels of pseudosparseness ranging from 0.59-0.98, substantially greater than the value of 0.00 for authentic sparseness. This was true for synthetic and natural stimuli, as well as for single-electrode and multielectrode data. A model indicates that a key variable producing high pseudosparseness is the standard deviation of spontaneous activity across the population. Consistently high values of pseudosparseness in the data demand reconsideration of the sparse coding literature as well as consideration of the degree to which authentic sparseness provides a useful framework for understanding neural coding in the cortex.

Representation of shape, space, and attention in monkey cortex

Attentional deficits are core to numerous developmental, neurological, and psychiatric disorders. At the single-cell level, much knowledge has been garnered from studies of shape and spatial properties, as well as from numerous demonstrations of attentional modulation of those properties. Despite this wealth of knowledge of single-cell responses across many brain regions, little is known about how these cellular characteristics relate to population level representations and how such representations relate to behavior; in particular, how these cellular responses relate to the representation of shape, space, and attention, and how these representations differ across cortical areas and streams. Here we will emphasize the role of population coding as a missing link for connecting single-cell properties with behavior. Using a data-driven intrinsic approach to population decoding, we show that both 'what' and 'where' cortical visual streams encode shape, space, and attention, yet demonstrate striking differences in these representations. We suggest that both pathways fully process shape and space, but that differences in representation may arise due to their differing functions and input and output constraints. Moreover, differences in the effects of attention on shape and spatial population representations in the two visual streams suggest two distinct strategies: in a ventral area, attention or task demands modulate the population representations themselves (perhaps to expand or enhance one part at the expense of other parts) while in a dorsal area, at a population representation level, attention effects are weak and nearly non-existent, perhaps in order to maintain veridical representations needed for visuomotor control. We show that an intrinsic approach, as opposed to theory-driven and labeled approaches, is useful for understanding how representations develop and differ across brain regions. Most importantly, these approaches help link cellular properties more tightly with behavior, a much-needed step to better understand and interpret cellular findings and key to providing insights to improve interventions in human disorders.

Corollary Discharge Contributions to Perceptual Continuity Across Saccades

Our vision depends upon shifting our high-resolution fovea to objects of interest in the visual field. Each saccade displaces the image on the retina, which should produce a chaotic scene with jerks occurring several times per second. It does not. This review examines how an internal signal in the primate brain (a corollary discharge) contributes to visual continuity across saccades. The article begins with a review of evidence for a corollary discharge in the monkey and evidence from inactivation experiments that it contributes to perception. The next section examines a specific neuronal mechanism for visual continuity, based on corollary discharge that is referred to as visual remapping. Both the basic characteristics of this anticipatory remapping and the factors that control it are enumerated. The last section considers hypotheses relating remapping to the perceived visual continuity across saccades, including remapping's contribution to perceived visual stability across saccades.

Visual stimulus-driven functional organization of macaque prefrontal cortex

The extent to which the major subdivisions of prefrontal cortex (PFC) can be functionally partitioned is unclear. In approaching the question, it is often assumed that the organization is task dependent. Here we use fMRI to show that PFC can respond in a task-independent way, and we leverage these responses to uncover a stimulus-driven functional organization. The results were generated by mapping the relative location of responses to faces, bodies, scenes, disparity, color, and eccentricity in four passively fixating macaques. The results control for individual differences in functional architecture and provide the first account of a systematic visual stimulus-driven functional organization across PFC. Responses were focused in dorsolateral PFC (DLPFC), in the ventral prearcuate region; and in ventrolateral PFC (VLPFC), extending into orbital PFC. Face patches were in the VLPFC focus and were characterized by a striking lack of response to non-face stimuli rather than an especially strong response to faces. Color-biased regions were near but distinct from face patches. One scene-biased region was consistently localized with different contrasts and overlapped the disparity-biased region to define the DLPFC focus. All visually responsive regions showed a peripheral visual-field bias. These results uncover an organizational scheme that presumably constrains the flow of information about different visual modalities into PFC.

Representation of Spatial and Feature Information in the Monkey Dorsal and Ventral Prefrontal Cortex

The primate prefrontal cortex (PFC) is critical for executive functions including working memory, task switching and response selection. The functional organization of this area has been a matter of debate over a period of decades. Early models proposed segregation of spatial and object information represented in working memory in the dorsal and ventral PFC, respectively. Other models emphasized the integrative ability of the entire PFC depending on task demands, not necessarily tied to working memory. An anterior-posterior hierarchy of specialization has also been speculated, in which progressively more abstract information is represented more anteriorly. Here we revisit this debate, updating these arguments in light of recent evidence in non-human primate neurophysiology studies. We show that spatial selectivity is predominantly represented in the posterior aspect of the dorsal PFC, regardless of training history and task performed. Objects of different features excite both dorsal and ventral prefrontal neurons, however neurons highly specialized for feature information are located predominantly in the posterior aspect of the ventral PFC. In accordance with neuronal selectivity, spatial working memory is primarily impaired by inactivation or lesion of the dorsal PFC and object working memory by ventral inactivation or lesion. Neuronal responses are plastic depending on task training but training too has dissociable effects on ventral and dorsal PFC, with the latter appearing to be more plastic. Despite the absence of an overall topography, evidence exists for the orderly localization of stimulus information at a sub-millimeter scale, within the dimensions of a cortical column. Unresolved questions remain, regarding the existence or not of a functional map at the areal and columnar scale, and the link between behavior and neuronal activity for different prefrontal subdivisions.

Gradual progression from sensory to task-related processing in cerebral cortex

The earliest stages of processing in cerebral cortex reflect a relatively faithful copy of sensory inputs, but intelligent behavior requires abstracting behaviorally relevant concepts and categories. We examined how this transformation progresses through multiple levels of the cortical hierarchy by comparing neural representations in six cortical areas in monkeys categorizing across three visual domains. We found that categorical abstraction occurred in a gradual fashion across the cortical hierarchy and reached an apex in prefrontal cortex. Categorical coding did not respect classical models of large-scale cortical organization. The dimensionality of neural population activity was reduced in parallel with these representational changes. Our results shed light on how raw sensory inputs are transformed into behaviorally relevant abstractions.

Somewhere along the cortical hierarchy, behaviorally relevant information is distilled from raw sensory inputs. We examined how this transformation progresses along multiple levels of the hierarchy by comparing neural representations in visual, temporal, parietal, and frontal cortices in monkeys categorizing across three visual domains (shape, motion direction, and color). Representations in visual areas middle temporal (MT) and V4 were tightly linked to external sensory inputs. In contrast, lateral prefrontal cortex (PFC) largely represented the abstracted behavioral relevance of stimuli (task rule, motion category, and color category). Intermediate-level areas, including posterior inferotemporal (PIT), lateral intraparietal (LIP), and frontal eye fields (FEF), exhibited mixed representations. While the distribution of sensory information across areas aligned well with classical functional divisions (MT carried stronger motion information, and V4 and PIT carried stronger color and shape information), categorical abstraction did not, suggesting these areas may participate in different networks for stimulus-driven and cognitive functions. Paralleling these representational differences, the dimensionality of neural population activity decreased progressively from sensory to intermediate to frontal cortex. This shows how raw sensory representations are transformed into behaviorally relevant abstractions and suggests that the dimensionality of neural activity in higher cortical regions may be specific to their current task.

Top-Down Control of Visual Attention by the Prefrontal Cortex. Functional Specialization and Long-Range Interactions

The ability to select information that is relevant to current behavioral goals is the hallmark of voluntary attention and an essential part of our cognition. Attention tasks are a prime example to study at the neuronal level, how task related information can be selectively processed in the brain while irrelevant information is filtered out. Whereas, numerous studies have focused on elucidating the mechanisms of visual attention at the single neuron and population level in the visual cortices, considerably less work has been devoted to deciphering the distinct contribution of higher-order brain areas, which are known to be critical for the employment of attention. Among these areas, the prefrontal cortex (PFC) has long been considered a source of top-down signals that bias selection in early visual areas in favor of the attended features. Here, we review recent experimental data that support the role of PFC in attention. We examine the existing evidence for functional specialization within PFC and we discuss how long-range interactions between PFC subregions and posterior visual areas may be implemented in the brain and contribute to the attentional modulation of different measures of neural activity in visual cortices.

How thoughts arise from sights: inferotemporal and prefrontal contributions to vision

We are rapidly approaching a comprehensive understanding of the neural mechanisms behind object recognition. How we use this knowledge of the visual world to plan and act is comparatively mysterious. To fill this gap, we must understand how visual representations are transformed within cognitive regions, and how these cognitive representations of visual information act back upon earlier sensory representations. Here, we summarize our current understanding of visual representation in inferotemporal cortex (IT) and prefrontal cortex (PFC), and the interactions between them. We emphasize the apparent consistency of visual representation in PFC across tasks, and suggest ways to leverage advances in our understanding of high-level vision to better understand cognitive processing.

Computational Architecture of the Parieto-Frontal Network Underlying Cognitive-Motor Control in Monkeys

The statistical structure of intrinsic parietal and parieto-frontal connectivity in monkeys was studied through hierarchical cluster analysis. Based on their inputs, parietal and frontal areas were grouped into different clusters, including a variable number of areas that in most instances occupied contiguous architectonic fields. Connectivity tended to be stronger locally: that is, within areas of the same cluster. Distant frontal and parietal areas were targeted through connections that in most instances were reciprocal and often of different strength. These connections linked parietal and frontal clusters formed by areas sharing basic functional properties. This led to five different medio-laterally oriented pillar domains spanning the entire extent of the parieto-frontal system, in the posterior parietal, anterior parietal, cingulate, frontal, and prefrontal cortex. Different information processing streams could be identified thanks to inter-domain connectivity. These streams encode fast hand reaching and its control, complex visuomotor action spaces, hand grasping, action/intention recognition, oculomotor intention and visual attention, behavioral goals and strategies, and reward and decision value outcome. Most of these streams converge on the cingulate domain, the main hub of the system. All of them are embedded within a larger eye–hand coordination network, from which they can be selectively set in motion by task demands.

Circuits for presaccadic visual remapping

Saccadic eye movements rapidly displace the image of the world that is projected onto the retinas. In anticipation of each saccade, many neurons in the visual system shift their receptive fields. This presaccadic change in visual sensitivity, known as remapping, was first documented in the parietal cortex and has been studied in many other brain regions. Remapping requires information about upcoming saccades via corollary discharge. Analyses of neurons in a corollary discharge pathway that targets the frontal eye field (FEF) suggest that remapping may be assembled in the FEF's local microcircuitry. Complementary data from reversible inactivation, neural recording, and modeling studies provide evidence that remapping contributes to transsaccadic continuity of action and perception. Multiple forms of remapping have been reported in the FEF and other brain areas, however, and questions remain about the reasons for these differences. In this review of recent progress, we identify three hypotheses that may help to guide further investigations into the structure and function of circuits for remapping.

Learning a New Selection Rule in Visual and Frontal Cortex

How do you make a decision if you do not know the rules of the game? Models of sensory decision-making suggest that choices are slow if evidence is weak, but they may only apply if the subject knows the task rules. Here, we asked how the learning of a new rule influences neuronal activity in the visual (area V1) and frontal cortex (area FEF) of monkeys. We devised a new icon-selection task. On each day, the monkeys saw 2 new icons (small pictures) and learned which one was relevant. We rewarded eye movements to a saccade target connected to the relevant icon with a curve. Neurons in visual and frontal cortex coded the monkey's choice, because the representation of the selected curve was enhanced. Learning delayed the neuronal selection signals and we uncovered the cause of this delay in V1, where learning to select the relevant icon caused an early suppression of surrounding image elements. These results demonstrate that the learning of a new rule causes a transition from fast and random decisions to a more considerate strategy that takes additional time and they reveal the contribution of visual and frontal cortex to the learning process.

Neural representation for object recognition in inferotemporal cortex

We suggest that population representation of objects in inferotemporal cortex lie on a continuum between a purely structural, parts-based description and a purely holistic description. The intrinsic dimensionality of object representation is estimated to be around 100, perhaps with lower dimensionalities for object representations more toward the holistic end of the spectrum. Cognitive knowledge in the form of semantic information and task information feed back to inferotemporal cortex from perirhinal and prefrontal cortex respectively, providing high-level multimodal-based expectations that assist in the interpretation of object stimuli. Integration of object information across eye movements may also contribute to object recognition through a process of active vision.

Numerosity processing is context driven even in the subitizing range: An fMRI study

Numerical judgments are involved in almost every aspect of our daily life. They are carried out so efficiently that they are often considered to be automatic and innate. However, numerosity of non-symbolic stimuli is highly correlated with its continuous properties (e.g., density, area), and so it is hard to determine whether numerosity and continuous properties rely on the same mechanism. Here we examined the behavioral and neuronal mechanisms underlying such judgments. We scanned subjects' hemodynamic responses to a numerosity comparison task and to a surface area comparison task. In these tasks, numerical and continuous magnitudes could be either congruent or incongruent. Behaviorally, an interaction between the order of the tasks and the relevant dimension modulated the congruency effects. Continuous magnitudes always interfered with numerosity comparison. Numerosity, on the other hand, interfered with the surface area comparison only when participants began with the numerosity task. Hemodynamic activity showed that context (induced by task order) determined the neuronal pathways in which the dimensions were processed. Starting with the numerosity task led to enhanced activity in the right hemisphere, while starting with the continuous task led to enhanced left hemisphere activity. Continuous magnitudes processing relied on activation of the frontal eye field and the post-central gyrus. Processing of numerosities, on the other hand, relied on deactivation of these areas, suggesting active suppression of the continuous dimension. Accordingly, we suggest that numerosities, even in the subitizing range, are not always processed automatically; their processing depends on context and task demands.

Different target-discrimination times can be followed by the same saccade-initiation timing in different stimulus conditions during visual searches

The neuronal processes that underlie visual searches can be divided into two stages: target discrimination and saccade preparation/generation. This predicts that the length of time of the prediscrimination stage varies according to the search difficulty across different stimulus conditions, whereas the length of the latter postdiscrimination stage is stimulus invariant. However, recent studies have suggested that the length of the postdiscrimination interval changes with different stimulus conditions. To address whether and how the visual stimulus affects determination of the postdiscrimination interval, we recorded single-neuron activity in the lateral intraparietal area (LIP) when monkeys (Macaca fuscata) performed a color-singleton search involving four stimulus conditions that differed regarding luminance (Bright vs. Dim) and target-distractor color similarity (Easy vs. Difficult). We specifically focused on comparing activities between the Bright-Difficult and Dim-Easy conditions, in which the visual stimuli were considerably different, but the mean reaction times were indistinguishable. This allowed us to examine the neuronal activity when the difference in the degree of search speed between different stimulus conditions was minimal. We found that not only prediscrimination but also postdiscrimination intervals varied across stimulus conditions: the postdiscrimination interval was longer in the Dim-Easy condition than in the Bright-Difficult condition. Further analysis revealed that the postdiscrimination interval might vary with stimulus luminance. A computer simulation using an accumulation-to-threshold model suggested that the luminance-related difference in visual response strength at discrimination time could be the cause of different postdiscrimination intervals.

Saliency and saccade encoding in the frontal eye field during natural scene search

The frontal eye field (FEF) plays a central role in saccade selection and execution. Using artificial stimuli, many studies have shown that the activity of neurons in the FEF is affected by both visually salient stimuli in a neuron's receptive field and upcoming saccades in a certain direction. However, the extent to which visual and motor information is represented in the FEF in the context of the cluttered natural scenes we encounter during everyday life has not been explored. Here, we model the activities of neurons in the FEF, recorded while monkeys were searching natural scenes, using both visual and saccade information. We compare the contribution of bottom-up visual saliency (based on low-level features such as brightness, orientation, and color) and saccade direction. We find that, while saliency is correlated with the activities of some neurons, this relationship is ultimately driven by activities related to movement. Although bottom-up visual saliency contributes to the choice of saccade targets, it does not appear that FEF neurons actively encode the kind of saliency posited by popular saliency map theories. Instead, our results emphasize the FEF's role in the stages of saccade planning directly related to movement generation.

Shape beyond recognition: form-derived directionality and its effects on visual attention and motion perception

The shape of an object restricts its movements and therefore its future location. The rules governing selective sampling of the environment likely incorporate any available data, including shape, that provide information about where important things are going to be in the near future so that the object can be located, tracked, and sampled for information. We asked people to assess in which direction several novel objects pointed or directed them. With independent groups of people, we investigated whether their attention and sense of motion were systematically biased in this direction. Our work shows that nearly any novel object has intrinsic directionality derived from its shape. This shape information is swiftly and automatically incorporated into the allocation of overt and covert visual orienting and the detection of motion, processes that themselves are inherently directional. The observed connection between form and space suggests that shape processing goes beyond recognition alone and may help explain why shape is a relevant dimension throughout the visual brain.

Shape selectivity and remapping in dorsal stream visual area LIP

We explore the visual world by making rapid eye movements (saccades) to focus on objects and locations of interest. Despite abrupt retinal image shifts, we see the world as stable. Remapping contributes to visual stability by updating the internal image with every saccade. Neurons in macaque lateral intraparietal cortex (LIP) and other brain areas update information about salient locations around the time of a saccade. The depth of information transfer remains to be thoroughly investigated. Area LIP, as part of the dorsal visual stream, is regarded as a spatially selective area, yet there is evidence that LIP neurons also encode object features. We sought to determine whether LIP remaps shape information. This knowledge is important for understanding what information is retained from each glance. We identified 82 remapping neurons. First, we presented shapes within the receptive field and tested for shape selectivity in a fixation task. Among the remapping neurons, 28 neurons (34%) were selective for shape. Second, we presented the same shapes in the future location of the receptive field around the time of the saccade and tested for shape selectivity during remapping. Thirty-one (38%) neurons were selective for shape. Of 11 neurons that were shape selective in both tasks, 5 showed significant correlation between shape selectivity in the two tasks. Across the population, there was a weak but significant correlation between responses to shape in the two tasks. Our results provide neurophysiological evidence that remapped responses in area LIP can encode shape information as well as spatial information.

Interaction between dorsal and ventral processing streams: where, when and how?

The execution of complex visual, auditory, and linguistic behaviors requires a dynamic interplay between spatial ('where/how') and non-spatial ('what') information processed along the dorsal and ventral processing streams. However, while it is acknowledged that there must be some degree of interaction between the two processing networks, how they interact, both anatomically and functionally, is a question which remains little explored. The current review examines the anatomical, temporal, and behavioral evidence regarding three potential models of dual stream interaction: (1) computations along the two pathways proceed independently and in parallel, reintegrating within shared target brain regions; (2) processing along the separate pathways is modulated by the existence of recurrent feedback loops; and (3) information is transferred directly between the two pathways at multiple stages and locations along their trajectories.

Differential frontal involvement in shifts of internal and perceptual attention

BACKGROUND

Perceptual attention enhances the processing of items in the environment, whereas internal attention enhances processing of items encoded in visual working memory. In perceptual and internal attention cueing paradigms, cues indicate the to-be-probed item before (pre-cueing) or after (retro-cueing) the memory display, respectively. Pre- and retro-cues confer similar behavioral accuracy benefits (pre-: 14-19%, retro-: 11-17%) and neuroimaging data show that they activate overlapping frontoparietal networks. Yet reports of behavioral and neuroimaging differences suggest that pre- and retro-cueing differentially recruit frontal and parietal cortices (Lepsien and Nobre, 2006).

OBJECTIVE/HYPOTHESIS

This study examined whether perceptual and internal attention are equally disrupted by neurostimulation to frontal and parietal cortices. We hypothesized that neurostimulation applied to frontal cortex would disrupt internal attention to a greater extent than perceptual attention.

METHODS

Cathodal transcranial direct current stimulation (tDCS) was applied to frontal or parietal cortices. After stimulation, participants completed a change detection task coupled with either pre- or retro-cues.

RESULTS

Cathodal tDCS across site (frontal, parietal) hindered performance. However, frontal tDCS had a greater negative impact on the retro-cued trials demonstrating greater frontal involvement during shifts of internal attention.

CONCLUSIONS

These results complement the neuroimaging data and provide further evidence suggesting that perceptual and internal attention are not identical processes. We conclude that although internal and perceptual attention are mediated by similar frontoparietal networks, the weight of contribution of these structures differs, with internal attention relying more heavily on the frontal cortex.

Persistent spatial information in the frontal eye field during object-based short-term memory

Spatial attention is known to gate entry into visual short-term memory, and some evidence suggests that spatial signals may also play a role in binding features or protecting object representations during memory maintenance. To examine the persistence of spatial signals during object short-term memory, the activity of neurons in the frontal eye field (FEF) of macaque monkeys was recorded during an object-based delayed match-to-sample task. In this task, monkeys were trained to remember an object image over a brief delay, regardless of the locations of the sample or target presentation. FEF neurons exhibited visual, delay, and target period activity, including selectivity for sample location and target location. Delay period activity represented the sample location throughout the delay, despite the irrelevance of spatial information for successful task completion. Furthermore, neurons continued to encode sample position in a variant of the task in which the matching stimulus never appeared in their response field, confirming that FEF maintains sample location independent of subsequent behavioral relevance. FEF neurons also exhibited target-position-dependent anticipatory activity immediately before target onset, suggesting that monkeys predicted target position within blocks. These results show that FEF neurons maintain spatial information during short-term memory, even when that information is irrelevant for task performance.

What and where information in the caudate tail guides saccades to visual objects

We understand the world by making saccadic eye movements to various objects. However, it is unclear how a saccade can be aimed at a particular object, because two kinds of visual information, what the object is and where it is, are processed separately in the dorsal and ventral visual cortical pathways. Here, we provide evidence suggesting that a basal ganglia circuit through the tail of the monkey caudate nucleus (CDt) guides such object-directed saccades. First, many CDt neurons responded to visual objects depending on where and what the objects were. Second, electrical stimulation in the CDt induced saccades whose directions matched the preferred directions of neurons at the stimulation site. Third, many CDt neurons increased their activity before saccades directed to the preferred objects and directions of the neurons in a free-viewing condition. Our results suggest that CDt neurons receive both "what" and "where" information and guide saccades to visual objects.

Frontal eye field neurons assess visual stability across saccades

The image on the retina may move because the eyes move, or because something in the visual scene moves. The brain is not fooled by this ambiguity. Even as we make saccades, we are able to detect whether visual objects remain stable or move. Here we test whether this ability to assess visual stability across saccades is present at the single-neuron level in the frontal eye field (FEF), an area that receives both visual input and information about imminent saccades. Our hypothesis was that neurons in the FEF report whether a visual stimulus remains stable or moves as a saccade is made. Monkeys made saccades in the presence of a visual stimulus outside of the receptive field. In some trials, the stimulus remained stable, but in other trials, it moved during the saccade. In every trial, the stimulus occupied the center of the receptive field after the saccade, thus evoking a reafferent visual response. We found that many FEF neurons signaled, in the strength and timing of their reafferent response, whether the stimulus had remained stable or moved. Reafferent responses were tuned for the amount of stimulus translation, and, in accordance with human psychophysics, tuning was better (more prevalent, stronger, and quicker) for stimuli that moved perpendicular, rather than parallel, to the saccade. Tuning was sometimes present as well for nonspatial transaccadic changes (in color, size, or both). Our results indicate that FEF neurons evaluate visual stability during saccades and may be general purpose detectors of transaccadic visual change.

Dorsal premotor cortex is involved in switching motor plans

Previous studies have shown that neural activity in primate dorsal premotor cortex (PMd) can simultaneously represent multiple potential movement plans, and that activity related to these movement options is modulated by their relative subjective desirability. These findings support the hypothesis that decisions about actions are made through a competition within the same circuits that guide the actions themselves. This hypothesis further predicts that the very same cells that guide initial decisions will continue to update their activities if an animal changes its mind. For example, if a previously selected movement option suddenly becomes unavailable, the correction will be performed by the same cells that selected the initial movement, as opposed to some different group of cells responsible for online guidance. We tested this prediction by recording neural activity in the PMd of a monkey performing an instructed-delay reach selection task. In the task, two targets were simultaneously presented and their border styles indicated whether each would be worth 1, 2, or 3 juice drops. In a random subset of trials (FREE), the monkey was allowed a choice while in the remaining trials (FORCED) one of the targets disappeared at the time of the GO signal. In FORCED-LOW trials the monkey was forced to move to the less valuable target and started moving either toward the new target (Direct) or toward the target that vanished and then curved to reach the remaining one (Curved). Prior to the GO signal, PMd activity clearly reflected the monkey's subjective preference, predicting his choices in FREE trials even with equally valued options. In FORCED-LOW trials, PMd activity reflected the switch of the monkey's plan as early as 100 ms after the GO signal, well before movement onset (MO). This confirms that the activity is not related to feedback from the movement itself, and suggests that PMd continues to participate in action selection even when the animal changes its mind on-line. These findings were reproduced by a computational model suggesting that switches between action plans can be explained by the same competition process responsible for initial decisions.

Shape effects on reflexive spatial attention are driven by the dorsal stream

In a modified reflexive spatial attention paradigm, when the cue and the target are at the same spatial location, processing of the target is faster when the cue and the target have different shapes compared to same (shape effect). Recent physiological findings suggest distinct population level encoding of shape in ventral versus dorsal cortical visual streams in monkeys. In human observers, we tested whether the effect of shape on reflexive spatial attention could be attributed to ventral and/or dorsal stream encoding of shape. In the modified reflexive spatial attention paradigm, we varied the shapes of the cue and target. Based on data from monkey physiology (Lehky & Sereno, 2007), we selected four pairs of cue and target shapes. In some pairs, cue and target were similarly encoded (similar encoding distance) by a population of cells in the lateral intraparietal cortex, a dorsal stream area, but more dissimilarly encoded (having a greater encoding distance) by a population of cells in the anterior inferotemporal cortex (AIT), a ventral stream area. In other pairs, cue and target were similarly encoded in AIT and had greater dissimilarity in LIP encoding. We found that pairs of cue and target with greater dissimilarity in LIP encoding produced larger and more consistent shape effects up to a cue to target onset asynchrony (CTOA) of 450 ms. The shape effects for cue and target pairs with greater dissimilarity in AIT encoding were smaller and inconsistent, suggesting that shape effects in reflexive spatial attention are largely driven by the dorsal stream.

Searching for the neural mechanisms of feature-based attention in the primate brain

In this issue, two studies, one by Zhou and Desimone and another by Cohen and Maunsell, provide new insights into the mechanisms of feature-based attention (FBA). The former demonstrates a new role of the frontal eye fields in the origins of FBA and the latter shows that FBA is coordinated across both hemispheres.

Feature-based attention in the frontal eye field and area V4 during visual search

When we search for a target in a crowded visual scene, we often use the distinguishing features of the target, such as color or shape, to guide our attention and eye movements. To investigate the neural mechanisms of feature-based attention, we simultaneously recorded neural responses in the frontal eye field (FEF) and area V4 while monkeys performed a visual search task. The responses of cells in both areas were modulated by feature attention, independent of spatial attention, and the magnitude of response enhancement was inversely correlated with the number of saccades needed to find the target. However, an analysis of the latency of sensory and attentional influences on responses suggested that V4 provides bottom-up sensory information about stimulus features, whereas the FEF provides a top-down attentional bias toward target features that modulates sensory processing in V4 and that could be used to guide the eyes to a searched-for target.

Stimulus selectivity in dorsal and ventral prefrontal cortex after training in working memory tasks

The prefrontal cortex is known to represent different types of information in working memory. Contrasting theories propose that the dorsal and ventral regions of the lateral prefrontal cortex are innately specialized for the representation of spatial and nonspatial information, respectively (Goldman-Rakic, 1996), or that the two regions are shaped by the demands of cognitive tasks imposed on them (Miller, 2000). To resolve this issue, we recorded from neurons in the two regions, before and at multiple stages of training monkeys on visual working memory tasks. Before training, substantial functional differences were present between the two regions. Dorsal prefrontal cortex exhibited higher overall responsiveness to visual stimuli and higher selectivity for spatial information. After training, stimulus selectivity generally decreased, although dorsal prefrontal cortex retained higher spatial selectivity regardless of task performed. Ventral prefrontal cortex appeared to be affected to a greater extent by the nature of the task. Our results indicate that regional specialization for stimulus selectivity is present in the primate prefrontal cortex regardless of training. Dorsal areas of the prefrontal cortex are inherently organized to represent spatial information, and training has little influence on this spatial bias. Ventral areas are biased toward nonspatial information, although they are more influenced by training both in terms of activation and changes in stimulus selectivity.

Magnetoencephalographic signatures of visual form and motion binding

This study investigates neural magneto-encephalographic (MEG) correlates of visual form and motion binding. Steady-state visual evoked fields (SSVEF) were recorded in MEG while observers reported their bound or unbound perception of moving bars arranged in a square shape. By using pairs of oscillating vertical and horizontal bars, "frequency-tagged" at f1 and f2, we identified a region with enhanced sustained power at 2f1+2f2 intermodulation frequency correlated with perceptual reports. Intermodulation power is more important during perceptual form/motion integration than during the perceptual segmentation of the stimulus into individual component motions, indicating that intermodulation frequency power is a neuromarker of form/motion integration. Source reconstruction of cortical activities at the relevant frequencies further reveals well segregated activity in the occipital lobe at the fundamental of the stimulation, f1 and f2, widely spread activity at 2f1 and 2f2 and a focal activity in the medial part of the right precentral sulcus region at the intermodulation component, 2f1+2f2. The present findings indicate that motion tagging provides a powerful way of investigating the processes underlying visual form/motion binding non-invasively in humans.

Visual stability

Our vision remains stable even though the movements of our eyes, head and bodies create a motion pattern on the retina. One of the most important, yet basic, feats of the visual system is to correctly determine whether this retinal motion is owing to real movement in the world or rather our own self-movement. This problem has occupied many great thinkers, such as Descartes and Helmholtz, at least since the time of Alhazen. This theme issue brings together leading researchers from animal neurophysiology, clinical neurology, psychophysics and cognitive neuroscience to summarize the state of the art in the study of visual stability. Recently, there has been significant progress in understanding the limits of visual stability in humans and in identifying many of the brain circuits involved in maintaining a stable percept of the world. Clinical studies and new experimental methods, such as transcranial magnetic stimulation, now make it possible to test the causal role of different brain regions in creating visual stability and also allow us to measure the consequences when the mechanisms of visual stability break down.

Population coding of visual space: comparison of spatial representations in dorsal and ventral pathways

Although the representation of space is as fundamental to visual processing as the representation of shape, it has received relatively little attention from neurophysiological investigations. In this study we characterize representations of space within visual cortex, and examine how they differ in a first direct comparison between dorsal and ventral subdivisions of the visual pathways. Neural activities were recorded in anterior inferotemporal cortex (AIT) and lateral intraparietal cortex (LIP) of awake behaving monkeys, structures associated with the ventral and dorsal visual pathways respectively, as a stimulus was presented at different locations within the visual field. In spatially selective cells, we find greater modulation of cell responses in LIP with changes in stimulus position. Further, using a novel population-based statistical approach (namely, multidimensional scaling), we recover the spatial map implicit within activities of neural populations, allowing us to quantitatively compare the geometry of neural space with physical space. We show that a population of spatially selective LIP neurons, despite having large receptive fields, is able to almost perfectly reconstruct stimulus locations within a low-dimensional representation. In contrast, a population of AIT neurons, despite each cell being spatially selective, provide less accurate low-dimensional reconstructions of stimulus locations. They produce instead only a topologically (categorically) correct rendition of space, which nevertheless might be critical for object and scene recognition. Furthermore, we found that the spatial representation recovered from population activity shows greater translation invariance in LIP than in AIT. We suggest that LIP spatial representations may be dimensionally isomorphic with 3D physical space, while in AIT spatial representations may reflect a more categorical representation of space (e.g., "next to" or "above").

Influence and limitations of popout in the selection of salient visual stimuli by area V4 neurons

The neural mechanism of bottom-up attention and its relationship to top-down attention are poorly understood. Visual stimuli that differ from others in their component features are salient and tend to draw attention in a bottom-up manner. "Popout" stimuli differ uniformly from surrounding items and are more easily detected than stimuli composed of a conjunction of surrounding features. We compared the responses of single area V4 neurons to popout and conjunction stimuli appearing within the classical receptive field (CRF) and found that their responses are modulated by popout. This selectivity was more robust when larger numbers of surrounding items and multiple features were included in the display, and it was absent when only a few items were presented immediately outside the CRF. In addition, the popout modulation of V4 activity was eliminated when top-down attention was directed to locations outside of the CRFs during saccade preparation, indicating that the salience of popout stimuli is not sufficient to drive selection by V4 neurons. These results demonstrate that neurons in feature-selective cortex are influenced by bottom-up attention, but that this influence is limited by top-down attention.

Frontal eye field activity enhances object identification during covert visual search

We investigated the link between neuronal activity in the frontal eye field (FEF) and the enhancement of visual processing associated with covert spatial attention in the absence of eye movements. We correlated activity recorded in the FEF of monkeys manually reporting the identity of a visual search target to performance accuracy and reaction time. Monkeys were cued to the most probable target location with a cue array containing a popout color singleton. Neurons exhibited spatially selective responses for the popout cue stimulus and for the target of the search array. The magnitude of activity related to the location of the cue prior to the presentation of the search array was correlated with trends in behavioral performance across valid, invalid, and neutral cue trial conditions. However, the speed and accuracy of the behavioral report on individual trials were predicted by the magnitude of spatial selectivity related to the target to be identified, not for the spatial cue. A minimum level of selectivity was necessary for target detection and a higher level for target identification. Muscimol inactivation of FEF produced spatially selective perceptual deficits in the covert search task that were correlated with the effectiveness of the inactivation and were strongest on invalid cue trials that require an endogenous attention shift. These results demonstrate a strong functional link between FEF activity and covert spatial attention and suggest that spatial signals from FEF directly influence visual processing during the time that a stimulus to be identified is being processed by the visual system.

Ultra-rapid sensory responses in the human frontal eye field region

Most of what we know about the human frontal eye field (FEF) is extrapolated from studies in animals. There is ample evidence that this region is crucial for eye movements. However, evidence is accumulating that this region also plays a role in sensory processing and that it belongs to a "fast brain" system. We set out to investigate these issues in humans, using intracerebral recordings in patients with drug-refractory epilepsy. Event-related potential recordings were obtained from 11 epileptic patients from within the FEF region while they passed a series of visual and auditory perceptual tests. No eye movement was required. Ultra-rapid responses were observed, with mean onset latencies at 24 ms after stimulus to auditory stimuli and 45 ms to visual stimuli. Such early responses were compatible with cortical routes as assessed with simultaneous recordings in primary auditory and visual cortices. Components were modulated very early by the sensory characteristics of the stimuli, in the 30-60 ms period for auditory stimuli and in the 45-60 ms period for visual stimuli. Although the frontal lobes in humans are generally viewed as being involved in high-level cognitive processes, these results indicate that the human FEF is a remarkably quickly activated multimodal region that belongs to a network of low-level neocortical sensory areas.

Neural control of visual search by frontal eye field: effects of unexpected target displacement on visual selection and saccade preparation

The dynamics of visual selection and saccade preparation by the frontal eye field was investigated in macaque monkeys performing a search-step task combining the classic double-step saccade task with visual search. Reward was earned for producing a saccade to a color singleton. On random trials the target and one distractor swapped locations before the saccade and monkeys were rewarded for shifting gaze to the new singleton location. A race model accounts for the probabilities and latencies of saccades to the initial and final singleton locations and provides a measure of the duration of a covert compensation process-target-step reaction time. When the target stepped out of a movement field, noncompensated saccades to the original location were produced when movement-related activity grew rapidly to a threshold. Compensated saccades to the final location were produced when the growth of the original movement-related activity was interrupted within target-step reaction time and was replaced by activation of other neurons producing the compensated saccade. When the target stepped into a receptive field, visual neurons selected the new target location regardless of the monkeys' response. When the target stepped out of a receptive field most visual neurons maintained the representation of the original target location, but a minority of visual neurons showed reduced activity. Chronometric analyses of the neural responses to the target step revealed that the modulation of visually responsive neurons and movement-related neurons occurred early enough to shift attention and saccade preparation from the old to the new target location. These findings indicate that visual activity in the frontal eye field signals the location of targets for orienting, whereas movement-related activity instantiates saccade preparation.