A pure salience response in posterior parietal cortex

Using Natural Scenes to Enhance our Understanding of the Cerebral Cortex's Role in Visual Search

Using natural scenes is an approach to studying the visual and eye movement systems approximating how these systems function in everyday life. This review examines the results from behavioral and neurophysiological studies using natural scene viewing in humans and monkeys. The use of natural scenes for the study of cerebral cortical activity is relatively new and presents challenges for data analysis. Methods and results from the use of natural scenes for the study of the visual and eye movement cortex are presented, with emphasis on new insights that this method provides enhancing what is known about these cortical regions from the use of conventional methods.

Inversion of pop-out for a distracting feature dimension in monkey visual cortex

During visual search, it is important to reduce the interference of distracting objects in the scene. The neuronal responses elicited by the search target stimulus are typically enhanced. However, it is equally important to suppress the representations of distracting stimuli, especially if they are salient and capture attention. We trained monkeys to make an eye movement to a unique "pop-out" shape stimulus among an array of distracting stimuli. One of these distractors had a salient color that varied across trials and differed from the color of the other stimuli, causing it to also pop-out. The monkeys were able to select the pop-out shape target with high accuracy and actively avoided the pop-out color distractor. This behavioral pattern was reflected in the activity of neurons in area V4. Responses to the shape targets were enhanced, while the activity evoked by the pop-out color distractor was only briefly enhanced, directly followed by a sustained period of pronounced suppression. These behavioral and neuronal results demonstrate a cortical selection mechanism that rapidly inverts a pop-out signal to "pop-in" for an entire feature dimension thereby facilitating goal-directed visual search in the presence of salient distractors.

Neuronal Response to Reward and Luminance in Macaque LIP During Saccadic Choice

Recent work in decision neuroscience suggests that visual saliency can interact with reward-based choice, and the lateral intraparietal cortex (LIP) is implicated in this process. In this study, we recorded from LIP neurons while monkeys performed a two alternative choice task in which the reward and luminance associated with each offer were varied independently. We discovered that the animal's choice was dictated by the reward amount while the luminance had a marginal effect. In the LIP, neuronal activity corresponded well with the animal's choice pattern, in that a majority of reward-modulated neurons encoded the reward amount in the neuron's preferred hemifield with a positive slope. In contrast, compared to their responses to low luminance, an approximately equal proportion of luminance-sensitive neurons responded to high luminance with increased or decreased activity, leading to a much weaker population-level response. Meanwhile, in the non-preferred hemifield, the strength of encoding for reward amount and luminance was positively correlated, suggesting the integration of these two factors in the LIP. Moreover, neurons encoding reward and luminance were homogeneously distributed along the anterior-posterior axis of the LIP. Overall, our study provides further evidence supporting the neural instantiation of a priority map in the LIP in reward-based decisions.

Task relevance and negative reward modulate the disengagement deficit of patients with spatial neglect

Though motivational value is a recognized trigger of approach and avoidance behavior, less is known about the potential of reward to capture attention. We here explored whether positive or negative reward modulates the characteristic deficit of patients with left spatial neglect to disengage attention from an ipsilesional distracter. We built our study on recent observations showing that the disengagement deficit is exaggerated for distracters with target-defining features, indicating that task-relevance captures attention. Patients with left neglect and matched healthy controls were asked to react to lateralized, colored targets preceded by a peripheral cue. Crucially, the cue either possessed the color of the target and was thus task-relevant, or was followed by a positive, negative, or neutral symbolic reward. Neglect patients only exhibited a disengagement deficit when cues were task-relevant or were followed by a negative reward. This finding indicates that attentional selection is driven by task-relevance and negative reward, possibly through interactions between limbic and attention networks.

Resting-state theta activity is linked to information content-specific coding levels during response inhibition

The neurophysiological processes underlying the inhibition of impulsive responses have been studied extensively. While also the role of theta oscillations during response inhibition is well examined, the relevance of resting-state theta activity for inhibitory control processes is largely unknown. We test the hypothesis that there are specific relationships between resting-state theta activity and sensory/motor coding levels during response inhibition using EEG methods. We show that resting theta activity is specifically linked to the stimulus-related fraction of neurophysiological activity in specific time windows during motor inhibition. In contrast, concomitantly coded processes related to decision-making or response selection as well as the behavioral inhibition performance were not associated with resting theta activity. Even at the peak of task-related theta power, where task-related theta activity and resting theta activity differed the most, there was still predominantly a significant correlation between both types of theta activity. This suggests that aspects similar to resting dynamics are evident in the proportion of inhibition-related neurophysiological activity that reflects an “alarm” signal, whose function is to process and indicate the need for cognitive control. Thus, specific aspects of task-related theta power may build upon resting theta activity when cognitive control is necessary.

Time-varying measures of cerebral network centrality correlate with visual saliency during movie watching

The extensive development of graph-theoretic analysis for functional connectivity has revealed the multifaceted characteristics of brain networks. Network centralities identify the principal functional regions, individual differences, and hub structure in brain networks. Neuroimaging studies using movie-watching have investigated brain function under naturalistic stimuli. Visual saliency is one of the promising measures for revealing cognition and emotions driven by naturalistic stimuli. This study investigated whether the visual saliency in movies was associated with network centrality. The study examined eigenvector centrality (EC), which is a measure of a region's influence in the brain network, and the participation coefficient (PC), which reflects the hub structure in the brain, was used for comparison. Static and time-varying EC and PC were analyzed by a parcel-based technique. While EC was correlated with brain activity in parcels in the visual and auditory areas during movie-watching, it was only correlated with parcels in the visual areas in the retinotopy task. In addition, high PC was consistently observed in parcels in the putative hub both during the tasks and the resting-state condition. Time-varying EC in the parietal parcels and time-varying PC in the primary sensory parcels significantly correlated with visual saliency in the movies. These results suggest that time-varying centralities in brain networks are distinctively associated with perceptual processing and subsequent higher processing of visual saliency.

Learning Invariant Object and Spatial View Representations in the Brain Using Slow Unsupervised Learning

First, neurophysiological evidence for the learning of invariant representations in the inferior temporal visual cortex is described. This includes object and face representations with invariance for position, size, lighting, view and morphological transforms in the temporal lobe visual cortex; global object motion in the cortex in the superior temporal sulcus; and spatial view representations in the hippocampus that are invariant with respect to eye position, head direction, and place. Second, computational mechanisms that enable the brain to learn these invariant representations are proposed. For the ventral visual system, one key adaptation is the use of information available in the statistics of the environment in slow unsupervised learning to learn transform-invariant representations of objects. This contrasts with deep supervised learning in artificial neural networks, which uses training with thousands of exemplars forced into different categories by neuronal teachers. Similar slow learning principles apply to the learning of global object motion in the dorsal visual system leading to the cortex in the superior temporal sulcus. The learning rule that has been explored in VisNet is an associative rule with a short-term memory trace. The feed-forward architecture has four stages, with convergence from stage to stage. This type of slow learning is implemented in the brain in hierarchically organized competitive neuronal networks with convergence from stage to stage, with only 4-5 stages in the hierarchy. Slow learning is also shown to help the learning of coordinate transforms using gain modulation in the dorsal visual system extending into the parietal cortex and retrosplenial cortex. Representations are learned that are in allocentric spatial view coordinates of locations in the world and that are independent of eye position, head direction, and the place where the individual is located. This enables hippocampal spatial view cells to use idiothetic, self-motion, signals for navigation when the view details are obscured for short periods.

Balancing Flexibility and Interference in Working Memory

Working memory is central to cognition, flexibly holding the variety of thoughts needed for complex behavior. Yet, despite its importance, working memory has a severely limited capacity, holding only three to four items at once. In this article, I review experimental and computational evidence that the flexibility and limited capacity of working memory reflect the same underlying neural mechanism. I argue that working memory relies on interactions between high-dimensional, integrative representations in the prefrontal cortex and structured representations in the sensory cortex. Together, these interactions allow working memory to flexibly maintain arbitrary representations. However, the distributed nature of working memory comes at the cost of causing interference between items in memory, resulting in a limited capacity. Finally, I discuss several mechanisms used by the brain to reduce interference and maximize the effective capacity of working memory. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

The functional roles of neural remapping in cortex

Remapping is a property of some cortical and subcortical neurons that update their responses around the time of an eye movement to account for the shift of stimuli on the retina due to the saccade. Physiologically, remapping is traditionally tested by briefly presenting a single stimulus around the time of the saccade and looking at the onset of the response and the locations in space to which the neuron is responsive. Here we suggest that a better way to understand the functional role of remapping is to look at the time at which the neural signal emerges when saccades are made across a stable scene. Based on data obtained using this approach, we suggest that remapping in the lateral intraparietal area is sufficient to play a role in maintaining visual stability across saccades, whereas in the frontal eye field, remapped activity carries information that affects future saccadic choices and, in a separate subset of neurons, is used to maintain a map of locations in the scene that have been previously fixated.

The roles of the lateral intraparietal area and frontal eye field in guiding eye movements in free viewing search behavior

The lateral intraparietal area (LIP) and frontal eye field (FEF) have been shown to play significant roles in oculomotor control, yet most studies have found that the two areas behave similarly. To identify the unique roles each area plays in guiding eye movements, we recorded 200 LIP neurons and 231 FEF neurons from four animals performing a free viewing visual foraging task. We analyzed how neuronal responses were modulated by stimulus identity and the animals' choice of where to make a saccade. We additionally analyzed the comodulation of the sensory signals and the choice signal to identify how the sensory signals drove the choice. We found a clearly defined division of labor: LIP provided a stable map integrating task rules and stimulus identity, whereas FEF responses were dynamic, representing more complex information and, just before the saccade, were integrated with task rules and stimulus identity to decide where to move the eye.NEW & NOTEWORTHY The lateral intrapareital area (LIP) and frontal eye field (FEF) are known to contribute to guiding eye movements, but little is known about the unique roles that each area plays. Using a free viewing visual search task, we found that LIP provides a stable map of the visual world, integrating task rules and stimulus identity. FEF activity is consistently modulated by more complex information but, just before the saccade, integrates all the information to make the final decision about where to move.

Multivoxel codes for representing and integrating acoustic features in human cortex

Using fMRI and multivariate pattern analysis, we determined whether spectral and temporal acoustic features are represented by independent or integrated multivoxel codes in human cortex. Listeners heard band-pass noise varying in frequency (spectral) and amplitude-modulation (AM) rate (temporal) features. In the superior temporal plane, changes in multivoxel activity due to frequency were largely invariant with respect to AM rate (and vice versa), consistent with an independent representation. In contrast, in posterior parietal cortex, multivoxel representation was exclusively integrated and tuned to specific conjunctions of frequency and AM features (albeit weakly). Direct between-region comparisons show that whereas independent coding of frequency weakened with increasing levels of the hierarchy, such a progression for AM and integrated coding was less fine-grained and only evident in the higher hierarchical levels from non-core to parietal cortex (with AM coding weakening and integrated coding strengthening). Our findings support the notion that primary auditory cortex can represent spectral and temporal acoustic features in an independent fashion and suggest a role for parietal cortex in feature integration and the structuring of sensory input.

An Integrative Framework for Sensory, Motor, and Cognitive Functions of the Posterior Parietal Cortex

Throughout the history of modern neuroscience, the parietal cortex has been associated with a wide array of sensory, motor, and cognitive functions. The use of non-human primates as a model organism has been instrumental in our current understanding of how areas in the posterior parietal cortex (PPC) modulate our perception and influence our behavior. In this Perspective, we highlight a series of influential studies over the last five decades examining the role of the PPC in visual perception and motor planning. We also integrate long-standing views of PPC functions with more recent evidence to propose a more general model framework to explain integrative sensory, motor, and cognitive functions of the PPC.

Detailed mapping of human habenula resting-state functional connectivity

The habenula (Hb) inhibits dopaminergic reward signaling in response to negative outcomes and has been linked to numerous functional domains relevant to mental health, including reward prediction, motivation, and aversion processing. Despite its important neuroscientific and clinical implications, however, the human Hb remains poorly understood due to its small size and the associated technical hurdles to in vivo functional magnetic resonance imaging (fMRI) investigation. Using high-resolution 3 T fMRI data from 68 healthy young adults acquired through the Human Connectome Project, we developed a rigorous approach for mapping the whole-brain resting-state functional connectivity of the human Hb. Our study combined an optimized strategy for defining subject-level connectivity seeds to maximize Hb blood-oxygen-level-dependent (BOLD) signal sensitivity with high-quality surface-based alignment for robust functional localization and cortical sensitivity. We identified significant positive Hb connectivity with: (i) conserved brainstem targets, including the dopaminergic ventral tegmental area, serotonergic raphe nuclei, and periaqueductal gray; (ii) subcortical structures related to reward and motor function, including the nucleus accumbens, dorsal striatum, pallidum, thalamus, and cerebellum; and (iii) cortical areas associated with the Salience Network and early sensory processing, including the dorsal anterior cingulate, anterior insula, and primary visual and auditory cortices. Hb connectivity was strongly biased towards task-positive brain regions, with weak or negative connectivity observed throughout the task-negative Default Mode Network. Our study provides a detailed characterization of Hb resting-state functional connectivity in healthy young adults, demonstrating both the feasibility and clinical potential of studying the human Hb using high-resolution 3 T fMRI.

The neural instantiation of a priority map

The term priority map is commonly used to describe a map of the visual scene, in which objects and locations are represented by their attentional priority, which itself is a combination of low-level salience and top-down control. The aim of this review is to examine how such a map may be represented at the neuronal level. We propose that there is not a single, common map in the brain, but that a number of cortical areas work together to generate the resultant behavior. Specifically, we suggest that the lateral intraparietal area (LIP) of posterior parietal cortex provides a simple representation of attentional priority, which remaps across saccades, so that there is an apparent allocentric map in a region with retinocentric encoding scheme. We propose that the frontal eye field (FEF) of prefrontal cortex receives the responses from LIP, but can suppress them to control the flow of eye movement behavior, and that the intermediate layers of the superior colliculus (SCi) reflect the final saccade goal. Together, these areas function to guide eye movements and may play a similar role in allocating covert visual attention.

Visual cortical activity before and after cochlear implantation: A follow up ERP prospective study in deaf children

ERPs were recorded in response to presentation of static colored patterned stimuli in 25 children (19 to 80months of age at cochlear implantation, CI) with very early prelingual profound deafness (PreLD), 21 postlingual profoundly deaf children (PostLD) (34 to 180months of age at CI) and gender- and age-matched control hearing children. Recording sessions were performed before CI, then 6 and 24months after CI. Results showed that prelingual and, at a lesser degree, postlingual auditory deprivation altered cortical visual neural activity associated to colored shapes from both P1 and N1 cortical processing stages. The P1 and N1 amplitude modifications vanished about 24months after CI in both PreLD and PostLD deaf children. In PreLD the visual processing pattern becomes similar to the typical one essentially by an amplitude decrease of P1 on the left hemisphere together with an amplitude increase of the N1 on the right hemisphere. Finally, in PreLD, increased LH advantage over the RH in N1 amplitude on the cerebellar-occipito-parietal region before CI showed a significant inverse relationship with speech perception outcomes 3years after CI. Investigating early visual processing development and its neural substrates in deaf children would help to understand the variability of CI outcome, because their cortical visual organization diverged from the one of typically developing hearing children, and cannot be predicted from what is observed in deaf adults.

Action history influences subsequent movement via two distinct processes

The characteristics of goal-directed actions tend to resemble those of previously executed actions, but it is unclear whether such effects depend strictly on action history, or also reflect context-dependent processes related to predictive motor planning. Here we manipulated the time available to initiate movements after a target was specified, and studied the effects of predictable movement sequences, to systematically dissociate effects of the most recently executed movement from the movement required next. We found that directional biases due to recent movement history strongly depend upon movement preparation time, suggesting an important contribution from predictive planning. However predictive biases co-exist with an independent source of bias that depends only on recent movement history. The results indicate that past experience influences movement execution through a combination of temporally-stable processes that are strictly use-dependent, and dynamically-evolving and context-dependent processes that reflect prediction of future actions.

Visual Responses in FEF, Unlike V1, Primarily Reflect When the Visual Context Renders a Receptive Field Salient

When light falls within a neuronal visual receptive field (RF) the resulting activity is referred to as the visual response. Recent work suggests this activity is in response to both the visual stimulation and the abrupt appearance, or salience, of the presentation. Here we present a novel method for distinguishing the two, based on the timing of random and nonrandom presentations. We examined these contributions in frontal eye field (FEF; N = 51) and as a comparison, an early stage in the primary visual cortex (V1; N = 15) of male monkeys (Macaca mulatta). An array of identical stimuli was presented within and outside the neuronal RF while we manipulated salience by varying the time between stimulus presentations. We hypothesized that the rapid presentation would reduce salience (the sudden appearance within the visual field) of a stimulus at any one location, and thus decrease responses driven by salience in the RF. We found that when the interstimulus interval decreased from 500 to 16 ms there was an approximate 79% reduction in the FEF response compared with an estimated 17% decrease in V1. This reduction in FEF response for rapid presentation was evident even when the random sequence preceding a stimulus did not stimulate the RF for 500 ms. The time course of these response changes in FEF suggest that salience is represented much earlier (<100 ms following stimulus onset) than previously estimated. Our results suggest that the contribution of salience dominates at higher levels of the visual system.SIGNIFICANCE STATEMENT The neuronal responses in early visual processing [e.g., primary visual cortex (V1)] reflect primarily the retinal stimulus. Processing in higher visual areas is modulated by a combination of the visual stimulation and contextual factors, such as salience, but identifying these components separately has been difficult. Here we quantified these contributions at a late stage of visual processing [frontal eye field (FEF)] and as a comparison, an early stage in V1. Our results suggest that as visual information continues through higher levels of processing the neural responses are no longer driven primarily by the visual stimulus in the receptive field, but by the broader context that stimulus defines-very different from current views about visual signals in FEF.

Superior colliculus encodes visual saliency before the primary visual cortex

Models of visual attention postulate the existence of a bottom-up saliency map that is formed early in the visual processing stream. Although studies have reported evidence of a saliency map in various cortical brain areas, determining the contribution of phylogenetically older pathways is crucial to understanding its origin. Here, we compared saliency coding from neurons in two early gateways into the visual system: the primary visual cortex (V1) and the evolutionarily older superior colliculus (SC). We found that, while the response latency to visual stimulus onset was earlier for V1 neurons than superior colliculus superficial visual-layer neurons (SCs), the saliency representation emerged earlier in SCs than in V1. Because the dominant input to the SCs arises from V1, these relative timings are consistent with the hypothesis that SCs neurons pool the inputs from multiple V1 neurons to form a feature-agnostic saliency map, which may then be relayed to other brain areas.

Object comparison in the lateral intraparietal area

We can search for and locate specific objects in our environment by looking for objects with similar features. Object recognition involves stimulus similarity responses in ventral visual areas and task-related responses in prefrontal cortex. We tested whether neurons in the lateral intraparietal area (LIP) of posterior parietal cortex could form an intermediary representation, collating information from object-specific similarity map representations to allow general decisions about whether a stimulus matches the object being looked for. We hypothesized that responses to stimuli would correlate with how similar they are to a sample stimulus. When animals compared two peripheral stimuli to a sample at their fovea, the response to the matching stimulus was similar, independent of the sample identity, but the response to the nonmatch depended on how similar it was to the sample: the more similar, the greater the response to the nonmatch stimulus. These results could not be explained by task difficulty or confidence. We propose that LIP uses its known mechanistic properties to integrate incoming visual information, including that from the ventral stream about object identity, to create a dynamic representation that is concise, low dimensional, and task relevant and that signifies the choice priorities in mental matching behavior.NEW & NOTEWORTHY Studies in object recognition have focused on the ventral stream, in which neurons respond as a function of how similar a stimulus is to their preferred stimulus, and on prefrontal cortex, where neurons indicate which stimulus is being looked for. We found that parietal area LIP uses its known mechanistic properties to form an intermediary representation in this process. This creates a perceptual similarity map that can be used to guide decisions in prefrontal areas.

Linking ADHD to the Neural Circuitry of Attention

Attention deficit hyperactivity disorder (ADHD) is a complex condition with a heterogeneous presentation. Current diagnosis is primarily based on subjective experience and observer reports of behavioral symptoms - an approach that has significant limitations. Many studies show that individuals with ADHD exhibit poorer performance on cognitive tasks than neurotypical controls, and at least seven main functional domains appear to be implicated in ADHD. We discuss the underlying neural mechanisms of cognitive functions associated with ADHD, with emphasis on the neural basis of selective attention, demonstrating the feasibility of basic research approaches for further understanding cognitive behavioral processes as they relate to human psychopathology. The study of circuit-level mechanisms underlying executive functions in nonhuman primates holds promise for advancing our understanding, and ultimately the treatment, of ADHD.

Sequential sensory and decision processing in posterior parietal cortex

Decisions about the behavioral significance of sensory stimuli often require comparing sensory inference of what we are looking at to internal models of what we are looking for. Here, we test how neuronal selectivity for visual features is transformed into decision-related signals in posterior parietal cortex (area LIP). Monkeys performed a visual matching task that required them to detect target stimuli composed of conjunctions of color and motion-direction. Neuronal recordings from area LIP revealed two main findings. First, the sequential processing of visual features and the selection of target-stimuli suggest that LIP is involved in transforming sensory information into decision-related signals. Second, the patterns of color and motion selectivity and their impact on decision-related encoding suggest that LIP plays a role in detecting target stimuli by comparing bottom-up sensory inputs (what the monkeys were looking at) and top-down cognitive encoding inputs (what the monkeys were looking for).

DOI:http://dx.doi.org/10.7554/eLife.23743.001

Superior colliculus neurons encode a visual saliency map during free viewing of natural dynamic video

Models of visual attention postulate the existence of a saliency map whose function is to guide attention and gaze to the most conspicuous regions in a visual scene. Although cortical representations of saliency have been reported, there is mounting evidence for a subcortical saliency mechanism, which pre-dates the evolution of neocortex. Here, we conduct a strong test of the saliency hypothesis by comparing the output of a well-established computational saliency model with the activation of neurons in the primate superior colliculus (SC), a midbrain structure associated with attention and gaze, while monkeys watched video of natural scenes. We find that the activity of SC superficial visual-layer neurons (SCs), specifically, is well-predicted by the model. This saliency representation is unlikely to be inherited from fronto-parietal cortices, which do not project to SCs, but may be computed in SCs and relayed to other areas via tectothalamic pathways.

Differential Contribution of Low- and High-level Image Content to Eye Movements in Monkeys and Humans

Oculomotor selection exerts a fundamental impact on our experience of the environment. To better understand the underlying principles, researchers typically rely on behavioral data from humans, and electrophysiological recordings in macaque monkeys. This approach rests on the assumption that the same selection processes are at play in both species. To test this assumption, we compared the viewing behavior of 106 humans and 11 macaques in an unconstrained free-viewing task. Our data-driven clustering analyses revealed distinct human and macaque clusters, indicating species-specific selection strategies. Yet, cross-species predictions were found to be above chance, indicating some level of shared behavior. Analyses relying on computational models of visual saliency indicate that such cross-species commonalities in free viewing are largely due to similar low-level selection mechanisms, with only a small contribution by shared higher level selection mechanisms and with consistent viewing behavior of monkeys being a subset of the consistent viewing behavior of humans.

Circuits for presaccadic visual remapping

Saccadic eye movements rapidly displace the image of the world that is projected onto the retinas. In anticipation of each saccade, many neurons in the visual system shift their receptive fields. This presaccadic change in visual sensitivity, known as remapping, was first documented in the parietal cortex and has been studied in many other brain regions. Remapping requires information about upcoming saccades via corollary discharge. Analyses of neurons in a corollary discharge pathway that targets the frontal eye field (FEF) suggest that remapping may be assembled in the FEF's local microcircuitry. Complementary data from reversible inactivation, neural recording, and modeling studies provide evidence that remapping contributes to transsaccadic continuity of action and perception. Multiple forms of remapping have been reported in the FEF and other brain areas, however, and questions remain about the reasons for these differences. In this review of recent progress, we identify three hypotheses that may help to guide further investigations into the structure and function of circuits for remapping.

Competition between Visual Events Modulates the Influence of Salience during Free-Viewing of Naturalistic Videos

In daily life the brain is exposed to a large amount of external signals that compete for processing resources. The attentional system can select relevant information based on many possible combinations of goal-directed and stimulus-driven control signals. Here, we investigate the behavioral and physiological effects of competition between distinctive visual events during free-viewing of naturalistic videos. Nineteen healthy subjects underwent functional magnetic resonance imaging (fMRI) while viewing short video-clips of everyday life situations, without any explicit goal-directed task. Each video contained either a single semantically-relevant event on the left or right side (Lat-trials), or multiple distinctive events in both hemifields (Multi-trials). For each video, we computed a salience index to quantify the lateralization bias due to stimulus-driven signals, and a gaze index (based on eye-tracking data) to quantify the efficacy of the stimuli in capturing attention to either side. Behaviorally, our results showed that stimulus-driven salience influenced spatial orienting only in presence of multiple competing events (Multi-trials). fMRI results showed that the processing of competing events engaged the ventral attention network, including the right temporoparietal junction (R TPJ) and the right inferior frontal cortex. Salience was found to modulate activity in the visual cortex, but only in the presence of competing events; while the orienting efficacy of Multi-trials affected activity in both the visual cortex and posterior parietal cortex (PPC). We conclude that in presence of multiple competing events, the ventral attention system detects semantically-relevant events, while regions of the dorsal system make use of saliency signals to select relevant locations and guide spatial orienting.

From Behavior to Neural Dynamics: An Integrated Theory of Attention

The brain has a limited capacity and therefore needs mechanisms to selectively enhance the information most relevant to one's current behavior. We refer to these mechanisms as "attention." Attention acts by increasing the strength of selected neural representations and preferentially routing them through the brain's large-scale network. This is a critical component of cognition and therefore has been a central topic in cognitive neuroscience. Here we review a diverse literature that has studied attention at the level of behavior, networks, circuits, and neurons. We then integrate these disparate results into a unified theory of attention.

Retinotopic patterns of background connectivity between V1 and fronto-parietal cortex are modulated by task demands

Attention facilitates the processing of task-relevant visual information and suppresses interference from task-irrelevant information. Modulations of neural activity in visual cortex depend on attention, and likely result from signals originating in fronto-parietal and cingulo-opercular regions of cortex. Here, we tested the hypothesis that attentional facilitation of visual processing is accomplished in part by changes in how brain networks involved in attentional control interact with sectors of V1 that represent different retinal eccentricities. We measured the strength of background connectivity between fronto-parietal and cingulo-opercular regions with different eccentricity sectors in V1 using functional MRI data that were collected while participants performed tasks involving attention to either a centrally presented visual stimulus or a simultaneously presented auditory stimulus. We found that when the visual stimulus was attended, background connectivity between V1 and the left frontal eye fields (FEF), left intraparietal sulcus (IPS), and right IPS varied strongly across different eccentricity sectors in V1 so that foveal sectors were more strongly connected than peripheral sectors. This retinotopic gradient was weaker when the visual stimulus was ignored, indicating that it was driven by attentional effects. Greater task-driven differences between foveal and peripheral sectors in background connectivity to these regions were associated with better performance on the visual task and faster response times on correct trials. These findings are consistent with the notion that attention drives the configuration of task-specific functional pathways that enable the prioritized processing of task-relevant visual information, and show that the prioritization of visual information by attentional processes may be encoded in the retinotopic gradient of connectivty between V1 and fronto-parietal regions.

Visual attention mitigates information loss in small- and large-scale neural codes

The visual system transforms complex inputs into robust and parsimonious neural codes that efficiently guide behavior. Because neural communication is stochastic, the amount of encoded visual information necessarily decreases with each synapse. This constraint requires that sensory signals are processed in a manner that protects information about relevant stimuli from degradation. Such selective processing--or selective attention--is implemented via several mechanisms, including neural gain and changes in tuning properties. However, examining each of these effects in isolation obscures their joint impact on the fidelity of stimulus feature representations by large-scale population codes. Instead, large-scale activity patterns can be used to reconstruct representations of relevant and irrelevant stimuli, thereby providing a holistic understanding about how neuron-level modulations collectively impact stimulus encoding.

The what, where, and why of priority maps and their interactions with visual working memory

Priority maps are winner-take-all neural mechanisms thought to guide the allocation of covert and overt attention. Here, we go beyond this standard definition and argue that priority maps play a much broader role in controlling goal-directed behavior. We start by defining what priority maps are and where they might be found in the brain; we then ask why they exist-the function that they serve. We propose that this function is to communicate a goal state to the different effector systems, thereby guiding behavior. Within this framework, we speculate on how priority maps interact with visual working memory and introduce our common source hypothesis, the suggestion that this goal state is maintained in visual working memory and used to construct all of the priority maps controlling the various motor systems. Finally, we look ahead and suggest questions about priority maps that should be asked next.

"More than skin deep": stress neurobiology and mental health consequences of racial discrimination

Ethnic minority groups across the world face a complex set of adverse social and psychological challenges linked to their minority status, often involving racial discrimination. Racial discrimination is increasingly recognized as an important contributing factor to health disparities among non-dominant ethnic minorities. A growing body of literature has recognized these health disparities and has investigated the relationship between racial discrimination and poor health outcomes. Chronically elevated cortisol levels and a dysregulated hypothalamic-pituitary-adrenal (HPA) axis appear to mediate effects of racial discrimination on allostatic load and disease. Racial discrimination seems to converge on the anterior cingulate cortex (ACC) and may impair the function of the prefrontal cortex (PFC), hence showing substantial similarities to chronic social stress. This review provides a summary of recent literature on hormonal and neural effects of racial discrimination and a synthesis of potential neurobiological pathways by which discrimination affects mental health.

Saliency and saccade encoding in the frontal eye field during natural scene search

The frontal eye field (FEF) plays a central role in saccade selection and execution. Using artificial stimuli, many studies have shown that the activity of neurons in the FEF is affected by both visually salient stimuli in a neuron's receptive field and upcoming saccades in a certain direction. However, the extent to which visual and motor information is represented in the FEF in the context of the cluttered natural scenes we encounter during everyday life has not been explored. Here, we model the activities of neurons in the FEF, recorded while monkeys were searching natural scenes, using both visual and saccade information. We compare the contribution of bottom-up visual saliency (based on low-level features such as brightness, orientation, and color) and saccade direction. We find that, while saliency is correlated with the activities of some neurons, this relationship is ultimately driven by activities related to movement. Although bottom-up visual saliency contributes to the choice of saccade targets, it does not appear that FEF neurons actively encode the kind of saliency posited by popular saliency map theories. Instead, our results emphasize the FEF's role in the stages of saccade planning directly related to movement generation.

Finding and recognizing objects in natural scenes: complementary computations in the dorsal and ventral visual systems

Searching for and recognizing objects in complex natural scenes is implemented by multiple saccades until the eyes reach within the reduced receptive field sizes of inferior temporal cortex (IT) neurons. We analyze and model how the dorsal and ventral visual streams both contribute to this. Saliency detection in the dorsal visual system including area LIP is modeled by graph-based visual saliency, and allows the eyes to fixate potential objects within several degrees. Visual information at the fixated location subtending approximately 9° corresponding to the receptive fields of IT neurons is then passed through a four layer hierarchical model of the ventral cortical visual system, VisNet. We show that VisNet can be trained using a synaptic modification rule with a short-term memory trace of recent neuronal activity to capture both the required view and translation invariances to allow in the model approximately 90% correct object recognition for 4 objects shown in any view across a range of 135° anywhere in a scene. The model was able to generalize correctly within the four trained views and the 25 trained translations. This approach analyses the principles by which complementary computations in the dorsal and ventral visual cortical streams enable objects to be located and recognized in complex natural scenes.

A role of the claustrum in auditory scene analysis by reflecting sensory change

The biological function of the claustrum remains speculative, despite many years of research. On the basis of its widespread connections it is often hypothesized that the claustrum may have an integrative function mainly reflecting objects rather than the details of sensory stimuli. Given the absence of a clear demonstration of any sensory integration in claustral neurons, however, we propose an alternative, data-driven, hypothesis: namely that the claustrum detects the occurrence of novel or salient sensory events. The detection of new events is critical for behavior and survival, as suddenly appearing objects may require rapid and coordinated reactions. Sounds are of particular relevance in this regard, and our conclusions are based on the analysis of neurons in the auditory zone of the primate claustrum. Specifically, we studied the responses to natural sounds, their preference to various sound categories, and to changes in the auditory scene. In a test for sound-category preference claustral neurons responded to but displayed a clear lack of selectivity between monkey vocalizations, other animal vocalizations or environmental sounds (Esnd). Claustral neurons were however able to detect target sounds embedded in a noisy background and their responses scaled with target signal to noise ratio (SNR). The single trial responses of individual neurons suggest that these neurons detected and reflected the occurrence of a change in the auditory scene. Given its widespread connectivity with sensory, motor and limbic structures the claustrum could play the essential role of identifying the occurrence of important sensory changes and notifying other brain areas-hence contributing to sensory awareness.

Shape beyond recognition: form-derived directionality and its effects on visual attention and motion perception

The shape of an object restricts its movements and therefore its future location. The rules governing selective sampling of the environment likely incorporate any available data, including shape, that provide information about where important things are going to be in the near future so that the object can be located, tracked, and sampled for information. We asked people to assess in which direction several novel objects pointed or directed them. With independent groups of people, we investigated whether their attention and sense of motion were systematically biased in this direction. Our work shows that nearly any novel object has intrinsic directionality derived from its shape. This shape information is swiftly and automatically incorporated into the allocation of overt and covert visual orienting and the detection of motion, processes that themselves are inherently directional. The observed connection between form and space suggests that shape processing goes beyond recognition alone and may help explain why shape is a relevant dimension throughout the visual brain.

Bottom-up and top-down attention: different processes and overlapping neural systems

The brain is limited in its capacity to process all sensory stimuli present in the physical world at any point in time and relies instead on the cognitive process of attention to focus neural resources according to the contingencies of the moment. Attention can be categorized into two distinct functions: bottom-up attention, referring to attentional guidance purely by externally driven factors to stimuli that are salient because of their inherent properties relative to the background; and top-down attention, referring to internal guidance of attention based on prior knowledge, willful plans, and current goals. Over the past few years, insights on the neural circuits and mechanisms of bottom-up and top-down attention have been gained through neurophysiological experiments. Attention affects the mean neuronal firing rate as well as its variability and correlation across neurons. Although distinct processes mediate the guidance of attention based on bottom-up and top-down factors, a common neural apparatus, the frontoparietal network, is essential in both types of attentional processes.

Production, control, and visual guidance of saccadic eye movements

Primate vision is served by rapid shifts of gaze called saccades. This review will survey current knowledge and particular problems concerning the neural control and guidance of gaze shifts.

Neural correlates of the divergence of instrumental probability distributions

Flexible action selection requires knowledge about how alternative actions impact the environment: a "cognitive map" of instrumental contingencies. Reinforcement learning theories formalize this map as a set of stochastic relationships between actions and states, such that for any given action considered in a current state, a probability distribution is specified over possible outcome states. Here, we show that activity in the human inferior parietal lobule correlates with the divergence of such outcome distributions-a measure that reflects whether discrimination between alternative actions increases the controllability of the future-and, further, that this effect is dissociable from those of other information theoretic and motivational variables, such as outcome entropy, action values, and outcome utilities. Our results suggest that, although ultimately combined with reward estimates to generate action values, outcome probability distributions associated with alternative actions may be contrasted independently of valence computations, to narrow the scope of the action selection problem.

A functional hierarchy within the parietofrontal network in stimulus selection and attention control

Although we are confronted with an ever-changing environment, we do not have the capacity to analyze all incoming sensory information. Perception is selective and is guided both by salient events occurring in our visual field and by cognitive premises about what needs our attention. Although the lateral intraparietal area (LIP) and frontal eye field (FEF) are known to represent the position of visual attention, their respective contributions to its control are still unclear. Here, we report LIP and FEF neuronal activities recorded while monkeys performed a voluntary attention-orientation target-detection task. We show that both encode behaviorally significant events, but that the FEF plays a specific role in mapping abstract cue instructions onto a spatial priority map to voluntarily guide attention. On the basis of a latency analysis, we show that the coding of stimulus identity and position precedes the emergence of an explicit attentional signal within the FEF. We also describe dynamic temporal hierarchies between LIP and FEF: stimuli carrying the highest intrinsic saliency are signaled by LIP before FEF, whereas stimuli carrying the highest extrinsic saliency are signaled in FEF before LIP. This suggests that whereas the parietofrontal attentional network most probably processes visual information in a recurrent way, exogenous processing predominates in the parietal cortex and the endogenous control of attention takes place in the FEF.

Pain and analgesia: the value of salience circuits

Evaluating external and internal stimuli is critical to survival. Potentially tissue-damaging conditions generate sensory experiences that the organism must respond to in an appropriate, adaptive manner (e.g., withdrawal from the noxious stimulus, if possible, or seeking relief from pain and discomfort). The importance we assign to a signal generated by a noxious state, its salience, reflects our belief as to how likely the underlying situation is to impact our chance of survival. Importantly, it has been hypothesized that aberrant functioning of the brain circuits which assign salience values to stimuli may contribute to chronic pain. We describe examples of this phenomenon, including 'feeling pain' in the absence of a painful stimulus, reporting minimal pain in the setting of major trauma, having an 'analgesic' response in the absence of an active treatment, or reporting no pain relief after administration of a potent analgesic medication, which may provide critical insights into the role that salience circuits play in contributing to numerous conditions characterized by persistent pain. Collectively, a refined understanding of abnormal activity or connectivity of elements within the salience network may allow us to more effectively target interventions to relevant components of this network in patients with chronic pain.

Anticipatory remapping of attentional priority across the entire visual field

It has been suggested that one way we may create a stable percept of the visual world across multiple eye movements is to pass information from one set of neurons to another around the time of each eye movement. Previous studies have shown that some neurons in the lateral intraparietal area (LIP) exhibit anticipatory remapping: these neurons produce a visual response to a stimulus that will enter their receptive field after a saccade but before it actually does so. LIP responses during fixation are thought to represent attentional priority, behavioral relevance, or value. In this study, we test whether the remapped response represents this attentional priority by examining the activity of LIP neurons while animals perform a visual foraging task. We find that the population responds more to a target than to a distractor before the saccade even begins to bring the stimulus into the receptive field. Within 20 ms of the saccade ending, the responses in almost one-third of LIP neurons closely resemble the responses that will emerge during stable fixation. Finally, we show that, in these neurons and in the population as a whole, this remapping occurs for all stimuli in all locations across the visual field and for both long and short saccades. We conclude that this complete remapping of attentional priority across the visual field could underlie spatial stability across saccades.

Visual response of neurons in the lateral intraparietal area and saccadic reaction time during a visual detection task

During visual detection with saccades, a target with higher luminance is detected with reduced reaction times. In such visual detection behaviors, luminance-related sensory signals should be converted into movement-related signals for saccade initiation. At the site where the visuomotor transformation takes place, there is the possibility that visual activity not only encodes the target luminance but also affects the generation of an upcoming saccade. To assess this possibility, we recorded single-cell activity from visually responsive neurons in the lateral intraparietal area (LIP) when monkeys made a saccade to an isolated target over five luminance levels. We found that as stimulus luminance increased, visual response strength increased, and response onset latency decreased. These luminance-related changes in activity were significantly correlated with changes in reaction time. In particular, changes in response onset latency accounted for a substantial part of the observed changes in reaction time, suggesting that luminance-related changes in response onset latency may propagate to the saccade generation process. However, the length of time from response onset to saccade onset was not constant but increased as luminance was reduced, suggesting the existence of other luminance-dependent processing in downstream and/or parallel pathways before saccade generation. Additionally, we failed to find strong covariance between response strength or latency and reaction time when the effect of luminance changes was removed. Thus, the present results reveal how visually responsive LIP neurons contribute to saccade generation in visual detection.

Cortical circuits for the control of attention

How are some thoughts favored over others? A wealth of data at the level of single neurons has yielded candidate brain areas and mechanisms for our best-understood model: visual attention. Recent work has naturally evolved toward efforts at a more integrative, network, understanding. It suggests that focusing attention arises from interactions between widespread cortical and subcortical networks that may be regulated via their rhythmic synchronization.

Anatomical substrates of the alerting, orienting and executive control components of attention: focus on the posterior parietal lobe

Both neuropsychological and functional neuroimaging studies have identified that the posterior parietal lobe (PPL) is critical for the attention function. However, the unique role of distinct parietal cortical subregions and their underlying white matter (WM) remains in question. In this study, we collected both magnetic resonance imaging and diffusion tensor imaging (DTI) data in normal participants, and evaluated their attention performance using attention network test (ANT), which could isolate three different attention components: alerting, orienting and executive control. Cortical thickness, surface area and DTI parameters were extracted from predefined PPL subregions and correlated with behavioural performance. Tract-based spatial statistics (TBSS) was used for the voxel-wise statistical analysis. Results indicated structure-behaviour relationships on multiple levels. First, a link between the cortical thickness and WM integrity of the right inferior parietal regions and orienting performance was observed. Specifically, probabilistic tractography demonstrated that the integrity of WM connectivity between the bilateral inferior parietal lobules mediated the orienting performance. Second, the scores of executive control were significantly associated with the WM diffusion metrics of the right supramarginal gyrus. Finally, TBSS analysis revealed that alerting performance was significant correlated with the fractional anisotropy of local WM connecting the right thalamus and supplementary motor area. We conclude that distinct areas and features within PPL are associated with different components of attention. These findings could yield a more complete understanding of the nature of the PPL contribution to visuospatial attention.

Supplementary eye field during visual search: salience, cognitive control, and performance monitoring

How supplementary eye field (SEF) contributes to visual search is unknown. Inputs from cortical and subcortical structures known to represent visual salience suggest that SEF may serve as an additional node in this network. This hypothesis was tested by recording action potentials and local field potentials (LFPs) in two monkeys performing an efficient pop-out visual search task. Target selection modulation, tuning width, and response magnitude of spikes and LFP in SEF were compared with those in frontal eye field. Surprisingly, only ∼2% of SEF neurons and ∼8% of SEF LFP sites selected the location of the search target. The absence of salience in SEF may be due to an absence of appropriate visual afferents, which suggests that these inputs are a necessary anatomical feature of areas representing salience. We also tested whether SEF contributes to overcoming the automatic tendency to respond to a primed color when the target identity switches during priming of pop-out. Very few SEF neurons or LFP sites modulated in association with performance deficits following target switches. However, a subset of SEF neurons and LFPs exhibited strong modulation following erroneous saccades to a distractor. Altogether, these results suggest that SEF plays a limited role in controlling ongoing visual search behavior, but may play a larger role in monitoring search performance.

In monkeys making value-based decisions, LIP neurons encode cue salience and not action value

In monkeys deciding between alternative saccadic eye movements, lateral intraparietal (LIP) neurons representing each saccade fire at a rate proportional to the value of the reward expected upon its completion. This observation has been interpreted as indicating that LIP neurons encode saccadic value and that they mediate value-based decisions between saccades. Here, we show that LIP neurons representing a given saccade fire strongly not only if it will yield a large reward but also if it will incur a large penalty. This finding indicates that LIP neurons are sensitive to the motivational salience of cues. It is compatible neither with the idea that LIP neurons represent action value nor with the idea that value-based decisions take place in LIP neurons.

Early involvement of prefrontal cortex in visual bottom-up attention

Visual attention is guided to stimuli based either on their intrinsic saliency against their background (bottom-up factors) or through willful search of known targets (top-down factors). Posterior parietal cortex is thought to play a critical role in the guidance of visual bottom-up attention, whereas prefrontal cortex is thought to represent top-down factors. Contrary to this established view, we found that when monkeys were tested in a task requiring detection of a salient stimulus defined purely by bottom-up factors and whose identity was unknown prior to the presentation of a visual display, prefrontal neurons represented the salient stimulus no later than those in the posterior parietal cortex. This was true even though visual response latency was shorter in parietal than in prefrontal cortex. These results suggest an early involvement of the prefrontal cortex in the bottom-up guidance of visual attention.