Attention as a decision in information space

Uncertainty modulates visual maps during noninstrumental information demand

Animals are intrinsically motivated to obtain information independently of instrumental incentives. This motivation depends on two factors: a desire to resolve uncertainty by gathering accurate information and a desire to obtain positively-valenced observations, which predict favorable rather than unfavorable outcomes. To understand the neural mechanisms, we recorded parietal cortical activity implicated in prioritizing stimuli for spatial attention and gaze, in a task in which monkeys were free (but not trained) to obtain information about probabilistic non-contingent rewards. We show that valence and uncertainty independently modulated parietal neuronal activity, and uncertainty but not reward-related enhancement consistently correlated with behavioral sensitivity. The findings suggest uncertainty-driven and valence-driven information demand depend on partially distinct pathways, with the former being consistently related to parietal responses and the latter depending on additional mechanisms implemented in downstream structures.

Curiosity is motivated by uncertainty and valence, but how uncertainty and valence are encoded in the brain remains poorly understood. Here, the authors show that parietal neurons are enhanced by both factors, but that they specifically predict visual information seeking based on uncertainty.

People confabulate with high confidence when their decisions are supported by weak internal variables

People can introspect on their internal state and report the reasons driving their decisions but choice blindness (CB) experiments suggest that this ability can sometimes be a retrospective illusion. Indeed, when presented with deceptive cues, people justify choices they did not make in the first place, suggesting that external cues largely contribute to introspective processes. Yet, it remains unclear what are the respective contributions of external cues and internal decision variables in forming introspective report. Here, using a brain–computer interface, we show that internal variables continue to be monitored but are less impactful than deceptive external cues during CB episodes. Moreover, we show that deceptive cues overturn the classical relationship between confidence and accuracy: introspective failures are associated with higher confidence than genuine introspective reports. We tracked back the origin of these overconfident confabulations by revealing their prominence when internal decision evidence is weak and variable. Thus, introspection is neither a direct reading of internal variables nor a mere retrospective illusion, but rather reflects the integration of internal decision evidence and external cues, with CB being a special instance where internal evidence is inconsistent.

The Posterior Parietal Cortex in Adaptive Visual Processing

Although the primate posterior parietal cortex (PPC) has been largely associated with space, attention, and action-related processing, a growing number of studies have reported the direct representation of a diverse array of action-independent nonspatial visual information in the PPC during both perception and visual working memory. By describing the distinctions and the close interactions of visual representation with space, attention, and action-related processing in the PPC, here I propose that we may understand these diverse PPC functions together through the unique contribution of the PPC to adaptive visual processing and form a more integrated and structured view of the role of the PPC in vision, cognition, and action.

Neurocognitive Development of the Resolution of Selective Visuo-Spatial Attention: Functional MRI Evidence From Object Tracking

Our ability to select relevant information from the environment is limited by the resolution of attention – i.e., the minimum size of the region that can be selected. Neural mechanisms that underlie this limit and its development are not yet understood. Functional magnetic resonance imaging (fMRI) was performed during an object tracking task in 7- and 11-year-old children, and in young adults. Object tracking activated canonical fronto-parietal attention systems and motion-sensitive area MT in children as young as 7 years. Object tracking performance improved with age, together with stronger recruitment of parietal attention areas and a shift from low-level to higher-level visual areas. Increasing the required resolution of spatial attention – which was implemented by varying the distance between target and distractors in the object tracking task – led to activation increases in fronto-insular cortex, medial frontal cortex including anterior cingulate cortex (ACC) and supplementary motor area, superior colliculi, and thalamus. This core circuitry for attentional precision was recruited by all age groups, but ACC showed an age-related activation reduction. Our results suggest that age-related improvements in selective visual attention and in the resolution of attention are characterized by an increased use of more functionally specialized brain regions during the course of development.

A Tale of Two Visual Systems: Invariant and Adaptive Visual Information Representations in the Primate Brain

Visual information processing contains two opposite needs. There is both a need to comprehend the richness of the visual world and a need to extract only pertinent visual information to guide thoughts and behavior at a given moment. I argue that these two aspects of visual processing are mediated by two complementary visual systems in the primate brain-specifically, the occipitotemporal cortex (OTC) and the posterior parietal cortex (PPC). The role of OTC in visual processing has been documented extensively by decades of neuroscience research. I review here recent evidence from human imaging and monkey neurophysiology studies to highlight the role of PPC in adaptive visual processing. I first document the diverse array of visual representations found in PPC. I then describe the adaptive nature of visual representation in PPC by contrasting visual processing in OTC and PPC and by showing that visual representations in PPC largely originate from OTC.

Habitual versus goal-driven attention

Recent research has expanded the list of factors that control spatial attention. Beside current goals and perceptual salience, statistical learning, reward, motivation and emotion also affect attention. But do these various factors influence spatial attention in the same manner, as suggested by the integrated framework of attention, or do they target different aspects of spatial attention? Here I present evidence that the control of attention may be implemented in two ways. Whereas current goals typically modulate where in space attention is prioritized, search habits affect how one moves attention in space. Using the location probability learning paradigm, I show that a search habit forms when people frequently find a visual search target in one region of space. Attentional cuing by probability learning differs from that by current goals. Probability cuing is implicit and persists long after the probability cue is no longer valid. Whereas explicit goal-driven attention codes space in an environment-centered reference frame, probability cuing is viewer-centered and is insensitive to secondary working memory load and aging. I propose a multi-level framework that separates the source of attentional control from its implementation. Similar to the integrated framework, the multi-level framework considers current goals, perceptual salience, and selection history as major sources of attentional control. However, these factors are implemented in two ways, controlling where spatial attention is allocated and how one shifts attention in space.

Rational metareasoning and the plasticity of cognitive control

The human brain has the impressive capacity to adapt how it processes information to high-level goals. While it is known that these cognitive control skills are malleable and can be improved through training, the underlying plasticity mechanisms are not well understood. Here, we develop and evaluate a model of how people learn when to exert cognitive control, which controlled process to use, and how much effort to exert. We derive this model from a general theory according to which the function of cognitive control is to select and configure neural pathways so as to make optimal use of finite time and limited computational resources. The central idea of our Learned Value of Control model is that people use reinforcement learning to predict the value of candidate control signals of different types and intensities based on stimulus features. This model correctly predicts the learning and transfer effects underlying the adaptive control-demanding behavior observed in an experiment on visual attention and four experiments on interference control in Stroop and Flanker paradigms. Moreover, our model explained these findings significantly better than an associative learning model and a Win-Stay Lose-Shift model. Our findings elucidate how learning and experience might shape people’s ability and propensity to adaptively control their minds and behavior. We conclude by predicting under which circumstances these learning mechanisms might lead to self-control failure.

The human brain has the impressive ability to adapt how it processes information to high level goals. While it is known that these cognitive control skills are malleable and can be improved through training, the underlying plasticity mechanisms are not well understood. Here, we derive a computational model of how people learn when to exert cognitive control, which controlled process to use, and how much effort to exert from a formal theory of the function of cognitive control. Across five experiments, we find that our model correctly predicts that people learn to adaptively regulate their attention and decision-making and how these learning effects transfer to novel situations. Our findings elucidate how learning and experience might shape people’s ability and propensity to adaptively control their minds and behavior. We conclude by predicting under which circumstances these learning mechanisms might lead to self-control failure.

The Role of the Lateral Intraparietal Area in (the Study of) Decision Making

Over the past two decades, neurophysiological responses in the lateral intraparietal area (LIP) have received extensive study for insight into decision making. In a parallel manner, inferred cognitive processes have enriched interpretations of LIP activity. Because of this bidirectional interplay between physiology and cognition, LIP has served as fertile ground for developing quantitative models that link neural activity with decision making. These models stand as some of the most important frameworks for linking brain and mind, and they are now mature enough to be evaluated in finer detail and integrated with other lines of investigation of LIP function. Here, we focus on the relationship between LIP responses and known sensory and motor events in perceptual decision-making tasks, as assessed by correlative and causal methods. The resulting sensorimotor-focused approach offers an account of LIP activity as a multiplexed amalgam of sensory, cognitive, and motor-related activity, with a complex and often indirect relationship to decision processes. Our data-driven focus on multiplexing (and de-multiplexing) of various response components can complement decision-focused models and provides more detailed insight into how neural signals might relate to cognitive processes such as decision making.

Multiple visual objects are sampled sequentially

When acting in a complex visual environment, it is essential to be able to flexibly allocate attention to parts of the visual scene that may contain goal-relevant information. The paper by Jia et al. provides novel evidence that our brains sequentially sample different objects in a visual scene. The results were obtained using “temporal response functions,” in which unique electroencephalographic (EEG) signals corresponding to the processing of 2 continuously presented objects were isolated in an object-specific way. These response functions were dominated by 10-Hz alpha-band activity. Crucially, the different objects were sequentially sampled at a rate of about 2 Hz. These findings provide important neurophysiological insights into how our visual system operates in complex environments.

Space-Time Dynamics of Membrane Currents Evolve to Shape Excitation, Spiking, and Inhibition in the Cortex at Small and Large Scales

In the cerebral cortex, membrane currents, i.e., action potentials and other membrane currents, express many forms of space-time dynamics. In the spontaneous asynchronous irregular state, their space-time dynamics are local non-propagating fluctuations and sparse spiking appearing at unpredictable positions. After transition to active spiking states, larger structured zones with active spiking neurons appear, propagating through the cortical network, driving it into various forms of widespread excitation, and engaging the network from microscopic scales to whole cortical areas. At each engaged cortical site, the amount of excitation in the network, after a delay, becomes matched by an equal amount of space-time fine-tuned inhibition that might be instrumental in driving the dynamics toward perception and action.

Neural Mechanisms of Updating under Reducible and Irreducible Uncertainty

Adaptive decision making depends on an agent's ability to use environmental signals to reduce uncertainty. However, because of multiple types of uncertainty, agents must take into account not only the extent to which signals violate prior expectations but also whether uncertainty can be reduced in the first place. Here we studied how human brains of both sexes respond to signals under conditions of reducible and irreducible uncertainty. We show behaviorally that subjects' value updating was sensitive to the reducibility of uncertainty, and could be quantitatively characterized by a Bayesian model where agents ignore expectancy violations that do not update beliefs or values. Using fMRI, we found that neural processes underlying belief and value updating were separable from responses to expectancy violation, and that reducibility of uncertainty in value modulated connections from belief-updating regions to value-updating regions. Together, these results provide insights into how agents use knowledge about uncertainty to make better decisions while ignoring mere expectancy violation.SIGNIFICANCE STATEMENT To make good decisions, a person must observe the environment carefully, and use these observations to reduce uncertainty about consequences of actions. Importantly, uncertainty should not be reduced purely based on how surprising the observations are, particularly because in some cases uncertainty is not reducible. Here we show that the human brain indeed reduces uncertainty adaptively by taking into account the nature of uncertainty and ignoring mere surprise. Behaviorally, we show that human subjects reduce uncertainty in a quasioptimal Bayesian manner. Using fMRI, we characterize brain regions that may be involved in uncertainty reduction, as well as the network they constitute, and dissociate them from brain regions that respond to mere surprise.

Thalamic control of functional cortical connectivity

The thalamus is an evolutionarily conserved structure with extensive reciprocal connections to cortical regions. While its role in transmitting sensory signals is well-studied, its broader engagement in cognition is unclear. In this review, we discuss evidence that the thalamus regulates functional connectivity within and between cortical regions, determining how a cognitive process is implemented across distributed cortical microcircuits. Within this framework, thalamic circuits do not necessarily determine the categorical content of a cognitive process (e.g., sensory details in feature-based attention), but rather provide a route by which task-relevant cortical representations are sustained and coordinated. Additionally, thalamic control of cortical connectivity bridges general arousal to the specific processing of categorical content, providing an intermediate level of cognitive and circuit description that will facilitate mapping neural computations onto thought and behavior.

Parietal neurons encode expected gains in instrumental information

In natural behavior, animals have access to multiple sources of information, but only a few of these sources are relevant for learning and actions. Beyond choosing an appropriate action, making good decisions entails the ability to choose the relevant information, but fundamental questions remain about the brain's information sampling policies. Recent studies described the neural correlates of seeking information about a reward, but it remains unknown whether, and how, neurons encode choices of instrumental information, in contexts in which the information guides subsequent actions. Here we show that parietal cortical neurons involved in oculomotor decisions encode, before an information sampling saccade, the reduction in uncertainty that the saccade is expected to bring for a subsequent action. These responses were distinct from the neurons' visual and saccadic modulations and from signals of expected reward or reward prediction errors. Therefore, even in an instrumental context when information and reward gains are closely correlated, individual cells encode decision variables that are based on informational factors and can guide the active sampling of action-relevant cues.

Understanding location- and feature-based processing along the human intraparietal sulcus

Based on different cognitive tasks and mapping methods, the human intraparietal sulcus (IPS) has been subdivided according to multiple different organizational schemes. The presence of topographically organized regions throughout IPS indicates a strong location-based processing in this brain region. However, visual short-term memory (VSTM) studies have shown that while a region in the inferior IPS region (inferior IPS) is involved in object individuation and selection based on location, a region in the superior IPS (superior IPS) primarily encodes and stores object featural information. Here, we determined the localization of these two VSTM IPS regions with respect to the topographic IPS regions in individual participants and the role of different IPS regions in location- and feature-based processing. Anatomically, inferior IPS showed an 85.2% overlap with topographic IPS regions, with the greatest overlap seen in V3A and V3B, and superior IPS showed a 73.6% overall overlap, with the greatest overlap seen in IPS0-2. Functionally, there appeared to be a partial overlap between IPS regions involved in location- and feature-based processing, with more inferior and medial regions showing a stronger location-based processing and more superior and lateral regions showing a stronger feature-based processing. Together, these results suggest that understanding the multiplex nature of IPS in visual cognition may not be reduced to examining the functions of the different IPS topographic regions, but rather, it can only be accomplished by understanding how regions identified by different tasks and methods may colocalize with each other.

Reconciling Saliency and Object Center-Bias Hypotheses in Explaining Free-Viewing Fixations

Predicting where people look in natural scenes has attracted a lot of interest in computer vision and computational neuroscience over the past two decades. Two seemingly contrasting categories of cues have been proposed to influence where people look: 1) low-level image saliency and 2) high-level semantic information. Our first contribution is to take a detailed look at these cues to confirm the hypothesis proposed by Henderson and Nuthmann and Henderson that observers tend to look at the center of objects. We analyzed fixation data for scene free-viewing over 17 observers on 60 object-annotated images with various types of objects. Images contained different types of scenes, such as natural scenes, line drawings, and 3-D rendered scenes. Our second contribution is to propose a simple combined model of low-level saliency and object center bias that outperforms each individual component significantly over our data, as well as on the Object and Semantic Images and Eye-tracking data set by Xu et al. The results reconcile saliency with object center-bias hypotheses and highlight that both types of cues are important in guiding fixations. Our work opens new directions to understand strategies that humans use in observing scenes and objects, and demonstrates the construction of combined models of low-level saliency and high-level object-based information.

Decision making from economic and signal detection perspectives: development of an integrated framework

Behavior is comprised of decisions made from moment to moment (i.e., to respond one way or another). Often, the decision maker cannot be certain of the value to be accrued from the decision (i.e., the outcome value). Decisions made under outcome value uncertainty form the basis of the economic framework of decision making. Behavior is also based on perception-perception of the external physical world and of the internal bodily milieu, which both provide cues that guide decision making. These perceptual signals are also often uncertain: another person's scowling facial expression may indicate threat or intense concentration, alternatives that require different responses from the perceiver. Decisions made under perceptual uncertainty form the basis of the signals framework of decision making. Traditional behavioral economic approaches to decision making focus on the uncertainty that comes from variability in possible outcome values, and typically ignore the influence of perceptual uncertainty. Conversely, traditional signal detection approaches to decision making focus on the uncertainty that arises from variability in perceptual signals and typically ignore the influence of outcome value uncertainty. Here, we compare and contrast the economic and signals frameworks that guide research in decision making, with the aim of promoting their integration. We show that an integrated framework can expand our ability to understand a wider variety of decision-making behaviors, in particular the complexly determined real-world decisions we all make every day.

The ecology of human fear: survival optimization and the nervous system

We propose a Survival Optimization System (SOS) to account for the strategies that humans and other animals use to defend against recurring and novel threats. The SOS attempts to merge ecological models that define a repertoire of contextually relevant threat induced survival behaviors with contemporary approaches to human affective science. We first propose that the goal of the nervous system is to reduce surprise and optimize actions by (i) predicting the sensory landscape by simulating possible encounters with threat and selecting the appropriate pre-encounter action and (ii) prevention strategies in which the organism manufactures safe environments. When a potential threat is encountered the (iii) threat orienting system is engaged to determine whether the organism ignores the stimulus or switches into a process of (iv) threat assessment, where the organism monitors the stimulus, weighs the threat value, predicts the actions of the threat, searches for safety, and guides behavioral actions crucial to directed escape. When under imminent attack, (v) defensive systems evoke fast reflexive indirect escape behaviors (i.e., fight or flight). This cascade of responses to threat of increasing magnitude are underwritten by an interconnected neural architecture that extends from cortical and hippocampal circuits, to attention, action and threat systems including the amygdala, striatum, and hard-wired defensive systems in the midbrain. The SOS also includes a modulatory feature consisting of cognitive appraisal systems that flexibly guide perception, risk and action. Moreover, personal and vicarious threat encounters fine-tune avoidance behaviors via model-based learning, with higher organisms bridging data to reduce face-to-face encounters with predators. Our model attempts to unify the divergent field of human affective science, proposing a highly integrated nervous system that has evolved to increase the organism's chances of survival.

Banishing the Control Homunculi in Studies of Action Control and Behavior Change

For centuries, human self-control has fascinated scientists and nonscientists alike. Current theories often attribute it to an executive control system. But even though executive control receives a great deal of attention across disciplines, most aspects of it are still poorly understood. Many theories rely on an ill-defined set of "homunculi" doing jobs like "response inhibition" or "updating" without explaining how they do so. Furthermore, it is not always appreciated that control takes place across different timescales. These two issues hamper major advances. Here we focus on the mechanistic basis for the executive control of actions. We propose that at the most basic level, action control depends on three cognitive processes: signal detection, action selection, and action execution. These processes are modulated via error-correction or outcome-evaluation mechanisms, preparation, and task rules maintained in working and long-term memory. We also consider how executive control of actions becomes automatized with practice and how people develop a control network. Finally, we discuss how the application of this unified framework in clinical domains can increase our understanding of control deficits and provide a theoretical basis for the development of novel behavioral change interventions.

Encoding and decoding in parietal cortex during sensorimotor decision-making

The lateral intraparietal area (LIP) of macaques has been asserted to play a fundamental role in sensorimotor decision-making. Here we dissect the neural code in LIP at the level of individual trial spike trains using a statistical approach based on generalized linear models. We show that LIP responses reflect a combination of temporally-overlapping task and decision-related signals. Our model accounts for the detailed statistics of LIP spike trains, and accurately predicts spike trains from task events on single trials. Moreover, we derive an optimal decoder for heterogeneous, multiplexed LIP responses that could be implemented in biologically plausible circuits. In contrast to interpretations of LIP as providing an instantaneous code for decision variables, we show that optimal decoding requires integrating LIP spikes over two timescales. These analyses provide a detailed understanding of the neural code in LIP, and a framework for studying the coding of multiplexed signals in higher brain areas.

Network plasticity in adaptive filtering and behavioral habituation

The ability of organisms to seamlessly ignore familiar, inconsequential stimuli improves their selective attention and response to salient features of the environment. Here, I propose that this fundamental but unexplained phenomenon substantially derives from the ability of any pattern of neural excitation to create an enhanced inhibitory (or "negative") image of itself through target-specific scaling of inhibitory inputs onto active excitatory neurons. Familiar stimuli encounter strong negative images and are therefore less likely to be transmitted to higher brain centers. Integrating historical and recent observations, the negative-image model described here provides a mechanistic framework for understanding habituation, which is connected to ideas on dynamic predictive coding. In addition, it suggests insights for understanding autism spectrum disorders.

The role of reward prediction in the control of attention

Previously rewarded stimuli involuntarily capture attention. The learning mechanisms underlying this value-driven attentional capture remain less understood. We tested whether theories of prediction-based associative reward learning explain the conditions under which reward feedback leads to value-based modulations of attentional priority. Across 4 experiments, we manipulated whether stimulus features served as unique predictors of reward outcomes. Participants received monetary rewards for correctly identifying a color-defined target in an initial search task (training phase) and then immediately completed a second, unrewarded visual search task in which color was irrelevant (test phase). In Experiments 1-3, monetary reward followed correct target selection during training, but critically, no target-defining features carried uniquely predictive information about reward outcomes. Under these conditions, we found no evidence of attentional capture by the previous target colors in the subsequent test phase. Conversely, when target colors in the training phase of Experiment 4 carried uniquely predictive information about reward magnitude, we observed significant attentional capture by the previously rewarded color. Our findings show that value-based attentional priority only develops for stimulus features that carry uniquely predictive information about reward, ruling out a purely motivational account and suggesting that mechanisms of reward prediction play an important role in shaping attentional priorities.

Novelty enhances visual salience independently of reward in the parietal lobe

Novelty modulates sensory and reward processes, but it remains unknown how these effects interact, i.e., how the visual effects of novelty are related to its motivational effects. A widespread hypothesis, based on findings that novelty activates reward-related structures, is that all the effects of novelty are explained in terms of reward. According to this idea, a novel stimulus is by default assigned high reward value and hence high salience, but this salience rapidly decreases if the stimulus signals a negative outcome. Here we show that, contrary to this idea, novelty affects visual salience in the monkey lateral intraparietal area (LIP) in ways that are independent of expected reward. Monkeys viewed peripheral visual cues that were novel or familiar (received few or many exposures) and predicted whether the trial will have a positive or a negative outcome--i.e., end in a reward or a lack of reward. We used a saccade-based assay to detect whether the cues automatically attracted or repelled attention from their visual field location. We show that salience--measured in saccades and LIP responses--was enhanced by both novelty and positive reward associations, but these factors were dissociable and habituated on different timescales. The monkeys rapidly recognized that a novel stimulus signaled a negative outcome (and withheld anticipatory licking within the first few presentations), but the salience of that stimulus remained high for multiple subsequent presentations. Therefore, novelty can provide an intrinsic bonus for attention that extends beyond the first presentation and is independent of physical rewards.

Designing for psychological change: individuals' reward and cost valuations in weight management

Background

Knowledge of the psychological constructs that underlie behavior offers valuable design opportunities for persuasive systems. We use the decision theory, which describes how behavior is underpinned by reward-cost valuations, as a framework for investigating such psychological constructs to deliver design objectives for weight management technologies.

Objective

We applied a decision theory–based analysis in the domain of weight management to understand the rewards and costs that surround individuals’ weight management behaviors, with the aim of uncovering design opportunities for weight management technologies.

Methods

We conducted qualitative interviews with 15 participants who were or had been trying to lose weight. Thematic analysis was used to extract themes that covered the rewards and costs surrounding weight management behaviors. We supplemented our qualitative study with a quantitative survey of 100 respondents investigating the extent to which they agreed with statements reflecting themes from the qualitative study.

Results

The primary obstacles to weight management were the rewards associated with unhealthy choices, such as the pleasures of unhealthy foods and unrestricted consumption in social situations, and the significant efforts required to change habits, plan, and exercise. Psychological constructs that supported positive weight management included feeling good after making healthy choices, being good to oneself, experiencing healthy yet still delicious foods, and receiving social support and encouraging messages (although opinions about encouraging messages was mixed).

Conclusions

A rewards-costs driven enquiry revealed a wide range of psychological constructs that contribute to discouraging and supporting weight management. The constructs extracted from our qualitative study were verified by our quantitative survey, in which the majority of respondents also reported similar thoughts and feelings. This understanding of the rewards and costs surrounding weight management offers a range of new opportunities for the design of weight management technologies that enhance the encouraging factors and alleviate the discouraging ones.

Neural correlates of temporal credit assignment in the parietal lobe

Empirical studies of decision making have typically assumed that value learning is governed by time, such that a reward prediction error arising at a specific time triggers temporally-discounted learning for all preceding actions. However, in natural behavior, goals must be acquired through multiple actions, and each action can have different significance for the final outcome. As is recognized in computational research, carrying out multi-step actions requires the use of credit assignment mechanisms that focus learning on specific steps, but little is known about the neural correlates of these mechanisms. To investigate this question we recorded neurons in the monkey lateral intraparietal area (LIP) during a serial decision task where two consecutive eye movement decisions led to a final reward. The underlying decision trees were structured such that the two decisions had different relationships with the final reward, and the optimal strategy was to learn based on the final reward at one of the steps (the “F” step) but ignore changes in this reward at the remaining step (the “I” step). In two distinct contexts, the F step was either the first or the second in the sequence, controlling for effects of temporal discounting. We show that LIP neurons had the strongest value learning and strongest post-decision responses during the transition after the F step regardless of the serial position of this step. Thus, the neurons encode correlates of temporal credit assignment mechanisms that allocate learning to specific steps independently of temporal discounting.

Reinforcement learning modulates the stability of cognitive control settings for object selection

Cognitive flexibility reflects both a trait that reliably differs between individuals and a state that can fluctuate moment-to-moment. Whether individuals can undergo persistent changes in cognitive flexibility as a result of reward learning is less understood. Here, we investigated whether reinforcing a periodic shift in an object selection strategy can make an individual more prone to switch strategies in a subsequent unrelated task. Participants completed two different choice tasks in which they selected one of four objects in an attempt to obtain a hidden reward on each trial. During a training phase, objects were defined by color. Participants received either consistent reward contingencies in which one color was more often rewarded, or contingencies in which the color that was more often rewarded changed periodically and without warning. Following the training phase, all participants completed a test phase in which reward contingencies were defined by spatial location and the location that was more often rewarded remained constant across the entire task. Those participants who received inconsistent contingencies during training continued to make more variable selections during the test phase in comparison to those who received the consistent training. Furthermore, a difference in the likelihood to switch selections on a trial-by-trial basis emerged between training groups: participants who received consistent contingencies during training were less likely to switch object selections following an unrewarded trial and more likely to repeat a selection following reward. Our findings provide evidence that the extent to which priority shifting is reinforced modulates the stability of cognitive control settings in a persistent manner, such that individuals become generally more or less prone to shifting priorities in the future.

Reward-based transfer from bottom-up to top-down search tasks

Recent evidence has suggested that reward modulates bottom-up and top-down attentional selection and that this effect persists within the same task even when reward is no longer offered. It remains unclear whether reward effects transfer across tasks, especially those engaging different modes of attention. We directly investigated whether reward-based contingency learned in a bottom-up search task was transferred to a subsequent top-down search task, and probed the nature of the transfer mechanism. Results showed that a reward-related benefit established in a pop-out-search task was transferred to a conjunction-search task, increasing participants' efficiency at searching for targets previously associated with a higher level of reward. Reward history influenced search efficiency by enhancing both target salience and distractor filtering, depending on whether the target and distractors shared a critical feature. These results provide evidence for reward-based transfer between different modes of attention and strongly suggest that an integrated priority map based on reward information guides both top-down and bottom-up attention.

The visual corticostriatal loop through the tail of the caudate: circuitry and function

Although high level visual cortex projects to a specific region of the striatum, the tail of the caudate, and participates in corticostriatal loops, the function of this visual corticostriatal system is not well understood. This article first reviews what is known about the anatomy of the visual corticostriatal loop across mammals, including rodents, cats, monkeys, and humans. Like other corticostriatal systems, the visual corticostriatal system includes both closed loop components (recurrent projections that return to the originating cortical location) and open loop components (projections that terminate in other neural regions). The article then reviews what previous empirical research has shown about the function of the tail of the caudate. The article finally addresses the possible functions of the closed and open loop connections of the visual loop in the context of theories and computational models of corticostriatal function.

Attention as foraging for information and value

What is the purpose of attention? One avenue of research has led to the proposal that attention might be crucial for gathering information about the environment, while other lines of study have demonstrated how attention may play a role in guiding behavior to rewarded options. Many experiments that study attention require participants to make a decision based on information acquired discretely at one point in time. In real-world situations, however, we are usually not presented with information about which option to select in such a manner. Rather we must initially search for information, weighing up reward values of options before we commit to a decision. Here, we propose that attention plays a role in both foraging for information and foraging for value. When foraging for information, attention is guided toward the unknown. When foraging for reward, attention is guided toward high reward values, allowing decision-making to proceed by accept-or-reject decisions on the currently attended option. According to this account, attention can be regarded as a low-cost alternative to moving around and physically interacting with the environment—“teleforaging”—before a decision is made to interact physically with the world. To track the timecourse of attention, we asked participants to seek out and acquire information about two gambles by directing their gaze, before choosing one of them. Participants often made multiple refixations on items before making a decision. Their eye movements revealed that early in the trial, attention was guided toward information, i.e., toward locations that reduced uncertainty about value. In contrast, late in the trial, attention was guided by expected value of the options. At the end of the decision period, participants were generally attending to the item they eventually chose. We suggest that attentional foraging shifts from an uncertainty-driven to a reward-driven mode during the evolution of a decision, permitting decisions to be made by an engage-or-search strategy.

Production, control, and visual guidance of saccadic eye movements

Primate vision is served by rapid shifts of gaze called saccades. This review will survey current knowledge and particular problems concerning the neural control and guidance of gaze shifts.

Information-seeking, curiosity, and attention: computational and neural mechanisms

Intelligent animals devote much time and energy to exploring and obtaining information, but the underlying mechanisms are poorly understood. We review recent developments on this topic that have emerged from the traditionally separate fields of machine learning, eye movements in natural behavior, and studies of curiosity in psychology and neuroscience. These studies show that exploration may be guided by a family of mechanisms that range from automatic biases toward novelty or surprise to systematic searches for learning progress and information gain in curiosity-driven behavior. In addition, eye movements reflect visual information searching in multiple conditions and are amenable for cellular-level investigations. This suggests that the oculomotor system is an excellent model system for understanding information-sampling mechanisms.

The differential effects of reward on space- and object-based attentional allocation

Estimating reward contingencies and allocating attentional resources to a subset of relevant information are the most important contributors to increasing adaptability of an organism. Although recent evidence suggests that reward- and attention-based guidance recruits overlapping cortical regions and has similar effects on sensory responses, the exact nature of the relationship between the two remains elusive. Here, using event-related fMRI on human participants, we contrasted the effects of reward on space- and object-based selection in the same experimental setting. Reward was either distributed randomly or biased a particular object. Behavioral and neuroimaging results show that space- and object-based attention is influenced by reward differentially. Space-based attentional allocation is mandatory, integrating reward information over time, whereas object-based attentional allocation is a default setting that is completely replaced by the reward signal. Nonadditivity of the effects of reward and object-based attention was observed consistently at multiple levels of analysis in early visual areas as well as in control regions. These results provide strong evidence that space- and object-based allocation are two independent attentional mechanisms, and suggest that reward serves to constrain attentional selection.

Prefrontal neurons transmit signals to parietal neurons that reflect executive control of cognition

Prefrontal cortex influences behavior largely through its connections with other association cortices; however the nature of the information conveyed by prefrontal output signals and what effect these signals have on computations performed by target structures is largely unknown. To address these questions, we simultaneously recorded the activity of neurons in prefrontal and posterior parietal cortex of monkeys performing a rule-based spatial categorization task. Parietal cortex receives direct prefrontal input, and parietal neurons, like their prefrontal counterparts, exhibit signals that reflect rule-based cognitive processing in this task. By analyzing rapid fluctuations in the cognitive information encoded by activity in the two areas, we obtained evidence that signals reflecting rule-dependent categories were selectively transmitted in a top-down direction from prefrontal to parietal neurons, suggesting prefrontal output is important for the executive control of distributed cognitive processing.

Attentional resource allocation during a cued saccade task

Attentional selection of sensory information and motor output is critical for successful interaction with one's surroundings. However, organization of attentional processes involved in selection of salient visual information, decision making, and movement planning has not yet been fully elucidated. We hypothesized that attentional processes involved in these tasks can function independently and draw from separate resources. If true, challenging the capacity limit of one attentional process would not affect performance of others. Healthy participants performed a cued saccade task in which target cues were embedded in a central stream of letters in a Rapid Serial Visual Presentation (RSVP). Participants performed saccades as quickly and as accurately as possible to a peripheral target location based on cue presentation within the central letter stream. To challenge visual attention, we parametrically varied the duration at which each letter of the RSVP was presented (50-200ms). In a separate experiment we challenged motor attention by increasing the number of possible saccade trajectories (1-6 peripheral targets). As expected, increasing attentional load in one domain of the task negatively affected performance in that domain, while performance in other domains was unaffected. We interpret our results as support for the independent allocation of attentional resources, at least in the early stages of processing, required across components of a cued saccade task. Deciphering the contributions of attention during visuomotor tasks is a critical step to understanding how humans process information necessary to successfully interact with the environment.

Dynamic excitatory and inhibitory gain modulation can produce flexible, robust and optimal decision-making

Behavioural and neurophysiological studies in primates have increasingly shown the involvement of urgency signals during the temporal integration of sensory evidence in perceptual decision-making. Neuronal correlates of such signals have been found in the parietal cortex, and in separate studies, demonstrated attention-induced gain modulation of both excitatory and inhibitory neurons. Although previous computational models of decision-making have incorporated gain modulation, their abstract forms do not permit an understanding of the contribution of inhibitory gain modulation. Thus, the effects of co-modulating both excitatory and inhibitory neuronal gains on decision-making dynamics and behavioural performance remain unclear. In this work, we incorporate time-dependent co-modulation of the gains of both excitatory and inhibitory neurons into our previous biologically based decision circuit model. We base our computational study in the context of two classic motion-discrimination tasks performed in animals. Our model shows that by simultaneously increasing the gains of both excitatory and inhibitory neurons, a variety of the observed dynamic neuronal firing activities can be replicated. In particular, the model can exhibit winner-take-all decision-making behaviour with higher firing rates and within a significantly more robust model parameter range. It also exhibits short-tailed reaction time distributions even when operating near a dynamical bifurcation point. The model further shows that neuronal gain modulation can compensate for weaker recurrent excitation in a decision neural circuit, and support decision formation and storage. Higher neuronal gain is also suggested in the more cognitively demanding reaction time than in the fixed delay version of the task. Using the exact temporal delays from the animal experiments, fast recruitment of gain co-modulation is shown to maximize reward rate, with a timescale that is surprisingly near the experimentally fitted value. Our work provides insights into the simultaneous and rapid modulation of excitatory and inhibitory neuronal gains, which enables flexible, robust, and optimal decision-making.

Perceptual decision-making involves not only simple transformation of sensory information to a motor decision, but can also be modulated by high-level cognition. For example, the latter may include strategic allocation of limited attentional resources over time in a decision task to improve performance. At the neurophysiological level, there is evidence supporting attention-induced neuronal gain modulation of both excitatory and inhibitory cortical neurons. In the context of perceptual discrimination tasks performed by animals, we make use of a biologically inspired computational model of decision-making to understand the computational capabilities of such co-modulation of neuronal gains. We find that dynamic co-modulation of both excitatory and inhibitory neurons is important for flexible, and cognitively demanding decision-making while also enhancing robustness in the decision circuit's functions. Our model captures the neuronal activity and behavioural data in the animal experiments remarkably well. Decision performance in a reaction time task can be optimized, maximizing the rate of receiving reward by using fast gain recruitment. Our experimentally fitted timescale is near the optimal one, suggesting that the animals performed almost optimally. By providing both computational simulations and theoretical analyses, our computational model sheds light into the multiple functions of rapid co-modulation of neuronal gains during decision-making.

Visual encoding and fixation target selection in free viewing: presaccadic brain potentials

In scrutinizing a scene, the eyes alternate between fixations and saccades. During a fixation, two component processes can be distinguished: visual encoding and selection of the next fixation target. We aimed to distinguish the neural correlates of these processes in the electrical brain activity prior to a saccade onset. Participants viewed color photographs of natural scenes, in preparation for a change detection task. Then, for each participant and each scene we computed an image heat map, with temperature representing the duration and density of fixations. The temperature difference between the start and end points of saccades was taken as a measure of the expected task-relevance of the information concentrated in specific regions of a scene. Visual encoding was evaluated according to whether subsequent change was correctly detected. Saccades with larger temperature difference were more likely to be followed by correct detection than ones with smaller temperature differences. The amplitude of presaccadic activity over anterior brain areas was larger for correct detection than for detection failure. This difference was observed for short "scrutinizing" but not for long "explorative" saccades, suggesting that presaccadic activity reflects top-down saccade guidance. Thus, successful encoding requires local scanning of scene regions which are expected to be task-relevant. Next, we evaluated fixation target selection. Saccades "moving up" in temperature were preceded by presaccadic activity of higher amplitude than those "moving down". This finding suggests that presaccadic activity reflects attention deployed to the following fixation location. Our findings illustrate how presaccadic activity can elucidate concurrent brain processes related to the immediate goal of planning the next saccade and the larger-scale goal of constructing a robust representation of the visual scene.