Value-based modulations in human visual cortex

Enriched encoding: reward motivation organizes cortical networks for hippocampal detection of unexpected events

Learning how to obtain rewards requires learning about their contexts and likely causes. How do long-term memory mechanisms balance the need to represent potential determinants of reward outcomes with the computational burden of an over-inclusive memory? One solution would be to enhance memory for salient events that occur during reward anticipation, because all such events are potential determinants of reward. We tested whether reward motivation enhances encoding of salient events like expectancy violations. During functional magnetic resonance imaging, participants performed a reaction-time task in which goal-irrelevant expectancy violations were encountered during states of high- or low-reward motivation. Motivation amplified hippocampal activation to and declarative memory for expectancy violations. Connectivity of the ventral tegmental area (VTA) with medial prefrontal, ventrolateral prefrontal, and visual cortices preceded and predicted this increase in hippocampal sensitivity. These findings elucidate a novel mechanism whereby reward motivation can enhance hippocampus-dependent memory: anticipatory VTA-cortical-hippocampal interactions. Further, the findings integrate literatures on dopaminergic neuromodulation of prefrontal function and hippocampus-dependent memory. We conclude that during reward motivation, VTA modulation induces distributed neural changes that amplify hippocampal signals and records of expectancy violations to improve predictions-a potentially unique contribution of the hippocampus to reward learning.

High stakes trigger the use of multiple memories to enhance the control of attention

We can more precisely tune attention to highly rewarding objects than other objects in our environment, but how our brains do this is unknown. After a few trials of searching for the same object, subjects' electrical brain activity indicated that they handed off the memory representations used to control attention from working memory to long-term memory. However, when a large reward was possible, the neural signature of working memory returned as subjects recruited working memory to supplement the cognitive control afforded by the representations accumulated in long-term memory. The amplitude of this neural signature of working memory predicted the magnitude of the subsequent behavioral reward-based attention effects across tasks and individuals, showing the ubiquity of this cognitive reaction to high-stakes situations.

Where do we store the memory representations that guide attention?

During the last decade one of the most contentious and heavily studied topics in the attention literature has been the role that working memory representations play in controlling perceptual selection. The hypothesis has been advanced that to have attention select a certain perceptual input from the environment, we only need to represent that item in working memory. Here we summarize the work indicating that the relationship between what representations are maintained in working memory and what perceptual inputs are selected is not so simple. First, it appears that attentional selection is also determined by high-level task goals that mediate the relationship between working memory storage and attentional selection. Second, much of the recent work from our laboratory has focused on the role of long-term memory in controlling attentional selection. We review recent evidence supporting the proposal that working memory representations are critical during the initial configuration of attentional control settings, but that after those settings are established long-term memory representations play an important role in controlling which perceptual inputs are selected by mechanisms of attention.

Visual cortical networks: of mice and men

The visual cortical network consists of a number of specialized areas that are connected in a highly structured way. Understanding the function of this network is a milestone goal of visual neuroscience. This goal is pursued at different levels of description, including large-scale neuroanatomical as well as molecular and cellular perspectives. As a consequence, visual cortical networks are studied with a diverse set of methods across the order of mammalian species, ranging from the human all the way down to the mouse. Remarkable progress has been made at both ends of the spectrum. On the basis of work in humans, the last decade has seen ongoing refinements of the intricate functional organization of the cortical visual network. Neuroimaging studies have opened up the possibility to map individual visual areas, characterize their function and, search for an overarching organizational principle. Meanwhile, the mouse has become a valuable model system for early visual processing. A number of studies have demonstrated that basic response properties observed in higher-order mammals are also present in the mouse, making it possible to apply genetic tools to study visual network function. Here, we discuss the progress in these two fields side-by-side. We summarize new findings that have shaped our current understanding of the human cortical network. In addition, we review recent work that has laid the foundation for a mouse model of visual cortical processing. Although their brains are different, the visual cortical networks of mice and men share structural and functional principles.

Rewards teach visual selective attention

Visual selective attention is the brain function that modulates ongoing processing of retinal input in order for selected representations to gain privileged access to perceptual awareness and guide behavior. Enhanced analysis of currently relevant or otherwise salient information is often accompanied by suppressed processing of the less relevant or salient input. Recent findings indicate that rewards exert a powerful influence on the deployment of visual selective attention. Such influence takes different forms depending on the specific protocol adopted in the given study. In some cases, the prospect of earning a larger reward in relation to a specific stimulus or location biases attention accordingly in order to maximize overall gain. This is mediated by an effect of reward acting as a type of incentive motivation for the strategic control of attention. In contrast, reward delivery can directly alter the processing of specific stimuli by increasing their attentional priority, and this can be measured even when rewards are no longer involved, reflecting a form of reward-mediated attentional learning. As a further development, recent work demonstrates that rewards can affect attentional learning in dissociable ways depending on whether rewards are perceived as feedback on performance or instead are registered as random-like events occurring during task performance. Specifically, it appears that visual selective attention is shaped by two distinct reward-related learning mechanisms: one requiring active monitoring of performance and outcome, and a second one detecting the sheer association between objects in the environment (whether attended or ignored) and the more-or-less rewarding events that accompany them. Overall this emerging literature demonstrates unequivocally that rewards "teach" visual selective attention so that processing resources will be allocated to objects, features and locations which are likely to optimize the organism's interaction with the surrounding environment and maximize positive outcome.

Persistence of value-driven attentional capture

Stimuli that have previously been associated with the delivery of reward involuntarily capture attention when presented as unrewarded and task-irrelevant distractors in a subsequent visual search task. It is unknown how long such effects of reward learning on attention persist. One possibility is that value-driven attentional biases are plastic and constantly evolve to reflect only recent reward history. According to such a mechanism of attentional control, only consistently reinforced patterns of attention allocation persist for extended periods of time. Another possibility is that reward learning creates enduring changes in attentional priority that can persist indefinitely without further learning. Here we provide evidence for an enduring effect of reward learning on attentional priority: stimuli previously associated with reward in a training phase capture attention when presented as irrelevant distractors over half a year later, without the need for further reward learning.

Reward associations magnify memory-based biases on perception

Long-term spatial contextual memories are a rich source of predictions about the likely locations of relevant objects in the environment and should enable tuning of neural processing of unfolding events to optimize perception and action. Of particular importance is whether and how the reward outcome of past events can impact perception. We combined behavioral measures with recordings of brain activity with high temporal resolution to test whether the previous reward outcome associated with a memory could modulate the impact of memory-based biases on perception, and if so, the level(s) at which visual neural processing is biased by reward-associated memory-guided attention. Data showed that past rewards potentiate the effects of spatial memories upon the discrimination of target objects embedded within complex scenes starting from early perceptual stages. We show that a single reward outcome of learning impacts on how we perceive events in our complex environments.

A sensitive period for language in the visual cortex: distinct patterns of plasticity in congenitally versus late blind adults

Recent evidence suggests that blindness enables visual circuits to contribute to language processing. We examined whether this dramatic functional plasticity has a sensitive period. BOLD fMRI signal was measured in congenitally blind, late blind (blindness onset 9-years-old or later) and sighted participants while they performed a sentence comprehension task. In a control condition, participants listened to backwards speech and made match/non-match to sample judgments. In both congenitally and late blind participants BOLD signal increased in bilateral foveal-pericalcarine cortex during response preparation, irrespective of whether the stimulus was a sentence or backwards speech. However, left occipital areas (pericalcarine, extrastriate, fusiform and lateral) responded more to sentences than backwards speech only in congenitally blind people. We conclude that age of blindness onset constrains the non-visual functions of occipital cortex: while plasticity is present in both congenitally and late blind individuals, recruitment of visual circuits for language depends on blindness during childhood.

Neural correlates of the formation and retention of cocaine-induced stimulus-reward associations

BACKGROUND

Cocaine can elicit drug-seeking behavior for drug-predicting stimuli, even after a single stimulus-cocaine pairing. Although orbitofrontal cortex is thought to be important during encoding and maintenance of stimulus-reward value, we still lack a comprehensive model of the neural circuitry underlying this cognitive process.

METHODS

We studied the conditioned effects of cocaine with monkey functional magnetic resonance imaging and classical conditioning by pairing a visual shape (conditioning stimulus [CS+]) with a noncontingent cocaine infusion; a control stimulus was never paired. We correlated the behavioral preference of the monkey for the CS+, as measured offline, with the activity induced by the CS+ relative to the control stimulus as function of time.

RESULTS

We observed that during formation of stimulus-cocaine associations strong CS+-induced functional magnetic resonance imaging activations emerged in frontal cortex that correlated significantly with behavioral CS+ preference. Afterward, CS+ preference correlated only with activity in early visual cortex. Control experiments suggest that these findings cannot be explained by increased familiarity for the CS+.

CONCLUSIONS

Our findings suggest a complex interaction between frontal and occipital cortex during cocaine conditioning. Frontal cortex is important for establishing novel representations of stimulus valence when cocaine is used as reinforcer, whereas early visual cortex is involved in retaining these cocaine-stimulus associations.

Implicit motivational value and salience are processed in distinct areas of orbitofrontal cortex

Recent studies have shown that motivational stimulus information is represented in the brain even in situations where the individual is not actively engaged in stimulus evaluation. However, it has remained unclear whether neural representations of such implicit motivational information reflect the motivational value or motivational salience of stimuli. While motivational values correspond to the desirability of stimuli, motivational salience is related to the arousal elicited by the stimulus. Here we aimed at disentangling the neural representation of both motivational dimensions. In the first part, participants learned the association of face stimuli with monetary reward and punishment. The same face stimuli were presented in a subsequent fMRI experiment, during which participants either performed a gender discrimination task on the faces or an orientation discrimination task on two simultaneously presented bars. Importantly, faces only differed regarding their implicit motivational information as acquired in the previous learning task, as participants neither received monetary reinforcement during the fMRI experiment nor were they asked to explicitly judge their face preferences. We found that neural responses in lateral OFC were modulated by implicit motivational value, whereas the faces' implicit motivational salience was coded in medial OFC. While the value-related responses in lateral OFC decreased over time, the salience-related modulation of medial OFC activity remained stable over the duration of the fMRI experiment. Neural responses to both motivational dimensions were observed independent of whether participants' attention was directed to the faces or to the surrounding bars, suggesting an automatic processing of implicit motivational value and salience. The functional dissociation within the OFC suggests that this region is critically involved in distinct motivation-related processes: In medial OFC, a representation of salient items may be maintained in order to facilitate responses towards behaviourally relevant stimuli in the future; in contrast the temporary value effect in lateral OFC might reflect decreasing stimulus valuation in the absence of explicit motivational stimulus differences.

Valence-specific modulation in the accumulation of perceptual evidence prior to visual scene recognition

Visual scene recognition is a dynamic process through which incoming sensory information is iteratively compared with predictions regarding the most likely identity of the input stimulus. In this study, we used a novel progressive unfolding task to characterize the accumulation of perceptual evidence prior to scene recognition, and its potential modulation by the emotional valence of these scenes. Our results show that emotional (pleasant and unpleasant) scenes led to slower accumulation of evidence compared to neutral scenes. In addition, when controlling for the potential contribution of non-emotional factors (i.e., familiarity and complexity of the pictures), our results confirm a reliable shift in the accumulation of evidence for pleasant relative to neutral and unpleasant scenes, suggesting a valence-specific effect. These findings indicate that proactive iterations between sensory processing and top-down predictions during scene recognition are reliably influenced by the rapidly extracted (positive) emotional valence of the visual stimuli. We interpret these findings in accordance with the notion of a genuine positivity offset during emotional scene recognition.

Social attention: a possible early indicator of efficacy in autism clinical trials

For decades, researchers have sought to clarify the nature of the social communication impairments in autism, highlighting impaired or atypical 'social attention' as a key measurable construct that helps to define the core impairment of social communication. In this paper, we provide an overview of research on social attention impairments in autism and their relation to deficiencies in neural circuitry related to social reward. We offer a framework for considering social attention as a potential moderator or mediator of response to early behavioral intervention, and as an early indicator of efficacy of behavioral and/or pharmacological treatments aimed at addressing the social impairments in autism.

Dynamic integration of information about salience and value for saccadic eye movements

Humans shift their gaze to a new location several times per second. It is still unclear what determines where they look next. Fixation behavior is influenced by the low-level salience of the visual stimulus, such as luminance, contrast, and color, but also by high-level task demands and prior knowledge. Under natural conditions, different sources of information might conflict with each other and have to be combined. In our paradigm, we trade off visual salience against expected value. We show that both salience and value information influence the saccadic end point within an object, but with different time courses. The relative weights of salience and value are not constant but vary from eye movement to eye movement, depending critically on the availability of the value information at the time when the saccade is programmed. Short-latency saccades are determined mainly by salience, but value information is taken into account for long-latency saccades. We present a model that describes these data by dynamically weighting and integrating detailed topographic maps of visual salience and value. These results support the notion of independent neural pathways for the processing of visual information and value.

Bias in the brain: a diffusion model analysis of prior probability and potential payoff

In perceptual decision-making, advance knowledge biases people toward choice alternatives that are more likely to be correct and more likely to be profitable. Accumulation-to-bound models provide two possible explanations for these effects: prior knowledge about the relative attractiveness of the alternatives at hand changes either the starting point of the decision process, or the rate of evidence accumulation. Here, we used model-based functional MRI to investigate whether these effects are similar for different types of prior knowledge, and whether there is a common neural substrate underlying bias in simple perceptual choices. We used two versions of the random-dot motion paradigm in which we manipulated bias by: (1) changing the prior likelihood of occurrence for two alternatives ("prior probability") and (2) assigning a larger reward to one of two alternatives ("potential payoff"). Human subjects performed the task inside and outside a 3T MRI scanner. For each manipulation, bias was quantified by fitting the drift diffusion model to the behavioral data. Individual measurements of bias were then used in the imaging analyses to identify regions involved in biasing choice behavior. Behavioral results showed that subjects tended to make more and faster choices toward the alternative that was most probable or had the largest payoff. This effect was primarily due to a change in the starting point of the accumulation process. Imaging results showed that, at cue level, regions of the frontoparietal network are involved in changing the starting points in both manipulations, suggesting a common mechanism underlying the biasing effects of prior knowledge.

Spatial pattern of BOLD fMRI activation reveals cross-modal information in auditory cortex

Recent findings suggest that neural representations in early auditory cortex reflect not only the physical properties of a stimulus, but also high-level, top-down, and even cross-modal information. However, the nature of cross-modal information in auditory cortex remains poorly understood. Here, we used pattern analyses of fMRI data to ask whether early auditory cortex contains information about the visual environment. Our data show that 1) early auditory cortex contained information about a visual stimulus when there was no bottom-up auditory signal, and that 2) no influence of visual stimulation was observed in auditory cortex when visual stimuli did not provide a context relevant to audition. Our findings attest to the capacity of auditory cortex to reflect high-level, top-down, and cross-modal information and indicate that the spatial patterns of activation in auditory cortex reflect contextual/implied auditory information but not visual information per se.

Neural basis of reinforcement learning and decision making

Reinforcement learning is an adaptive process in which an animal utilizes its previous experience to improve the outcomes of future choices. Computational theories of reinforcement learning play a central role in the newly emerging areas of neuroeconomics and decision neuroscience. In this framework, actions are chosen according to their value functions, which describe how much future reward is expected from each action. Value functions can be adjusted not only through reward and penalty, but also by the animal's knowledge of its current environment. Studies have revealed that a large proportion of the brain is involved in representing and updating value functions and using them to choose an action. However, how the nature of a behavioral task affects the neural mechanisms of reinforcement learning remains incompletely understood. Future studies should uncover the principles by which different computational elements of reinforcement learning are dynamically coordinated across the entire brain.

Mechanisms underlying cortical activity during value-guided choice

When choosing between two options, correlates of their value are represented in neural activity throughout the brain. Whether these representations reflect activity that is fundamental to the computational process of value comparison, as opposed to other computations covarying with value, is unknown. We investigated activity in a biophysically plausible network model that transforms inputs relating to value into categorical choices. A set of characteristic time-varying signals emerged that reflect value comparison. We tested these model predictions using magnetoencephalography data recorded from human subjects performing value-guided decisions. Parietal and prefrontal signals matched closely with model predictions. These results provide a mechanistic explanation of neural signals recorded during value-guided choice and a means of distinguishing computational roles of different cortical regions whose activity covaries with value.

The functional neuroanatomy of decision making: prefrontal control of thought and action

Humans exhibit a remarkable capacity for flexible thought and action. Despite changing internal needs and external context, individuals maintain stable goals and pursue purposeful action. Functional neuroimaging research examining the neural underpinnings of such behavioral flexibility has progressed within several distinct traditions, as evident in the largely separate literatures on "cognitive control" and on "decision making." Both topics investigate the formulation of desires and intentions, the integration of knowledge and context, and the resolution of conflict and uncertainty. Additionally, each recognizes the fundamental role of the prefrontal cortex in supporting flexible selection of behavior. But despite this notable overlap, neuroimaging studies in cognitive control and decision making have exerted only limited influence on each other, in part due to differences in their theoretical and experimental groundings. Additionally, the precise organization of control processing within prefrontal cortex has remained unclear, fostering an acceptance of vague descriptions of decision making in terms of canonical cognitive control functions such as "inhibition" or "self-control." We suggest a unifying role for models of the hierarchical organization of action selection within prefrontal cortex. These models provide an important conceptual link between decision-making phenomena and cognitive-control processes, potentially facilitating cross-fertilization between these topics.

Reward and attentional control in visual search

It has long been known that the control of attention in visual search depends both on voluntary, top-down deployment according to context-specific goals, and on involuntary, stimulus-driven capture based on the physical conspicuity of perceptual objects. Recent evidence suggests that pairing target stimuli with reward can modulate the voluntary deployment of attention, but there is little evidence that reward modulates the involuntary deployment of attention to task-irrelevant distractors. We report several experiments that investigate the role of reward learning on attentional control. Each experiment involved a training phase and a test phase. In the training phase, different colors were associated with different amounts of monetary reward. In the test phase, color was not task-relevant and participants searched for a shape singleton; in most experiments no reward was delivered in the test phase. We first show that attentional capture by physically salient distractors is magnified by a previous association with reward. In subsequent experiments we demonstrate that physically inconspicuous stimuli previously associated with reward capture attention persistently during extinction--even several days after training. Furthermore, vulnerability to attentional capture by high-value stimuli is negatively correlated across individuals with working memory capacity and positively correlated with trait impulsivity. An analysis of intertrial effects reveals that value-driven attentional capture is spatially specific. Finally, when reward is delivered at test contingent on the task-relevant shape feature, recent reward history modulates value-driven attentional capture by the irrelevant color feature. The influence of learned value on attention may provide a useful model of clinical syndromes characterized by similar failures of cognitive control, including addiction, attention-deficit/hyperactivity disorder, and obesity.

Specific contributions of ventromedial, anterior cingulate, and lateral prefrontal cortex for attentional selection and stimulus valuation

Functional clusters of neurons in the monkey prefrontal and anterior cingulate cortex are involved in guiding attention to the most valuable objects in a scene.

Attentional control ensures that neuronal processes prioritize the most relevant stimulus in a given environment. Controlling which stimulus is attended thus originates from neurons encoding the relevance of stimuli, i.e. their expected value, in hand with neurons encoding contextual information about stimulus locations, features, and rules that guide the conditional allocation of attention. Here, we examined how these distinct processes are encoded and integrated in macaque prefrontal cortex (PFC) by mapping their functional topographies at the time of attentional stimulus selection. We find confined clusters of neurons in ventromedial PFC (vmPFC) that predominantly convey stimulus valuation information during attention shifts. These valuation signals were topographically largely separated from neurons predicting the stimulus location to which attention covertly shifted, and which were evident across the complete medial-to-lateral extent of the PFC, encompassing anterior cingulate cortex (ACC), and lateral PFC (LPFC). LPFC responses showed particularly early-onset selectivity and primarily facilitated attention shifts to contralateral targets. Spatial selectivity within ACC was delayed and heterogeneous, with similar proportions of facilitated and suppressed responses during contralateral attention shifts. The integration of spatial and valuation signals about attentional target stimuli was observed in a confined cluster of neurons at the intersection of vmPFC, ACC, and LPFC. These results suggest that valuation processes reflecting stimulus-specific outcome predictions are recruited during covert attentional control. Value predictions and the spatial identification of attentional targets were conveyed by largely separate neuronal populations, but were integrated locally at the intersection of three major prefrontal areas, which may constitute a functional hub within the larger attentional control network.

To navigate within an environment filled with sensory stimuli, the brain must selectively process only the most relevant sensory information. Identifying and shifting attention to the most relevant sensory stimulus requires integrating information about its sensory features as well as its relative value, that is, whether it's worth noticing. In this study, we describe groups of neurons in the monkey prefrontal cortex that convey signals relating to the value of a stimulus and its defining feature and location at the very moment when attention is shifted to the stimulus. We found that signals conveying information about value were clustered in a ventromedial prefrontal region, and were separated from sensory signals within the anterior cingulate cortex and the lateral prefrontal cortex. The integration of valuation and other “top-down” processes, however, was achieved by neurons clustered at the intersection of ventromedial, anterior cingulate, and lateral prefrontal cortex. We conclude that valuation processes are recruited when attention is shifted, independent of any overt behavior. Moreover, our analysis suggests that valuation processes can bias the initiation of attention shifts, as well as ensure sustained attentional focusing.

Fast and automatic activation of an abstract representation of money in the human ventral visual pathway

Money, when used as an incentive, activates the same neural circuits as rewards associated with physiological needs. However, unlike physiological rewards, monetary stimuli are cultural artifacts: how are monetary stimuli identified in the first place? How and when does the brain identify a valid coin, i.e. a disc of metal that is, by social agreement, endowed with monetary properties? We took advantage of the changes in the Euro area in 2002 to compare neural responses to valid coins (Euros, Australian Dollars) with neural responses to invalid coins that have lost all monetary properties (French Francs, Finnish Marks). We show in magneto-encephalographic recordings, that the ventral visual pathway automatically distinguishes between valid and invalid coins, within only ∼150 ms. This automatic categorization operates as well on coins subjects were familiar with as on unfamiliar coins. No difference between neural responses to scrambled controls could be detected. These results could suggest the existence of a generic, all-purpose neural representation of money that is independent of experience. This finding is reminiscent of a central assumption in economics, money fungibility, or the fact that a unit of money is substitutable to another. From a neural point of view, our findings may indicate that the ventral visual pathway, a system previously thought to analyze visual features such as shape or color and to be influenced by daily experience, could also able to use conceptual attributes such as monetary validity to categorize familiar as well as unfamiliar visual objects. The symbolic abilities of the posterior fusiform region suggested here could constitute an efficient neural substrate to deal with culturally defined symbols, independently of experience, which probably fostered money's cultural emergence and success.

Challenges of Interpreting Frontal Neurons during Value-Based Decision-Making

The frontal cortex is crucial to sound decision-making, and the activity of frontal neurons correlates with many aspects of a choice, including the reward value of options and outcomes. However, rewards are of high motivational significance and have widespread effects on neural activity. As such, many neural signals not directly involved in the decision process can correlate with reward value. With correlative techniques such as electrophysiological recording or functional neuroimaging, it can be challenging to distinguish neural signals underlying value-based decision-making from other perceptual, cognitive, and motor processes. In the first part of the paper, we examine how different value-related computations can potentially be confused. In particular, error-related signals in the anterior cingulate cortex, generated when one discovers the consequences of an action, might actually represent violations of outcome expectation, rather than errors per se. Also, signals generated at the time of choice are typically interpreted as reflecting predictions regarding the outcomes associated with the different choice alternatives. However, these signals could instead reflect comparisons between the presented choice options and previously presented choice alternatives. In the second part of the paper, we examine how value signals have been successfully dissociated from saliency-related signals, such as attention, arousal, and motor preparation in studies employing outcomes with both positive and negative valence. We hope that highlighting these issues will prove useful for future studies aimed at disambiguating the contribution of different neuronal populations to choice behavior.

Learned value magnifies salience-based attentional capture

Visual attention is captured by physically salient stimuli (termed salience-based attentional capture), and by otherwise task-irrelevant stimuli that contain goal-related features (termed contingent attentional capture). Recently, we reported that physically nonsalient stimuli associated with value through reward learning also capture attention involuntarily (Anderson, Laurent, & Yantis, PNAS, 2011). Although it is known that physical salience and goal-relatedness both influence attentional priority, it is unknown whether or how attentional capture by a salient stimulus is modulated by its associated value. Here we show that a physically salient, task-irrelevant distractor previously associated with a large reward slows visual search more than an equally salient distractor previously associated with a smaller reward. This magnification of salience-based attentional capture by learned value extinguishes over several hundred trials. These findings reveal a broad influence of learned value on involuntary attentional capture.

Neural signatures of economic parameters during decision-making: a functional MRI (FMRI), electroencephalography (EEG) and autonomic monitoring study

Adaptive behaviour requires an ability to obtain rewards by choosing between different risky options. Financial gambles can be used to study effective decision-making experimentally, and to distinguish processes involved in choice option evaluation from outcome feedback and other contextual factors. Here, we used a paradigm where participants evaluated 'mixed' gambles, each presenting a potential gain and a potential loss and an associated variable outcome probability. We recorded neural responses using autonomic monitoring, electroencephalography (EEG) and functional neuroimaging (fMRI), and used a univariate, parametric design to test for correlations with the eleven economic parameters that varied across gambles, including expected value (EV) and amount magnitude. Consistent with behavioural economic theory, participants were risk-averse. Gamble evaluation generated detectable autonomic responses, but only weak correlations with outcome uncertainty were found, suggesting that peripheral autonomic feedback does not play a major role in this task. Long-latency stimulus-evoked EEG potentials were sensitive to expected gain and expected value, while alpha-band power reflected expected loss and amount magnitude, suggesting parallel representations of distinct economic qualities in cortical activation and central arousal. Neural correlates of expected value representation were localized using fMRI to ventromedial prefrontal cortex, while the processing of other economic parameters was associated with distinct patterns across lateral prefrontal, cingulate, insula and occipital cortices including default-mode network and early visual areas. These multimodal data provide complementary evidence for distributed substrates of choice evaluation across multiple, predominantly cortical, brain systems wherein distinct regions are preferentially attuned to specific economic features. Our findings extend biologically-plausible models of risky decision-making while providing potential biomarkers of economic representations that can be applied to the study of deficits in motivational behaviour in neurological and psychiatric patients.

The neural underpinnings of how reward associations can both guide and misguide attention

It is commonly accepted that reward is an effective motivator of behavior, but little is known about potential costs resulting from reward associations. Here, we used functional magnetic resonance imaging (fMRI) to investigate the neural underpinnings of such reward-related performance-disrupting effects in a reward-modulated Stroop task in humans. While reward associations in the task-relevant dimension (i.e., ink color) facilitated performance, behavioral detriments were found when the task-irrelevant dimension (i.e., word meaning) implicitly referred to reward-predictive ink colors. Neurally, only relevant reward associations invoked a typical reward-anticipation response in the nucleus accumbens (NAcc), which was in turn predictive of behavioral facilitation. In contrast, irrelevant reward associations increased activity in a medial prefrontal motor-control-related region, namely the presupplementary motor area (pre-SMA), which likely reflects the preemption and inhibition of automatic response tendencies that are amplified by irrelevant reward-related words. This view was further supported by a positive relationship between pre-SMA activity and pronounced response slowing in trials containing reward-related as compared with reward-unrelated incongruent words. Importantly, the distinct neural processes related to the beneficial and detrimental behavioral effects of reward associations appeared to arise from preferential-coding mechanisms in visual-processing areas that were shared by the two stimulus dimensions, suggesting a transfer of reward-related saliency to the irrelevant dimension, but with highly differential behavioral and neural ramifications. More generally, the data demonstrate that even entirely irrelevant reward associations can influence stimulus-processing and response-selection pathways relatively automatically, thereby representing an important flipside of reward-driven performance enhancements.

Prior expectation modulates the interaction between sensory and prefrontal regions in the human brain

How do expectations about the identity of a forthcoming visual stimulus influence the neural mechanisms of perceptual decision making in the human brain? Previous investigations into this issue have mostly involved changing the subjects' attentional focus or the behavioral relevance of certain targets but rarely manipulated subjects' prior expectation about the likely identity of the stimulus. Also, because perceptual decisions were often paired with specific motor responses, it has been difficult to dissociate neural activity that reflects perceptual decisions from motor preparatory activity. Here we designed a task in which we induced prior expectations about the direction of a moving-dot pattern and withheld the stimulus-response mapping until the subjects were prompted to respond. In line with current models of perceptual decision making, we found that subjects' performance was influenced by their expectation about upcoming motion direction. The integration of such information into the decision process was reflected by heightened activity in the dorsolateral prefrontal cortex. Activity in this area reflected the degree to which subjects adjusted their decisions based on the prior expectation cue. Furthermore, there was increased effective connectivity between sensory regions (motion-sensitive medial temporal area MT+) and dorsolateral prefrontal cortex when subjects had a prior expectation about the upcoming motion direction. Dynamic causal modeling suggested that stimulus expectation modulated both the feedforward and feedback connectivity between MT+ and prefrontal cortex. These results provide a mechanism of how prior expectations may affect perceptual decision making, namely by changing neural activity in, and sensory drive to, prefrontal areas.

The involvement of the dopaminergic midbrain and cortico-striatal-thalamic circuits in the integration of reward prospect and attentional task demands

Reward has been shown to promote human performance in multiple task domains. However, an important debate has developed about the uniqueness of reward-related neural signatures associated with such facilitation, as similar neural patterns can be triggered by increased attentional focus independent of reward. Here, we used functional magnetic resonance imaging to directly investigate the neural commonalities and interactions between the anticipation of both reward and task difficulty, by independently manipulating these factors in a cued-attention paradigm. In preparation for the target stimulus, both factors increased activity within the midbrain, dorsal striatum, and fronto-parietal areas, while inducing deactivations in default-mode regions. Additionally, reward engaged the ventral striatum, posterior cingulate, and occipital cortex, while difficulty engaged medial and dorsolateral frontal regions. Importantly, a network comprising the midbrain, caudate nucleus, thalamus, and anterior midcingulate cortex exhibited an interaction between reward and difficulty, presumably reflecting additional resource recruitment for demanding tasks with profitable outcome. This notion was consistent with a negative correlation between cue-related midbrain activity and difficulty-induced performance detriments in reward-predictive trials. Together, the data demonstrate that expected value and attentional demands are integrated in cortico-striatal-thalamic circuits in coordination with the dopaminergic midbrain to flexibly modulate resource allocation for an effective pursuit of behavioral goals.

Value-driven attentional capture

Attention selects which aspects of sensory input are brought to awareness. To promote survival and well-being, attention prioritizes stimuli both voluntarily, according to context-specific goals (e.g., searching for car keys), and involuntarily, through attentional capture driven by physical salience (e.g., looking toward a sudden noise). Valuable stimuli strongly modulate voluntary attention allocation, but there is little evidence that high-value but contextually irrelevant stimuli capture attention as a consequence of reward learning. Here we show that visual search for a salient target is slowed by the presence of an inconspicuous, task-irrelevant item that was previously associated with monetary reward during a brief training session. Thus, arbitrary and otherwise neutral stimuli imbued with value via associative learning capture attention powerfully and persistently during extinction, independently of goals and salience. Vulnerability to such value-driven attentional capture covaries across individuals with working memory capacity and trait impulsivity. This unique form of attentional capture may provide a useful model for investigating failures of cognitive control in clinical syndromes in which value assigned to stimuli conflicts with behavioral goals (e.g., addiction, obesity).

Reciprocal relations between cognitive neuroscience and formal cognitive models: opposites attract?

Cognitive neuroscientists study how the brain implements particular cognitive processes such as perception, learning, and decision-making. Traditional approaches in which experiments are designed to target a specific cognitive process have been supplemented by two recent innovations. First, formal cognitive models can decompose observed behavioral data into multiple latent cognitive processes, allowing brain measurements to be associated with a particular cognitive process more precisely and more confidently. Second, cognitive neuroscience can provide additional data to inform the development of formal cognitive models, providing greater constraint than behavioral data alone. We argue that these fields are mutually dependent; not only can models guide neuroscientific endeavors, but understanding neural mechanisms can provide key insights into formal models of cognition.

Would the field of cognitive neuroscience be advanced by sharing functional MRI data?

During the past two decades, the advent of functional magnetic resonance imaging (fMRI) has fundamentally changed our understanding of brain-behavior relationships. However, the data from any one study add only incrementally to the big picture. This fact raises important questions about the dominant practice of performing studies in isolation. To what extent are the findings from any single study reproducible? Are researchers who lack the resources to conduct a fMRI study being needlessly excluded? Is pre-existing fMRI data being used effectively to train new students in the field? Here, we will argue that greater sharing and synthesis of raw fMRI data among researchers would make the answers to all of these questions more favorable to scientific discovery than they are today and that such sharing is an important next step for advancing the field of cognitive neuroscience.

Reward sharpens orientation coding independently of attention

It has long been known that rewarding improves performance. However it is unclear whether this is due to high level modulations in the output modules of associated neural systems or due to low level mechanisms favoring more “generous” inputs? Some recent studies suggest that primary sensory areas, including V1 and A1, may form part of the circuitry of reward-based modulations, but there is no data indicating whether reward can be dissociated from attention or cross-trial forms of perceptual learning. Here we address this issue with a psychophysical dual task, to control attention, while perceptual performance on oriented targets associated with different levels of reward is assessed by measuring both orientation discrimination thresholds and behavioral tuning functions for tilt values near threshold. We found that reward, at any rate, improved performance. However, higher reward rates showed an improvement of orientation discrimination thresholds by about 50% across conditions and sharpened behavioral tuning functions. Data were unaffected by changing the attentional load and by dissociating the feature of the reward cue from the task-relevant feature. These results suggest that reward may act within the span of a single trial independently of attention by modulating the activity of early sensory stages through a improvement of the signal-to-noise ratio of task-relevant channels.

Representation of reward feedback in primate auditory cortex

It is well established that auditory cortex is plastic on different time scales and that this plasticity is driven by the reinforcement that is used to motivate subjects to learn or to perform an auditory task. Motivated by these findings, we study in detail properties of neuronal firing in auditory cortex that is related to reward feedback. We recorded from the auditory cortex of two monkeys while they were performing an auditory categorization task. Monkeys listened to a sequence of tones and had to signal when the frequency of adjacent tones stepped in downward direction, irrespective of the tone frequency and step size. Correct identifications were rewarded with either a large or a small amount of water. The size of reward depended on the monkeys' performance in the previous trial: it was large after a correct trial and small after an incorrect trial. The rewards served to maintain task performance. During task performance we found three successive periods of neuronal firing in auditory cortex that reflected (1) the reward expectancy for each trial, (2) the reward-size received, and (3) the mismatch between the expected and delivered reward. These results, together with control experiments suggest that auditory cortex receives reward feedback that could be used to adapt auditory cortex to task requirements. Additionally, the results presented here extend previous observations of non-auditory roles of auditory cortex and shows that auditory cortex is even more cognitively influenced than lately recognized.

Neural processing of reward magnitude under varying attentional demands

Central to the organization of behavior is the ability to represent the magnitude of a prospective reward and the costs related to obtaining it. Therein, reward-related neural activations are discounted in dependence of the effort required to resolve a given task. Varying attentional demands of the task might however affect reward-related neural activations. Here we employed fMRI to investigate the neural representation of expected values during a monetary incentive delay task with varying attentional demands. Following a cue, indicating at the same time the difficulty (hard/easy) and the reward magnitude (high/low) of the upcoming trial, subjects performed an attention task and subsequently received feedback about their monetary reward. Consistent with previous results, activity in anterior-cingulate, insular/orbitofrontal and mesolimbic regions co-varied with the anticipated reward-magnitude, but also with the attentional requirements of the task. These activations occurred contingent on action-execution and resembled the response time pattern of the subjects. In contrast, cue-related activations, signaling the forthcoming task-requirements, were only observed within attentional control structures. These results suggest that anticipated reward-magnitude and task-related attentional demands are concurrently processed in partially overlapping neural networks of anterior-cingulate, insular/orbitofrontal, and mesolimbic regions.

Brain dynamics of upstream perceptual processes leading to visual object recognition: a high density ERP topographic mapping study

Recent studies suggest that visual object recognition is a proactive process through which perceptual evidence accumulates over time before a decision can be made about the object. However, the exact electrophysiological correlates and time-course of this complex process remain unclear. In addition, the potential influence of emotion on this process has not been investigated yet. We recorded high density EEG in healthy adult participants performing a novel perceptual recognition task. For each trial, an initial blurred visual scene was first shown, before the actual content of the stimulus was gradually revealed by progressively adding diagnostic high spatial frequency information. Participants were asked to stop this stimulus sequence as soon as they could correctly perform an animacy judgment task. Behavioral results showed that participants reliably gathered perceptual evidence before recognition. Furthermore, prolonged exploration times were observed for pleasant, relative to either neutral or unpleasant scenes. ERP results showed distinct effects starting at 280 ms post-stimulus onset in distant brain regions during stimulus processing, mainly characterized by: (i) a monotonic accumulation of evidence, involving regions of the posterior cingulate cortex/parahippocampal gyrus, and (ii) true categorical recognition effects in medial frontal regions, including the dorsal anterior cingulate cortex. These findings provide evidence for the early involvement, following stimulus onset, of non-overlapping brain networks during proactive processes eventually leading to visual object recognition.

Top-down effects on early visual processing in humans: a predictive coding framework

An increasing number of human electroencephalography (EEG) studies examining the earliest component of the visual evoked potential, the so-called C1, have cast doubts on the previously prevalent notion that this component is impermeable to top-down effects. This article reviews the original studies that (i) described the C1, (ii) linked it to primary visual cortex (V1) activity, and (iii) suggested that its electrophysiological characteristics are exclusively determined by low-level stimulus attributes, particularly the spatial position of the stimulus within the visual field. We then describe conflicting evidence from animal studies and human neuroimaging experiments and provide an overview of recent EEG and magnetoencephalography (MEG) work showing that initial V1 activity in humans may be strongly modulated by higher-level cognitive factors. Finally, we formulate a theoretical framework for understanding top-down effects on early visual processing in terms of predictive coding.

Economic value biases uncertain perceptual choices in the parietal and prefrontal cortices

An observer detecting a noisy sensory signal is biased by the costs and benefits associated with its presence or absence. When these costs and benefits are asymmetric, sensory, and economic information must be integrated to inform the final choice. However, it remains unknown how this information is combined at the neural or computational levels. To address this question, we asked healthy human observers to judge the presence or absence of a noisy sensory signal under economic conditions that favored yes responses (liberal blocks), no responses (conservative blocks), or neither response (neutral blocks). Economic information biased fast choices more than slow choices, suggesting that value and sensory information are integrated early in the decision epoch. More formal simulation analyses using an Ornstein–Uhlenbeck process demonstrated that the influence of economic information was best captured by shifting the origin of evidence accumulation toward the more valuable bound. We then used the computational model to generate trial-by-trial estimates of decision-related evidence that were based on combined sensory and economic information (the decision variable or DV), and regressed these against fMRI activity recorded whilst participants performed the task. Extrastriate visual regions responded to the level of sensory input (momentary evidence), but fMRI signals in the parietal and prefrontal cortices responded to the decision variable. These findings support recent single-neuron data suggesting that economic information biases decision-related signals in higher cortical regions.

Embedding reward signals into perception and cognition

Despite considerable interest in the neural basis of valuation, the question of how valuation affects cognitive processing has received relatively less attention. Here, we review evidence from recent behavioral and neuroimaging studies supporting the notion that motivation can enhance perceptual and executive control processes to achieve more efficient goal-directed behavior. Specifically, in the context of cognitive tasks offering monetary gains, improved behavioral performance has been repeatedly observed in conjunction with elevated neural activations in task-relevant perceptual, cognitive and reward-related regions. We address the neural basis of motivation-cognition interactions by suggesting various modes of communication between relevant neural networks: (1) global hub regions may integrate information from multiple inputs providing a communicative link between specialized networks; (2) point-to-point interactions allow for more specific cross-network communication; and (3) diffuse neuromodulatory systems can relay motivational signals to cortex and enhance signal processing. Together, these modes of communication allow information regarding motivational significance to reach relevant brain regions and shape behavior.

Motor preparatory activity in posterior parietal cortex is modulated by subjective absolute value

Cortical motor planning is shaped by “subjective absolute value”: planning activity is strongly enhanced for large expected gains in subjects who believe they perform well; conversely, activity is higher for large expected losses in subjects who think they perform poorly.

For optimal response selection, the consequences associated with behavioral success or failure must be appraised. To determine how monetary consequences influence the neural representations of motor preparation, human brain activity was scanned with fMRI while subjects performed a complex spatial visuomotor task. At the beginning of each trial, reward context cues indicated the potential gain and loss imposed for correct or incorrect trial completion. FMRI-activity in canonical reward structures reflected the expected value related to the context. In contrast, motor preparatory activity in posterior parietal and premotor cortex peaked in high “absolute value” (high gain or loss) conditions: being highest for large gains in subjects who believed they performed well while being highest for large losses in those who believed they performed poorly. These results suggest that the neural activity preceding goal-directed actions incorporates the absolute value of that action, predicated upon subjective, rather than objective, estimates of one's performance.

The expected outcome of voluntary actions profoundly shapes human decision making. For instance, expected monetary reward and punishment are powerful modulators of human behavior. Yet how these factors influence brain activity responsible for the preparation of such behavior is not fully understood. This is especially true for demanding tasks, in which the outcome—e.g. reward versus punishment—critically depends on the accuracy of actions. In our human fMRI study we investigated brain activity in specific cortical areas that are related to the planning of voluntary behavior. We show that planning activity in these areas is strongly influenced by the expected monetary gain or loss that subjects associated with their performance in a demanding motor task. Planning activity was highest for large expected gains in subjects who believed that they performed well; conversely, activity was highest for large expected losses in subjects who thought that they performed poorly. This pattern of planning activity was best described by a model which we refer to as the “subjective absolute value model.” We suggest that absolute value signals in motor planning areas can be used to mobilize motor resources in behaviorally relevant situations—both to maximize gains and to avoid losses.

Rewarding feedback after correct visual discriminations has both general and specific influences on visual cortex

Reward can influence visual performance, but the neural basis of this effect remains poorly understood. Here we used functional magnetic resonance imaging to investigate how rewarding feedback affected activity in distinct areas of human visual cortex, separating rewarding feedback events after correct performance from preceding visual events. Participants discriminated oriented gratings in either hemifield, receiving auditory feedback at trial end that signaled financial reward after correct performance. Greater rewards improved performance for all but the most difficult trials. Rewarding feedback increased blood-oxygen-level-dependent (BOLD) signals in striatum and orbitofrontal cortex. It also increased BOLD signals in visual areas beyond retinotopic cortex, but not in primary visual cortex representing the judged stimuli. These modulations were seen at a time point in which no visual stimuli were presented or expected, demonstrating a novel type of activity change in visual cortex that cannot reflect modulation of response to incoming or anticipated visual stimuli. Rewarded trials led on the next trial to improved performance and enhanced visual activity contralateral to the judged stimulus, for retinotopic representations of the judged visual stimuli in V1. Our findings distinguish general effects in nonretinotopic visual cortex when receiving rewarding feedback after correct performance from consequences of reward for spatially specific responses in V1.

Effects of category-specific costs on neural systems for perceptual decision-making

Perceptual judgments are often biased by prospective losses, leading to changes in decision criteria. Little is known about how and where sensory evidence and cost information interact in the brain to influence perceptual categorization. Here we show that prospective losses systematically bias the perception of noisy face-house images. Asymmetries in category-specific cost were associated with enhanced blood-oxygen-level-dependent signal in a frontoparietal network. We observed selective activation of parahippocampal gyrus for changes in category-specific cost in keeping with the hypothesis that loss functions enact a particular task set that is communicated to visual regions. Across subjects, greater shifts in decision criteria were associated with greater activation of the anterior cingulate cortex (ACC). Our results support a hypothesis that costs bias an intermediate representation between perception and action, expressed via general effects on frontal cortex, and selective effects on extrastriate cortex. These findings indicate that asymmetric costs may affect a neural implementation of perceptual decision making in a similar manner to changes in category expectation, constituting a step toward accounting for how prospective losses are flexibly integrated with sensory evidence in the brain.

Dissociating valuation and saliency signals during decision-making

There is a growing consensus that the brain computes value and saliency-like signals at the time of decision-making. Value signals are essential for making choices. Saliency signals are related to motivation, attention, and arousal. Unfortunately, an unequivocal characterization of the areas involved in these 2 distinct sets of processes is made difficult by the fact that, in most experiments, both types of signals are highly correlated. We dissociated value and saliency signals using a novel human functional magnetic resonance imaging decision-making task. Activity in the medial orbitofrontal, rostral anterior cingulate, and posterior cingulate cortices was modulated by value but not saliency. The opposite was true for dorsal anterior cingulate, supplementary motor area, insula, and the precentral and fusiform gyri. Only the ventral striatum and the cuneus were modulated by both value and saliency.

A mechanistic account of value computation in the human brain

To make decisions based on the value of different options, we often have to combine different sources of probabilistic evidence. For example, when shopping for strawberries on a fruit stand, one uses their color and size to infer-with some uncertainty-which strawberries taste best. Despite much progress in understanding the neural underpinnings of value-based decision making in humans, it remains unclear how the brain represents different sources of probabilistic evidence and how they are used to compute value signals needed to drive the decision. Here, we use a visual probabilistic categorization task to show that regions in ventral temporal cortex encode probabilistic evidence for different decision alternatives, while ventromedial prefrontal cortex integrates information from these regions into a value signal using a difference-based comparator operation.

Population response profiles in early visual cortex are biased in favor of more valuable stimuli

Voluntary and stimulus-driven shifts of attention can modulate the representation of behaviorally relevant stimuli in early areas of visual cortex. In turn, attended items are processed faster and more accurately, facilitating the selection of appropriate behavioral responses. Information processing is also strongly influenced by past experience and recent studies indicate that the learned value of a stimulus can influence relatively late stages of decision making such as the process of selecting a motor response. However, the learned value of a stimulus can also influence the magnitude of cortical responses in early sensory areas such as V1 and S1. These early effects of stimulus value are presumed to improve the quality of sensory representations; however, the nature of these modulations is not clear. They could reflect nonspecific changes in response amplitude associated with changes in general arousal or they could reflect a bias in population responses so that high-value features are represented more robustly. To examine this issue, subjects performed a two-alternative forced choice paradigm with a variable-interval payoff schedule to dynamically manipulate the relative value of two stimuli defined by their orientation (one was rotated clockwise from vertical, the other counterclockwise). Activation levels in visual cortex were monitored using functional MRI and feature-selective voxel tuning functions while subjects performed the behavioral task. The results suggest that value not only modulates the relative amplitude of responses in early areas of human visual cortex, but also sharpens the response profile across the populations of feature-selective neurons that encode the critical stimulus feature (orientation). Moreover, changes in space- or feature-based attention cannot easily explain the results because representations of both the selected and the unselected stimuli underwent a similar feature-selective modulation. This sharpening in the population response profile could theoretically improve the probability of correctly discriminating high-value stimuli from low-value alternatives.

Within- and cross-participant classifiers reveal different neural coding of information

Analyzing distributed patterns of brain activation using multivariate pattern analysis (MVPA) has become a popular approach for using functional magnetic resonance imaging (fMRI) data to predict mental states. While the majority of studies currently build separate classifiers for each participant in the sample, in principle a single classifier can be derived from and tested on data from all participants. These two approaches, within- and cross-participant classification, rely on potentially different sources of variability and thus may provide distinct information about brain function. Here, we used both approaches to identify brain regions that contain information about passively received monetary rewards (i.e., images of currency that influenced participant payment) and social rewards (i.e., images of human faces). Our within-participant analyses implicated regions in the ventral visual processing stream-including fusiform gyrus and primary visual cortex-and ventromedial prefrontal cortex (VMPFC). Two key results indicate these regions may contain statistically discriminable patterns that contain different informational representations. First, cross-participant analyses implicated additional brain regions, including striatum and anterior insula. The cross-participant analyses also revealed systematic changes in predictive power across brain regions, with the pattern of change consistent with the functional properties of regions. Second, individual differences in classifier performance in VMPFC were related to individual differences in preferences between our two reward modalities. We interpret these results as reflecting a distinction between patterns showing participant-specific functional organization and those indicating aspects of brain organization that generalize across individuals.