Value-based modulations in human visual cortex

Feasibility of functional neuroimaging to understand adolescent women's sexual decision making

PURPOSE

For young women, new sexual experiences normatively increase after puberty and coincide with extensive changes to brain regions governing self-regulation of risk behavior. These neurodevelopmental changes could leave some young women vulnerable for negative sexual outcomes, including sexually transmitted infection and unintended pregnancy. We evaluated the feasibility of using functional neuroimaging to understand the sexual decision making of adolescent women.

METHODS

Adolescent women (N = 14; 14-15 years) completed enrollment interviews, a neuroimaging task gauging neural activation to appetitive stimuli, and 30 days of prospective diaries following the scan characterizing daily affect and sexual behaviors. Descriptive and inferential statistics assessed the association between imaging and behavioral data.

RESULTS

Young women were highly compliant with neuroimaging and diary protocol. Neural activity in a cognitive-affective network, including prefrontal and anterior cingulate regions, was significantly greater during low-risk decisions. Compared with other decisions, high-risk sexual decisions elicited greater activity in the anterior cingulate, and low-risk sexual decision elicited greater activity in regions of the visual cortex. Young women's sexual decision ratings were linked to their sexual history characteristics and daily self-reports of sexual emotions and behaviors.

CONCLUSIONS

It is feasible to recruit and retain a cohort of female participants to perform a functional magnetic resonance imaging task focused on making decisions about sex, on the basis of varying levels of hypothetical sexual risk, and to complete longitudinal prospective diaries following this task. Preliminary evidence suggests that risk level differentially impacts brain activity related to sexual decision making in these women, which may be related to past and future sexual behaviors.

Visually cued action timing in the primary visual cortex

Most behaviors are generated in three steps: sensing the external world, processing that information to instruct decision-making, and producing a motor action. Sensory areas, especially primary sensory cortices, have long been held to be involved only in the first step of this sequence. Here, we develop a visually cued interval timing task that requires rats to decide when to perform an action following a brief visual stimulus. Using single-unit recordings and optogenetics in this task, we show that activity generated by the primary visual cortex (V1) embodies the target interval and may instruct the decision to time the action on a trial-by-trial basis. A spiking neuronal model of local recurrent connections in V1 produces neural responses that predict and drive the timing of future actions, rationalizing our observations. Our data demonstrate that the primary visual cortex may contribute to the instruction of visually cued timed actions.

Multiple brain networks contribute to the acquisition of bias in perceptual decision-making

Bias occurs in perceptual decisions when the reward associated with a particular response dominates the sensory evidence in support of a choice. However, it remains unclear how this bias is acquired and once acquired, how it influences perceptual decision processes in the brain. We addressed these questions using model-based neuroimaging in a motion discrimination paradigm where contextual cues suggested which one of two options would receive higher rewards on each trial. We found that participants gradually learned to choose the higher-rewarded option in each context when making a perceptual decision. The amount of bias on each trial was fit well by a reinforcement-learning model that estimated the subjective value of each option within the current context. The brain mechanisms underlying this bias acquisition process were similar to those observed in reward-based decision tasks: prediction errors correlated with the fMRI signals in ventral striatum, dlPFC, and parietal cortex, whereas the amount of acquired bias correlated with activity in ventromedial prefrontal (vmPFC), dorsolateral frontal (dlPFC), and parietal cortices. Moreover, psychophysiological interaction analysis revealed that as bias increased, functional connectivity increased within multiple brain networks (dlPFC-vmPFC-visual, vmPFC-motor, and parietal-anterior-cingulate), suggesting that multiple mechanisms contribute to bias in perceptual decisions through integration of value processing with action, sensory, and control systems. These provide a novel link between the neural mechanisms underlying perceptual and economic decision-making.

Neural mechanisms of high-risk decisions-to-drink in alcohol-dependent women

A hallmark of alcohol dependence (AD) is continually drinking despite the risk of negative consequences. Currently, it is not known if the pattern of disordered activation in AD is more compatible with an over-sensitive reward system, a deficit in control systems or a combination of both to produce the high risk-taking behavior observed in alcohol dependents (ADs). Here, alcohol cues were used in an ecological decisions-to-drink task that involved high- and low-risk scenarios where the chance of serious negative imagined consequences was varied. Non-alcohol cues were included as control stimuli. Functional magnetic resonance imaging (fMRI) was used to measure blood oxygen level-dependent (BOLD) signal change in 15 alcohol-dependent and 16 control women. This design allowed us to address two major questions concerning AD: first, is there a specific pattern of disordered activation that drives the heightened endorsement of high-risk decisions-to-drink in ADs? And, second, is that pattern specific to decisions-to-drink or does it generalize to other appetitive and/or neutral cues? The results showed that, during high-risk decisions-to-drink, alcohol-dependent women activated reward circuits, cognitive control circuits and regions of the default-mode network (DMN), while control women deactivated approach circuits and showed enhanced activation in regions of the DMN. Group differences were found only for decisions-to-drink, suggesting that they are specific to alcohol cues. Simultaneous activation of reward networks, cognitive control networks and the DMN in alcohol-dependent women suggests that over-endorsement of high-risk drinking decisions by alcohol-dependent women may be due to a problem with switching between different neural networks.

Interests shape how adolescents pay attention: the interaction of motivation and top-down attentional processes in biasing sensory activations to anticipated events

The voluntary allocation of attention to environmental inputs is a crucial mechanism of healthy cognitive functioning, and is probably influenced by an observer's level of interest in a stimulus. For example, an individual who is passionate about soccer but bored by botany will obviously be more attentive at a soccer match than an orchid show. The influence of monetary rewards on attention has been examined, but the impact of more common motivating factors (i.e. the level of interest in the materials under observation) remains unclear, especially during development. Here, stimulus sets were designed based on survey measures of the level of interest of adolescent participants in several item classes. High-density electroencephalography was recorded during a cued spatial attention task in which stimuli of high or low interest were presented in separate blocks. The motivational impact on performance of a spatial attention task was assessed, along with event-related potential measures of anticipatory top-down attention. As predicted, performance was improved for the spatial target detection of high interest items. Further, the impact of motivation was observed in parieto-occipital processes associated with anticipatory top-down spatial attention. The anticipatory activity over these regions was also increased for high vs. low interest stimuli, irrespective of the direction of spatial attention. The results also showed stronger anticipatory attentional and motivational modulations over the right vs. left parieto-occipital cortex. These data suggest that motivation enhances top-down attentional processes, and can independently shape activations in sensory regions in anticipation of events. They also suggest that attentional functions across hemispheres may not fully mature until late adolescence.

Reward activates stimulus-specific and task-dependent representations in visual association cortices

Humans reliably learn which actions lead to rewards. One prominent question is how credit is assigned to environmental stimuli that are acted upon. Recent functional magnetic resonance imaging (fMRI) studies have provided evidence that representations of rewarded stimuli are activated upon reward delivery, providing possible eligibility traces for credit assignment. Our study sought evidence of postreward activation in sensory cortices satisfying two conditions of instrumental learning: postreward activity should reflect the stimulus category that preceded reward (stimulus specificity), and should occur only if the stimulus was acted on to obtain reward (task dependency). Our experiment implemented two tasks in the fMRI scanner. The first was a perceptual decision-making task on degraded face and house stimuli. Stimulus specificity was evident as rewards activated the sensory cortices associated with face versus house perception more strongly after face versus house decisions, respectively, particularly in the fusiform face area. Stimulus specificity was further evident in a psychophysiological interaction analysis wherein face-sensitive areas correlated with nucleus accumbens activity after face-decision rewards, whereas house-sensitive areas correlated with nucleus accumbens activity after house-decision rewards. The second task required participants to make an instructed response. The criterion of task dependency was fulfilled as rewards after face versus house responses activated the respective association cortices to a larger degree when faces and houses were relevant to the performed task. Our study is the first to show that postreward sensory cortex activity meets these two key criteria of credit assignment, and does so independently from bottom-up perceptual processing.

Common and distinct neural correlates of personal and vicarious reward: A quantitative meta-analysis

Individuals experience reward not only when directly receiving positive outcomes (e.g., food or money), but also when observing others receive such outcomes. This latter phenomenon, known as vicarious reward, is a perennial topic of interest among psychologists and economists. More recently, neuroscientists have begun exploring the neuroanatomy underlying vicarious reward. Here we present a quantitative whole-brain meta-analysis of this emerging literature. We identified 25 functional neuroimaging studies that included contrasts between vicarious reward and a neutral control, and subjected these contrasts to an activation likelihood estimate (ALE) meta-analysis. This analysis revealed a consistent pattern of activation across studies, spanning structures typically associated with the computation of value (especially ventromedial prefrontal cortex) and mentalizing (including dorsomedial prefrontal cortex and superior temporal sulcus). We further quantitatively compared this activation pattern to activation foci from a previous meta-analysis of personal reward. Conjunction analyses yielded overlapping VMPFC activity in response to personal and vicarious reward. Contrast analyses identified preferential engagement of the nucleus accumbens in response to personal as compared to vicarious reward, and in mentalizing-related structures in response to vicarious as compared to personal reward. These data shed light on the common and unique components of the reward that individuals experience directly and through their social connections.

The attribution of value-based attentional priority in individuals with depressive symptoms

The capture of attention by stimuli previously associated with reward has been demonstrated across a wide range of studies. Such value-based attentional priority appears to be robust, and cases where reward feedback fails to modulate subsequent attention have not been reported. However, individuals differ in their sensitivity to external rewards, and such sensitivity is abnormally blunted in depression. Here, we show that depressive symptomology is accompanied by insensitivity to value-based attentional bias. We replicate attentional capture by stimuli previously associated with reward in a control sample and show that these same reward-related stimuli do not capture attention in individuals experiencing symptoms of depression. This sharp contrast in performance indicates that value-based attentional biases depend on the normal functioning of the brain's reward system and suggests that a failure to preferentially attend to reward-related information may play a role in the experience of depression.

Value is in the eye of the beholder: early visual cortex codes monetary value of objects during a diverted attention task

A central concern in the study of learning and decision-making is the identification of neural signals associated with the values of choice alternatives. An important factor in understanding the neural correlates of value is the representation of the object itself, separate from the act of choosing. Is it the case that the representation of an object within visual areas will change if it is associated with a particular value? We used fMRI adaptation to measure the neural similarity of a set of novel objects before and after participants learned to associate monetary values with the objects. We used a range of both positive and negative values to allow us to distinguish effects of behavioral salience (i.e., large vs. small values) from effects of valence (i.e., positive vs. negative values). During the scanning session, participants made a perceptual judgment unrelated to value. Crucially, the similarity of the visual features of any pair of objects did not predict the similarity of their value, so we could distinguish adaptation effects due to each dimension of similarity. Within early visual areas, we found that value similarity modulated the neural response to the objects after training. These results show that an abstract dimension, in this case, monetary value, modulates neural response to an object in visual areas of the brain even when attention is diverted.

Value-driven attentional priority signals in human basal ganglia and visual cortex

Goal-directed and stimulus-driven factors determine attentional priority through a well defined dorsal frontal-parietal and ventral temporal-parietal network of brain regions, respectively. Recent evidence demonstrates that reward-related stimuli also have high attentional priority, independent of their physical salience and goal-relevance. The neural mechanisms underlying such value-driven attentional control are unknown. Using human functional magnetic resonance imaging, we demonstrate that the tail of the caudate nucleus and extrastriate visual cortex respond preferentially to task-irrelevant but previously reward-associated objects, providing an attentional priority signal that is sensitive to reward history. The caudate tail has not been implicated in the control of goal-directed or stimulus-driven attention, but is well suited to mediate the value-driven control of attention. Our findings reveal the neural basis of value-based attentional priority.

A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes

Recent multi-voxel pattern classification (MVPC) studies have shown that in early visual cortex patterns of brain activity generated during mental imagery are similar to patterns of activity generated during perception. This finding implies that low-level visual features (e.g., space, spatial frequency, and orientation) are encoded during mental imagery. However, the specific hypothesis that low-level visual features are encoded during mental imagery is difficult to directly test using MVPC. The difficulty is especially acute when considering the representation of complex, multi-object scenes that can evoke multiple sources of variation that are distinct from low-level visual features. Therefore, we used a voxel-wise modeling and decoding approach to directly test the hypothesis that low-level visual features are encoded in activity generated during mental imagery of complex scenes. Using fMRI measurements of cortical activity evoked by viewing photographs, we constructed voxel-wise encoding models of tuning to low-level visual features. We also measured activity as subjects imagined previously memorized works of art. We then used the encoding models to determine if putative low-level visual features encoded in this activity could pick out the imagined artwork from among thousands of other randomly selected images. We show that mental images can be accurately identified in this way; moreover, mental image identification accuracy depends upon the degree of tuning to low-level visual features in the voxels selected for decoding. These results directly confirm the hypothesis that low-level visual features are encoded during mental imagery of complex scenes. Our work also points to novel forms of brain-machine interaction: we provide a proof-of-concept demonstration of an internet image search guided by mental imagery.

Cross-modal effects of value on perceptual acuity and stimulus encoding

Cross-modal interactions are very common in perception. An important feature of many perceptual stimuli is their reward-predicting properties, the utilization of which is essential for adaptive behavior. What is unknown is whether reward associations in one sensory modality influence perception of stimuli in another modality. Here we show that auditory stimuli with high-reward associations increase the sensitivity of visual perception, even when sounds and reward associations are both irrelevant for the visual task. This increased sensitivity correlates with a change in stimulus representation in the visual cortex, indexed by increased multivariate decoding accuracy in simultaneously acquired functional MRI data. Univariate analysis showed that reward associations modulated responses in regions associated with multisensory processing in which the strength of modulation was a better predictor of the magnitude of the behavioral effect than the modulation in classical reward regions. Our findings demonstrate a value-driven cross-modal interaction that affects perception and stimulus encoding, with a resemblance to well-described modulatory effects of attention. We suggest that multisensory processing areas may mediate the transfer of value signals across senses.

Neural associations of the early retinotopic cortex with the lateral occipital complex during visual perception

Previous studies have demonstrated that the early retinotopic cortex (ERC, i.e., V1/V2/V3) is highly associated with the lateral occipital complex (LOC) during visual perception. However, it remains largely unclear how to evaluate their associations in quantitative way. The present study tried to apply a multivariate pattern analysis (MVPA) to quantify the neural activity in ERC and its association with that of the LOC when participants saw visual images. To this end, we assessed whether low-level visual features (Gabor features) could predict the neural activity in the ERC and LOC according to a voxel-based encoding model (VBEM), and then quantified the association of the neural activity between these regions by using an analogical VBEM. We found that the Gabor features remarkably predicted the activity of the ERC (e.g., the predicted accuracy was 52.5% for a participant) instead of that of the LOC (4.2%). Moreover, the MVPA approach can also be used to establish corresponding relationships between the activity patterns in the LOC and those in the ERC (64.2%). In particular, we found that the integration of the Gabor features and LOC visual information could dramatically improve the ‘prediction’ of ERC activity (88.3%). Overall, the present study provides new evidences for the possibility of quantifying the association of the neural activity between the regions of ERC and LOC. This approach will help to provide further insights into the neural substrates of the visual processing.

Higher brain functions served by the lowly rodent primary visual cortex

It has been more than 50 years since the first description of ocular dominance plasticity--the profound modification of primary visual cortex (V1) following temporary monocular deprivation. This discovery immediately attracted the intense interest of neurobiologists focused on the general question of how experience and deprivation modify the brain as a potential substrate for learning and memory. The pace of discovery has quickened considerably in recent years as mice have become the preferred species to study visual cortical plasticity, and new studies have overturned the dogma that primary sensory cortex is immutable after a developmental critical period. Recent work has shown that, in addition to ocular dominance plasticity, adult visual cortex exhibits several forms of response modification previously considered the exclusive province of higher cortical areas. These "higher brain functions" include neural reports of stimulus familiarity, reward-timing prediction, and spatiotemporal sequence learning. Primary visual cortex can no longer be viewed as a simple visual feature detector with static properties determined during early development. Rodent V1 is a rich and dynamic cortical area in which functions normally associated only with "higher" brain regions can be studied at the mechanistic level.

Value-driven attentional capture in adolescence

Adolescence has been characterized as a period of both opportunity and vulnerability. Numerous clinical conditions, including substance-use disorders, often emerge during adolescence. These maladaptive behaviors have been linked to problems with cognitive control, yet few studies have investigated how rewards differentially modulate attentional processes in adolescents versus adults. Here, we trained adults and adolescents on a visual task to establish stimulus-reward associations. Later, we assessed learning in an extinction task in which previously rewarded stimuli periodically appeared as distractors. Both age groups initially demonstrated value-driven attentional capture; however, the effect persisted longer in adolescents than in adults. The results could not be explained by developmental differences in visual working memory. Given the importance of attentional control to daily behaviors and clinical conditions such as attention-deficit/hyperactivity disorder, these results reveal that cognitive control failures in adolescence may be linked to a value-based attentional-capture effect.

Dynamic integration of information about salience and value for smooth pursuit eye movements

Eye movement behavior can be determined by bottom-up factors like visual salience and by top-down factors like expected value. These different types of signals have to be combined for the control of eye movements. In this study we investigated how smooth pursuit eye movements integrate salience and value information. Observers were asked to track a random-dot kinematogram containing two coherent motion directions. To manipulate salience, the coherence or the density of one of the motion signals was varied. To manipulate value, observers won or lost money in a separate experiment if they were tracking one or the other motion direction. Our results show that pursuit direction was initially determined only by salience. 300-400 ms after target motion onset, pursuit steered towards the rewarded direction and the salience effects disappeared. The time course of this effect depended crucially on the difficulty to segment the two signal directions. These results indicate that salience determines early pursuit responses in the same way as saccades with short latencies. Value information is processed slower and dominates pursuit after several 100 ms.

Cingulate neglect in humans: disruption of contralesional reward learning in right brain damage

Motivational valence plays a key role in orienting spatial attention. Nonetheless, clinical documentation and understanding of motivationally based deficits of spatial orienting in the human is limited. Here in a series of one group-study and two single-case studies, we have examined right brain damaged patients (RBD) with and without left spatial neglect in a spatial reward-learning task, in which the motivational valence of the left contralesional and the right ipsilesional space was contrasted. In each trial two visual boxes were presented, one to the left and one to the right of central fixation. In one session monetary rewards were released more frequently in the box on the left side (75% of trials) whereas in another session they were released more frequently on the right side. In each trial patients were required to: 1) point to each one of the two boxes; 2) choose one of the boxes for obtaining monetary reward; 3) report explicitly the position of reward and whether this position matched or not the original choice. Despite defective spontaneous allocation of attention toward the contralesional space, RBD patients with left spatial neglect showed preserved contralesional reward learning, i.e., comparable to ipsilesional learning and to reward learning displayed by patients without neglect. A notable exception in the group of neglect patients was L.R., who showed no sign of contralesional reward learning in a series of 120 consecutive trials despite being able of reaching learning criterion in only 20 trials in the ipsilesional space. L.R. suffered a cortical-subcortical brain damage affecting the anterior components of the parietal-frontal attentional network and, compared with all other neglect and non-neglect patients, had additional lesion involvement of the medial anterior cingulate cortex (ACC) and of the adjacent sectors of the corpus callosum. In contrast to his lateralized motivational learning deficit, L.R. had no lateral bias in the early phases of attentional processing as he suffered no contralesional visual or auditory extinction on double simultaneous tachistoscopic and dichotic stimulation and detected, with no exception, single contralesional visual and auditory stimuli. In a separate study, we were able to compare L.R. with another RBD patient, G.P., who had a selective lesion in the right ACC, in the adjacent callosal connections and the medial-basal prefrontal cortex. G.P. had no contralesional neglect and displayed normal reward learning both in the left and right side of space. These findings show that contralesional reward learning is generally preserved in RBD patients with left spatial neglect and that this can be exploited in rehabilitation protocols. Contralesional reward learning is severely disrupted in neglect patients when an additional lesion of the ACC is present: however, as demonstrated by the comparison between L.R. and G.P. cases, selective unilateral lesion of the right ACC does not produce motivational neglect for the contralesional space.

Altering spatial priority maps via reward-based learning

Spatial priority maps are real-time representations of the behavioral salience of locations in the visual field, resulting from the combined influence of stimulus driven activity and top-down signals related to the current goals of the individual. They arbitrate which of a number of (potential) targets in the visual scene will win the competition for attentional resources. As a result, deployment of visual attention to a specific spatial location is determined by the current peak of activation (corresponding to the highest behavioral salience) across the map. Here we report a behavioral study performed on healthy human volunteers, where we demonstrate that spatial priority maps can be shaped via reward-based learning, reflecting long-lasting alterations (biases) in the behavioral salience of specific spatial locations. These biases exert an especially strong influence on performance under conditions where multiple potential targets compete for selection, conferring competitive advantage to targets presented in spatial locations associated with greater reward during learning relative to targets presented in locations associated with lesser reward. Such acquired biases of spatial attention are persistent, are nonstrategic in nature, and generalize across stimuli and task contexts. These results suggest that reward-based attentional learning can induce plastic changes in spatial priority maps, endowing these representations with the "intelligent" capacity to learn from experience.

Reward-priming of location in visual search

Existing visual search research has demonstrated that the receipt of reward will be beneficial for subsequent perceptual and attentional processing of features that have characterized targets, but detrimental for processing of features that have characterized irrelevant distractors. Here we report a similar effect of reward on location. Observers completed a visual search task in which they selected a target, ignored a salient distractor, and received random-magnitude reward for correct performance. Results show that when target selection garnered rewarding outcome attention is subsequently a.) primed to return to the target location, and b.) biased away from the location that was occupied by the salient, task-irrelevant distractor. These results suggest that in addition to priming features, reward acts to guide visual search by priming contextual locations of visual stimuli.

Dissociable contributions of ventromedial prefrontal and posterior parietal cortex to value-guided choice

Two long-standing traditions have highlighted cortical decision mechanisms in the parietal and prefrontal cortices of primates, but it has not been clear how these processes differ, or when each cortical region may influence behaviour. Recent data from ventromedial prefrontal cortex (vmPFC) and posterior parietal cortex (PPC) have suggested one possible axis on which the two decision processes might be delineated. Fast decisions may be resolved primarily by parietal mechanisms, whereas decisions made without time pressure may rely on prefrontal mechanisms. Here, we report direct evidence for such dissociation. During decisions under time pressure, a value comparison process was evident in PPC, but not in vmPFC. Value-related activity was still found in vmPFC under time pressure. However, vmPFC represented overall input value rather than compared output value. In contrast, when decisions were made without time pressure, vmPFC transitioned to encode a value comparison while value-related parameters were entirely absent from PPC. Furthermore, under time pressure, decision performance was primarily governed by PPC, while it was dominated by vmPFC at longer decision times. These data demonstrate that parallel cortical mechanisms may resolve the same choices in differing circumstances, and offer an explanation of the diverse neural signals reported in vmPFC and PPC during value-guided choice.

•

Value parameter represented in ventromedial PFC depends on available decision time.

•

Under time pressure, vmPFC activity represents overall input value.

•

Without time pressure, vmPFC transitions to encode a value comparison.

The role of prefrontal catecholamines in attention and working memory

While much progress has been made in identifying the brain regions and neurochemical systems involved in the cognitive processes disrupted in mental illnesses, to date, the level of detail at which neurobiologists can describe the chain of events giving rise to cognitive functions is very rudimentary. Much of the intense interest in understanding cognitive functions is motivated by the hope that it might be possible to understand these complex functions at the level of neurons and neural circuits. Here, we review the current state of the literature regarding how modulations in catecholamine levels within the prefrontal cortex (PFC) alter the neuronal and behavioral correlates of cognitive functions, particularly attention and working memory.

Mapping the semantic structure of cognitive neuroscience

Cognitive neuroscience, as a discipline, links the biological systems studied by neuroscience to the processing constructs studied by psychology. By mapping these relations throughout the literature of cognitive neuroscience, we visualize the semantic structure of the discipline and point to directions for future research that will advance its integrative goal. For this purpose, network text analyses were applied to an exhaustive corpus of abstracts collected from five major journals over a 30-month period, including every study that used fMRI to investigate psychological processes. From this, we generate network maps that illustrate the relationships among psychological and anatomical terms, along with centrality statistics that guide inferences about network structure. Three terms--prefrontal cortex, amygdala, and anterior cingulate cortex--dominate the network structure with their high frequency in the literature and the density of their connections with other neuroanatomical terms. From network statistics, we identify terms that are understudied compared with their importance in the network (e.g., insula and thalamus), are underspecified in the language of the discipline (e.g., terms associated with executive function), or are imperfectly integrated with other concepts (e.g., subdisciplines like decision neuroscience that are disconnected from the main network). Taking these results as the basis for prescriptive recommendations, we conclude that semantic analyses provide useful guidance for cognitive neuroscience as a discipline, both by illustrating systematic biases in the conduct and presentation of research and by identifying directions that may be most productive for future research.

Reward- and attention-related biasing of sensory selection in visual cortex

Attention to task-relevant features leads to a biasing of sensory selection in extrastriate cortex. Features signaling reward seem to produce a similar bias, but how modulatory effects due to reward and attention relate to each other is largely unexplored. To address this issue, it is critical to separate top-down settings defining reward relevance from those defining attention. To this end, we used a visual search paradigm in which the target's definition (attention to color) was dissociated from reward relevance by delivering monetary reward on search frames where a certain task-irrelevant color was combined with the target-defining color to form the target object. We assessed the state of neural biasing for the attended and reward-relevant color by analyzing the neuromagnetic brain response to asynchronously presented irrelevant distractor probes drawn in the target-defining color, the reward-relevant color, and a completely irrelevant color as a reference. We observed that for the prospect of moderate rewards, the target-defining color but not the reward-relevant color produced a selective enhancement of the neuromagnetic response between 180 and 280 msec in ventral extrastriate visual cortex. Increasing reward prospect caused a delayed attenuation (220-250 msec) of the response to reward probes, which followed a prior (160-180 msec) response enhancement in dorsal ACC. Notably, shorter latency responses in dorsal ACC were associated with stronger attenuation in extrastriate visual cortex. Finally, an analysis of the brain response to the search frames revealed that the presence of the reward-relevant color in search distractors elicited an enhanced response that was abolished after increasing reward size. The present data together indicate that when top-down definitions of reward relevance and attention are separated, the behavioral significance of reward-associated features is still rapidly coded in higher-level cortex areas, thereby commanding effective top-down inhibitory control to counter a selection bias for those features in extrastriate visual cortex.

Reward-based transfer from bottom-up to top-down search tasks

Recent evidence has suggested that reward modulates bottom-up and top-down attentional selection and that this effect persists within the same task even when reward is no longer offered. It remains unclear whether reward effects transfer across tasks, especially those engaging different modes of attention. We directly investigated whether reward-based contingency learned in a bottom-up search task was transferred to a subsequent top-down search task, and probed the nature of the transfer mechanism. Results showed that a reward-related benefit established in a pop-out-search task was transferred to a conjunction-search task, increasing participants' efficiency at searching for targets previously associated with a higher level of reward. Reward history influenced search efficiency by enhancing both target salience and distractor filtering, depending on whether the target and distractors shared a critical feature. These results provide evidence for reward-based transfer between different modes of attention and strongly suggest that an integrated priority map based on reward information guides both top-down and bottom-up attention.

Attentional bias for nondrug reward is magnified in addiction

Attentional biases for drug-related stimuli play a prominent role in addiction, predicting treatment outcomes. Attentional biases also develop for stimuli that have been paired with nondrug rewards in adults without a history of addiction, the magnitude of which is predicted by visual working-memory capacity and impulsiveness. We tested the hypothesis that addiction is associated with an increased attentional bias for nondrug (monetary) reward relative to that of healthy controls, and that this bias is related to working-memory impairments and increased impulsiveness. Seventeen patients receiving methadone-maintenance treatment for opioid dependence and 17 healthy controls participated. Impulsiveness was measured using the Barratt Impulsiveness Scale (BIS-11; Patton, Stanford, & Barratt, 1995), visual working-memory capacity was measured as the ability to recognize briefly presented color stimuli, and attentional bias was measured as the magnitude of response time slowing caused by irrelevant but previously reward-associated distractors in a visual-search task. The results showed that attention was biased toward the distractors across all participants, replicating previous findings. It is important to note, this bias was significantly greater in the patients than in the controls and was negatively correlated with visual working-memory capacity. Patients were also significantly more impulsive than controls as a group. Our findings demonstrate that patients in treatment for addiction experience greater difficulty ignoring stimuli associated with nondrug reward. This nonspecific reward-related bias could mediate the distracting quality of drug-related stimuli previously observed in addiction.

ERP evidence for human early visual sensitivity to co-linearity compared to co-circularity

Our complex visual environment is constrained by natural geometric regularities, including spatiotemporal regularity, co-linearity and co-circularity. To investigate human visual processing associated with these regularities we directly compared the neural processes in encoding dynamic co-linearity and co-circularity using event-related potentials (ERPs). By recording ERPs to a target bar presented alone (no context) or in a dynamic sequence of bars following a co-linear or co-circular path, we observed earlier ERPs to targets embedded in co-linear sequence at early (66ms) and later stages (197ms) of post-target processing. In contrast, targets in co-circular sequence only modulated ERPs at later stages of processing. It is proposed that early visual processing may have adapted to efficiently process co-linearity to improve target identification, whereas sensitivity to co-circularity does not occur until later stages of processing. These results have significant impact for understanding brain-behaviour relationships when processing natural geometric regularities.

Neural dynamics of reward probability coding: a Magnetoencephalographic study in humans

Prediction of future rewards and discrepancy between actual and expected outcomes (prediction error) are crucial signals for adaptive behavior. In humans, a number of fMRI studies demonstrated that reward probability modulates these two signals in a large brain network. Yet, the spatio-temporal dynamics underlying the neural coding of reward probability remains unknown. Here, using magnetoencephalography, we investigated the neural dynamics of prediction and reward prediction error computations while subjects learned to associate cues of slot machines with monetary rewards with different probabilities. We showed that event-related magnetic fields (ERFs) arising from the visual cortex coded the expected reward value 155 ms after the cue, demonstrating that reward value signals emerge early in the visual stream. Moreover, a prediction error was reflected in ERF peaking 300 ms after the rewarded outcome and showing decreasing amplitude with higher reward probability. This prediction error signal was generated in a network including the anterior and posterior cingulate cortex. These findings pinpoint the spatio-temporal characteristics underlying reward probability coding. Together, our results provide insights into the neural dynamics underlying the ability to learn probabilistic stimuli-reward contingencies.

Factors modulating neural reactivity to drug cues in addiction: a survey of human neuroimaging studies

Human neuroimaging studies suggest that neural cue reactivity is strongly associated with indices of drug use, including addiction severity and treatment success. However, little is known about factors that modulate cue reactivity. The goal of this review, in which we survey published fMRI and PET studies on drug cue reactivity in cocaine, alcohol, and tobacco cigarette users, is to highlight major factors that modulate brain reactivity to drug cues. First, we describe cue reactivity paradigms used in neuroimaging research and outline the brain circuits that underlie cue reactivity. We then discuss major factors that have been shown to modulate cue reactivity and review specific evidence as well as outstanding questions related to each factor. Building on previous model-building reviews on the topic, we then outline a simplified model that includes the key modulatory factors and a tentative ranking of their relative impact. We conclude with a discussion of outstanding challenges and future research directions, which can inform future neuroimaging studies as well as the design of treatment and prevention programs.

Set and setting: how behavioral state regulates sensory function and plasticity

Recently developed neuroimaging and electrophysiological techniques are allowing us to answer fundamental questions about how behavioral states regulate our perception of the external environment. Studies using these techniques have yielded surprising insights into how sensory processing is affected at the earliest stages by attention and motivation, and how new sensory information received during wakefulness (e.g., during learning) continues to affect sensory brain circuits (leading to plastic changes) during subsequent sleep. This review aims to describe how brain states affect sensory response properties among neurons in primary and secondary sensory cortices, and how this relates to psychophysical detection thresholds and performance on sensory discrimination tasks. This is not intended to serve as a comprehensive overview of all brain states, or all sensory systems, but instead as an illustrative description of how three specific state variables (attention, motivation, and vigilance [i.e., sleep vs. wakefulness]) affect sensory systems in which they have been best studied.

Electrophysiological evidence for emotional valence and competitive arousal effects on insight problem solving

Accumulating evidence suggests that insight can be substantially influenced by task-irrelevant emotion stimuli and interpersonal competitive situation, and a close link might exist between them. Using a learning-testing paradigm and Event-Related Potentials (ERPs), the present study investigated the independent and joint effects of emotional and competitive information on insight problem solving especially their neural mechanisms. Subjects situated in either competitive or non-competitive condition learned heuristic logogriphs first and then viewed task-irrelevant positive or negative emotional pictures, which were followed by test logogriphs to solve. Both behavioral and ERP findings showed a more evident insight boost following negative emotional pictures in competitive context. Results demonstrated that negative emotion and competitive situation might promote insight by a defocused mode of attention (as indicated by N1 and P2), the enhanced semantic integration and breaking mental set (as indicated by N450), and the increased forming of novel associations activated by motivational arousal originating from competition (as indicated by P800-1600 and P1600-2500). These results indicate that the dynamic interactions between emotional valence and competitive arousal effects on insight.

Value-driven attentional and oculomotor capture during goal-directed, unconstrained viewing

Covert shifts of attention precede and direct overt eye movements to stimuli that are task relevant or physically salient. A growing body of evidence suggests that the learned value of perceptual stimuli strongly influences their attentional priority. For example, previously rewarded but otherwise irrelevant and inconspicuous stimuli capture covert attention involuntarily. It is unknown, however, whether stimuli also draw eye movements involuntarily as a consequence of their reward history. Here, we show that previously rewarded but currently task-irrelevant stimuli capture both attention and the eyes. Value-driven oculomotor capture was observed during unconstrained viewing, when neither eye movements nor fixations were required, and was strongly related to individual differences in visual working memory capacity. The appearance of a reward-associated stimulus came to evoke pupil dilation over the course of training, which provides physiological evidence that the stimuli that elicit value-driven capture come to serve as reward-predictive cues. These findings reveal a close coupling of value-driven attentional capture and eye movements that has broad implications for theories of attention and reward learning.

The differential effects of reward on space- and object-based attentional allocation

Estimating reward contingencies and allocating attentional resources to a subset of relevant information are the most important contributors to increasing adaptability of an organism. Although recent evidence suggests that reward- and attention-based guidance recruits overlapping cortical regions and has similar effects on sensory responses, the exact nature of the relationship between the two remains elusive. Here, using event-related fMRI on human participants, we contrasted the effects of reward on space- and object-based selection in the same experimental setting. Reward was either distributed randomly or biased a particular object. Behavioral and neuroimaging results show that space- and object-based attention is influenced by reward differentially. Space-based attentional allocation is mandatory, integrating reward information over time, whereas object-based attentional allocation is a default setting that is completely replaced by the reward signal. Nonadditivity of the effects of reward and object-based attention was observed consistently at multiple levels of analysis in early visual areas as well as in control regions. These results provide strong evidence that space- and object-based allocation are two independent attentional mechanisms, and suggest that reward serves to constrain attentional selection.

Oscillatory coupling reveals the dynamic reorganization of large-scale neural networks as cognitive demands change

Cognitive operations are thought to emerge from dynamic interactions between spatially distinct brain areas. Synchronization of oscillations has been proposed to regulate these interactions, but we do not know whether this large-scale synchronization can respond rapidly to changing cognitive demands. Here we show that, as task demands change during a trial, multiple distinct networks are dynamically formed and reformed via oscillatory synchronization. Distinct frequency-coupled networks were rapidly formed to process reward value, maintain information in visual working memory, and deploy visual attention. Strong single-trial correlations showed that networks formed even before the presentation of imperative stimuli could predict the strength of subsequent networks, as well as the speed and accuracy of behavioral responses seconds later. These frequency-coupled networks better predicted single-trial behavior than either local oscillations or ERPs. Our findings demonstrate the rapid reorganization of networks formed by dynamic activity in response to changing task demands within a trial.

Informatic parcellation of the network involved in the computation of subjective value

Understanding how the brain computes value is a basic question in neuroscience. Although individual studies have driven this progress, meta-analyses provide an opportunity to test hypotheses that require large collections of data. We carry out a meta-analysis of a large set of functional magnetic resonance imaging studies of value computation to address several key questions. First, what is the full set of brain areas that reliably correlate with stimulus values when they need to be computed? Second, is this set of areas organized into dissociable functional networks? Third, is a distinct network of regions involved in the computation of stimulus values at decision and outcome? Finally, are different brain areas involved in the computation of stimulus values for different reward modalities? Our results demonstrate the centrality of ventromedial prefrontal cortex (VMPFC), ventral striatum and posterior cingulate cortex (PCC) in the computation of value across tasks, reward modalities and stages of the decision-making process. We also find evidence of distinct subnetworks of co-activation within VMPFC, one involving central VMPFC and dorsal PCC and another involving more anterior VMPFC, left angular gyrus and ventral PCC. Finally, we identify a posterior-to-anterior gradient of value representations corresponding to concrete-to-abstract rewards.

Reward predictions bias attentional selection

Attention selects stimuli for perceptual and cognitive processing according to an adaptive selection schedule. It has long been known that attention selects stimuli that are task relevant or perceptually salient. Recent evidence has shown that stimuli previously associated with reward persistently capture attention involuntarily, even when they are no longer associated with reward. Here we examine whether the capture of attention by previously reward-associated stimuli is modulated by the processing of current but unrelated rewards. Participants learned to associate two color stimuli with different amounts of reward during a training phase. In a subsequent test phase, these previously rewarded color stimuli were occasionally presented as to-be-ignored distractors while participants performed visual search for each of two differentially rewarded shape-defined targets. The results reveal that attentional capture by formerly rewarded distractors was the largest when both recently received and currently expected reward were the highest in the test phase, even though such rewards were unrelated to the color distractors. Our findings support a model in which value-driven attentional biases acquired through reward learning are maintained via the cognitive mechanisms involved in predicting future rewards.

What is Bottom-Up and What is Top-Down in Predictive Coding?

Everyone knows what bottom-up is, and how it is different from top-down. At least one is tempted to think so, given that both terms are ubiquitously used, but only rarely defined in the psychology and neuroscience literature. In this review, we highlight the problems and limitations of our current understanding of bottom-up and top-down processes, and we propose a reformulation of this distinction in terms of predictive coding.

A unified selection signal for attention and reward in primary visual cortex

Stimuli associated with high rewards evoke stronger neuronal activity than stimuli associated with lower rewards in many brain regions. It is not well understood how these reward effects influence activity in sensory cortices that represent low-level stimulus features. Here, we investigated the effects of reward information in the primary visual cortex (area V1) of monkeys. We found that the reward value of a stimulus relative to the value of other stimuli is a good predictor of V1 activity. Relative value biases the competition between stimuli, just as has been shown for selective attention. The neuronal latency of this reward value effect in V1 was similar to the latency of attentional influences. Moreover, V1 neurons with a strong value effect also exhibited a strong attention effect, which implies that relative value and top-down attention engage overlapping, if not identical, neuronal selection mechanisms. Our findings demonstrate that the effects of reward value reach down to the earliest sensory processing levels of the cerebral cortex and imply that theories about the effects of reward coding and top-down attention on visual representations should be unified.

Dopaminergic reward signals selectively decrease fMRI activity in primate visual cortex

Stimulus-reward coupling without attention can induce highly specific perceptual learning effects, suggesting that reward triggers selective plasticity within visual cortex. Additionally, dopamine-releasing events-temporally surrounding stimulus-reward associations-selectively enhance memory. These forms of plasticity may be evoked by selective modulation of stimulus representations during dopamine-inducing events. However, it remains to be shown whether dopaminergic signals can selectively modulate visual cortical activity. We measured fMRI activity in monkey visual cortex during reward-only trials apart from intermixed cue-reward trials. Reward without visual stimulation selectively decreased fMRI activity within the cue representations that had been paired with reward during other trials. Behavioral tests indicated that these same uncued reward trials strengthened cue-reward associations. Furthermore, such spatially-specific activity modulations depended on prediction error, as shown by manipulations of reward magnitude, cue-reward probability, cue-reward familiarity, and dopamine signaling. This cue-selective negative reward signal offers a mechanism for selectively gating sensory cortical plasticity.

Temporal characteristics of the influence of punishment on perceptual decision making in the human brain

Perceptual decision making is the process by which information from sensory systems is combined and used to influence our behavior. In addition to the sensory input, this process can be affected by other factors, such as reward and punishment for correct and incorrect responses. To investigate the temporal dynamics of how monetary punishment influences perceptual decision making in humans, we collected electroencephalography (EEG) data during a perceptual categorization task whereby the punishment level for incorrect responses was parametrically manipulated across blocks of trials. Behaviorally, we observed improved accuracy for high relative to low punishment levels. Using multivariate linear discriminant analysis of the EEG, we identified multiple punishment-induced discriminating components with spatially distinct scalp topographies. Compared with components related to sensory evidence, components discriminating punishment levels appeared later in the trial, suggesting that punishment affects primarily late postsensory, decision-related processing. Crucially, the amplitude of these punishment components across participants was predictive of the size of the behavioral improvements induced by punishment. Finally, trial-by-trial changes in prestimulus oscillatory activity in the alpha and gamma bands were good predictors of the amplitude of these components. We discuss these findings in the context of increased motivation/attention, resulting from increases in punishment, which in turn yields improved decision-related processing.

A value-driven mechanism of attentional selection

Attention selects stimuli for cognitive processing, and the mechanisms that underlie the process of attentional selection have been a major topic of psychological research for over 30 years. From this research, it has been well documented that attentional selection can proceed both voluntarily, driven by visual search goals, and involuntarily, driven by the physical salience of stimuli. In this review, I provide a conceptual framework for attentional control that emphasizes the need for stimulus selection to promote the survival and wellbeing of an organism. I argue that although goal-driven and salience-driven mechanisms of attentional selection fit within this framework, a central component that is missing is a mechanism of attentional selection that is uniquely driven by learned associations between stimuli and rewards. I go on to review recent evidence for such a value-driven mechanism of attentional selection, and describe how this mechanism functions independently of the well-documented salience-driven and goal-driven mechanisms. I conclude by arguing that reward learning modifies the attentional priority of stimuli, allowing them to compete more effectively for selection even when nonsalient and task-irrelevant.