Fatal attraction: ventral striatum predicts costly choice errors in humans

Sign-tracking modulates reward-related neural activation to reward cues, but not reward feedback

Research shows cognitive and neurobiological overlap between sign-tracking [value-modulated attentional capture (VMAC) by response-irrelevant, discrete cues] and maladaptive behaviour (e.g. substance abuse). We investigated the neural correlates of sign-tracking in 20 adults using an additional singleton task (AST) and functional magnetic resonance imaging (fMRI). Participants responded to a target to win monetary reward, the amount of which was signalled by singleton type (reward cue: high value vs. low value). Singleton responses resulted in monetary deductions. Sign-tracking-greater distraction by high-value vs. low-value singletons (H > L)-was observed, with high-value singletons producing slower responses to the target than low-value singletons. Controlling for age and sex, analyses revealed no differential brain activity across H > L singletons. Including sign-tracking as a regressor of interest revealed increased activity (H > L singletons) in cortico-subcortical loops, regions associated with Pavlovian conditioning, reward processing, attention shifts and relative value coding. Further analyses investigated responses to reward feedback (H > L). Controlling for age and sex, increased activity (H > L reward feedback) was found in regions associated with reward anticipation, attentional control, success monitoring and emotion regulation. Including sign-tracking as a regressor of interest revealed increased activity in the temporal pole, a region related to value discrimination. Results suggest sign-tracking is associated with activation of the 'attention and salience network' in response to reward cues but not reward feedback, suggesting parcellation between the two at the level of the brain. Results add to the literature showing considerable overlap in neural systems implicated in reward processing, learning, habit formation, emotion regulation and substance craving.

Congruence of Inherent and Acquired Values Facilitates Reward-Based Decision-Making

Most real-life cues exhibit certain inherent values that may interfere with or facilitate the acquisition of new expected values during associative learning. In particular, when inherent and acquired values are congruent, learning may progress more rapidly. Here we investigated such an influence through a 2 × 2 factorial design, using attractiveness (high/low) of the facial picture as a proxy for the inherent value of the cue and its reward probability (high/low) as a surrogate for the acquired value. Each picture was paired with a monetary win or loss either congruently or incongruently. Behavioral results from 32 human participants indicated both faster response time and faster learning rate for value-congruent cue-outcome pairings. Model-based fMRI analysis revealed a fractionation of reinforcement learning (RL) signals in the ventral striatum, including a strong and novel correlation between the cue-specific decaying learning rate and BOLD activity in the ventral caudate. Additionally, we detected a functional link between neural signals of both learning rate and reward prediction error in the ventral striatum, and the signal of expected value in the ventromedial prefrontal cortex, showing a novel confirmation of the mathematical RL model via functional connectivity.

SIGNIFICANCE STATEMENT

Most real-world decisions require the integration of inherent value and sensitivity to outcomes to facilitate adaptive learning. Inherent value is drawing increasing interest from decision scientists because it influences decisions in contexts ranging from advertising to investing. This study provides novel insight into how inherent value influences the acquisition of new expected value during associative learning. Specifically, we find that the congruence between the inherent value and the acquired reward influences the neural coding of learning rate. We also show for the first time that neuroimaging signals coding the learning rate, prediction error, and acquired value follow the multiplicative Rescorla-Wagner learning rule, a finding predicted by reinforcement learning theory.

To work or not to work: Neural representation of cost and benefit of instrumental action

By definition, instrumental actions are performed in order to obtain certain goals. Nevertheless, the attainment of goals typically implies obstacles, and response vigor is known to reflect an integration of subjective benefit and cost. Whereas several brain regions have been associated with cost/benefit ratio decision-making, trial-by-trial fluctuations in motivation are not well understood. We review recent evidence supporting the motivational implications of signal fluctuations in the mesocorticolimbic system. As an extension of "set-point" theories of instrumental action, we propose that response vigor is determined by a rapid integration of brain signals that reflect value and cost on a trial-by-trial basis giving rise to an online estimate of utility. Critically, we posit that fluctuations in key nodes of the network can predict deviations in response vigor and that variability in instrumental behavior can be accounted for by models devised from optimal control theory, which incorporate the effortful control of noise. Notwithstanding, the post hoc analysis of signaling dynamics has caveats that can effectively be addressed in future research with the help of two novel fMRI imaging techniques. First, adaptive fMRI paradigms can be used to establish a time-order relationship, which is a prerequisite for causality, by using observed signal fluctuations as triggers for stimulus presentation. Second, real-time fMRI neurofeedback can be employed to induce predefined brain states that may facilitate benefit or cost aspects of instrumental actions. Ultimately, understanding temporal dynamics in brain networks subserving response vigor holds the promise for targeted interventions that could help to readjust the motivational balance of behavior.

Using fMRI to study reward processing in humans: past, present, and future

Functional magnetic resonance imaging (fMRI) is a noninvasive tool used to probe cognitive and affective processes. Although fMRI provides indirect measures of neural activity, the advent of fMRI has allowed for1) the corroboration of significant animal findings in the human brain, and2) the expansion of models to include more common human attributes that inform behavior. In this review, we briefly consider the neural basis of the blood oxygenation level dependent signal to set up a discussion of how fMRI studies have applied it in examining cognitive models in humans and the promise of using fMRI to advance such models. Specifically, we illustrate the contribution that fMRI has made to the study of reward processing, focusing on the role of the striatum in encoding reward-related learning signals that drive anticipatory and consummatory behaviors. For instance, we discuss how fMRI can be used to link neural signals (e.g., striatal responses to rewards) to individual differences in behavior and traits. While this functional segregation approach has been constructive to our understanding of reward-related functions, many fMRI studies have also benefitted from a functional integration approach that takes into account how interconnected regions (e.g., corticostriatal circuits) contribute to reward processing. We contend that future work using fMRI will profit from using a multimodal approach, such as combining fMRI with noninvasive brain stimulation tools (e.g., transcranial electrical stimulation), that can identify causal mechanisms underlying reward processing. Consequently, advancements in implementing fMRI will promise new translational opportunities to inform our understanding of psychopathologies.

Dopamine ups and downs in vulnerability to addictions: a neurodevelopmental model

Addictions are commonly presaged by problems in childhood and adolescence. For many individuals this starts with the early expression of impulsive risk-taking, social gregariousness, and oppositional behaviors. Here we propose that these early diverse manifestations reflect a heightened ability of emotionally salient stimuli to activate dopamine pathways that foster behavioral approach. If substance use is initiated, these at-risk youth can also develop heightened responses to drug-paired cues. Through conditioning and drug-induced sensitization, these effects strengthen and accumulate, leading to responses that exceed those elicited by other rewards. At the same time, cues not paired with drug become associated with comparatively lower dopamine release, accentuating further the difference between drug and non-drug rewards. Together, these enhancing and inhibiting processes steer a pre-existing vulnerability toward a disproportionate concern for drugs and drug-related stimuli. Implications for prevention and treatment are discussed.