Goals and task difficulty expectations modulate striatal responses to feedback

Dorsal posterior cingulate cortex responds to negative feedback information supporting learning and relearning of response policies

Many challenges in life come without explicit instructions. Instead, humans need to test, select, and adapt their behavioral responses based on feedback from the environment. While reward-centric accounts of feedback processing primarily stress the reinforcing aspect of positive feedback, feedback’s central function from an information-processing perspective is to offer an opportunity to correct errors, thus putting a greater emphasis on the informational content of negative feedback. Independent of its potential rewarding value, the informational value of performance feedback has recently been suggested to be neurophysiologically encoded in the dorsal portion of the posterior cingulate cortex (dPCC). To further test this association, we investigated multidimensional categorization and reversal learning by comparing negative and positive feedback in an event-related functional magnetic resonance imaging experiment. Negative feedback, compared with positive feedback, increased activation in the dPCC as well as in brain regions typically involved in error processing. Only in the dPCC, subarea d23, this effect was significantly enhanced in relearning, where negative feedback signaled the need to shift away from a previously established response policy. Together with previous findings, this result contributes to a more fine-grained functional parcellation of PCC subregions and supports the dPCC’s involvement in the adaptation to behaviorally relevant information from the environment.

The involvement of the posterior parietal cortex in promotion and prevention focus

Motivation plays a critical role in human cognitive function, while acting as a driving force for the necessary behavior to achieve a desired goal and success (i.e., achievement motivation). Based on the theoretical background of achievement motivation, this study designed an incentive delay task with four motivational orientations (i.e., promotion, prevention, mastery/self, and performance/other). To investigate whether people would have their behavioral patterns toward achievement motivation orientation, we applied an unsupervised clustering algorithm to classify individuals’ behavioral responses acquired from the task by categorizing certain behavioral similarities. As a result, this hierarchical clustering approach classified subjects into two distinctive subgroups: Group#1 (i.e., the pro/pre group, n = 52) and Group#2 (i.e., the self/other group, n = 48). Based on clustering, Group#1 showed significantly better performance with promotion/prevention orientations, whereas Group#2 exhibited significantly higher performance with self/other orientations. Structural brain analyses discovered increased gray matter volume and sulcal depth in the posterior parietal cortex (PPC) in the pro/pre group compared to the self/other group. With resting-state functional magnetic resonance imaging data, we found higher local brain fluctuations in the medial prefrontal cortex (mPFC) in the self/other group compared to the pro/pre group. Furthermore, mPFC seed-based functional connectivity showed significantly increased functional coupling with the posterior cingulate cortex in the self/other group relative to the pro/pre group. Taken together, these results shed light on structural and functional neural mechanisms related to achievement motivation and, furthermore, provide novel insights regarding PPC’s role in motivational processing toward promotion- and prevention-focused orientation.

Frontopolar Cortex Response to Positive Feedback Relates to Nonincentivized Task Persistence

When individuals make decisions whether to persist at a task, their decision-making is informed by whether success is pending or accomplished. If pending, the brain facilitates behavioral persistence; if the goal is accomplished or no longer desired, the brain enables switching away from the current task. Feedback, which is known to differentially engage reward neurocircuitry, may modulate goal-directed behavior such as task persistence. However, prior studies are confounded by offering external incentives for persistence. This study tested whether neural response to feedback differed as a function of nonincentivized task persistence in 99 human participants ages 13-30 (60 females). Individuals who persisted engaged the frontopolar cortex (FPC) to a greater extent during receipt of task-relevant positive feedback compared with negative feedback. For individuals who quit, task-irrelevant monetary reward engaged the FPC to a greater extent compared with positive feedback. FPC activation in response to positive feedback is identified as a key contributor to task persistence.

Neural correlates of extrinsic and intrinsic outcome processing during learning in individuals with TBI: a pilot investigation

Outcome processing, the ability to learn from feedback, is an important component of adaptive behavior and rehabilitation. Evidence from healthy adults implicates the striatum and dopamine in outcome processing. Animal research shows that damage to dopaminergic pathways in the brain can lead to a disruption of dopamine tone and transmission. Such evidence thus suggests that persons with TBI experience deficits in outcome processing. However, no research has directly investigated outcome processing and associated neural mechanisms in TBI. Here, we examine outcome processing in individuals with TBI during learning. Given that TBI negatively impacts striatal and dopaminergic systems, we hypothesize that individuals with TBI exhibit deficits in learning from outcomes. To test this hypothesis, individuals with moderate-to-severe TBI and healthy adults were presented with a declarative paired-associate word learning task. Outcomes indicating performance accuracy were presented immediately during task performance and in the form of either monetary or performance-based feedback. Two types of feedback provided the opportunity to test whether extrinsic and intrinsic motivational aspects of outcome presentation play a role during learning and outcome processing. Our results show that individuals with TBI exhibited impaired learning from feedback compared to healthy participants. Additionally, individuals with TBI exhibited increased activation in the striatum during outcome processing. The results of this study suggest that outcome processing and learning from immediate outcomes is impaired in individuals with TBI and might be related to inefficient use of neural resources during task performance as reflected by increased activation of the striatum.

Intrinsic prefrontal organization underlies associations between achievement motivation and delay discounting

Achievement motivation is a core component of human decision making. However, neural mechanisms that link achievement motivation and intertemporal choice have not yet been elucidated. Here, we examined neural pathways underlying the relationship between achievement motivation and intertemporal choice using a delay discounting task and resting-state functional magnetic resonance imaging on 86 healthy subjects. Behaviorally, delay discounting rate was positively correlated with achievement motivation. Functional coupling of the dorsolateral prefrontal cortex (dlPFC) with the medial prefrontal cortex (mPFC), medial orbitofrontal cortex and ventral striatum was positively correlated with achievement motivation. Notably, the mediation analysis showed that the impact of achievement motivation on delay discounting was mediated by intrinsic connectivity between the dlPFC and mPFC. Our findings suggest that intrinsic organization within the prefrontal cortex plays a key role in linking achievement motivation and intertemporal choice.

Beyond Reward Prediction Errors: Human Striatum Updates Rule Values During Learning

Humans naturally group the world into coherent categories defined by membership rules. Rules can be learned implicitly by building stimulus-response associations using reinforcement learning or by using explicit reasoning. We tested if the striatum, in which activation reliably scales with reward prediction error, would track prediction errors in a task that required explicit rule generation. Using functional magnetic resonance imaging during a categorization task, we show that striatal responses to feedback scale with a "surprise" signal derived from a Bayesian rule-learning model and are inconsistent with RL prediction error. We also find that striatum and caudal inferior frontal sulcus (cIFS) are involved in updating the likelihood of discriminative rules. We conclude that the striatum, in cooperation with the cIFS, is involved in updating the values assigned to categorization rules when people learn using explicit reasoning.

Effects of Expressive Writing on Neural Processing During Learning

Expressive writing about past negative events has been shown to lead to a slew of positive outcomes. However, little is known about why writing about something negative would have positive effects. While some have posited that telling a narrative of a past negative event or current anxiety "frees up" cognitive resources, allowing individuals to focus more on the task at hand, there is little neural evidence suggesting that expressive writing has an effect on cognitive load. Moreover, little is known about how individual differences in the content of expressive writing could affect neural processing and the cognitive benefits writing confers. In our experiment, we compared brain activity in a group that had engaged in expressive writing vs. a control group, during performance on a feedback-based paired-associate word-learning task. We found that across groups, differential activation in the dorsal striatum in response to positive vs. negative feedback significantly predicted better later memory. Moreover, writing about a past failure resulted in more activation relative to the control group during the learning task in the mid-cingulate cortex (MCC), an area of the brain crucial to processing negative emotion. While our results do not provide support for the assertion that expressive writing alters attentional processing, our findings suggest that choosing to write about particularly intense past negative experiences like a difficult past failure may have resulted in changes in neural activation during task processing.

Intelligence mindset shapes neural learning signals and memory

Intelligence mindset, which denotes individual beliefs about whether intelligence is fixed versus malleable, shapes academic success, but the neural mechanisms underlying mindset-related differences in learning are unknown. Here, we probe the effects of individual differences in mindset on neural responses to negative feedback after a competence threat manipulation. We hypothesized that when their competence was threatened, participants with fixed mindsets would interpret further negative feedback as punishing. After receiving either no score or a competence-threatening IQ score, participants performed a learning task with feedback that emphasized either the evaluative or informational weight of negative feedback. Participants who experienced the competence threat had the strongest predictive relationships between mindset, performance, and caudate activation. The competence threat may have compounded the subjective punishment of negative feedback for fixed mindsets relative to growth mindsets, causing poorer learning from negative feedback in the evaluative context and inflexible striatal responses to negative feedback across feedback contexts.

Success and failure of controlling the real-time functional magnetic resonance imaging neurofeedback signal are reflected in the striatum

INTRODUCTION

Over the last decades, neurofeedback has been applied in variety of research contexts and therapeutic interventions. Despite this extensive use, its neural mechanisms are still under debate. Several scientific advances have suggested that different networks become jointly active during neurofeedback, including regions generally involved in self-regulation, regions related to the specific mental task driving the neurofeedback and regions generally involved in feedback learning (Sitaram et al., 2017, Nature Reviews Neuroscience, 18, 86).

METHODS

To investigate the neural mechanisms specific to neurofeedback but independent from general effects of self-regulation, we compared brain activation as measured with functional magnetic resonance imaging (fMRI) across different mental tasks involving gradual self-regulation with and without providing neurofeedback. Ten participants freely chose one self-regulation task and underwent two training sessions during fMRI scanning, one with and one without receiving neurofeedback. During neurofeedback sessions, feedback signals were provided in real-time based on activity in task-related, individually defined target regions. In both sessions, participants aimed at reaching and holding low, medium, or high brain-activation levels in the target region.

RESULTS

During gradual self-regulation with neurofeedback, a network of cortical control regions as well as regions implicated in reward and feedback processing were activated. Self-regulation with feedback was accompanied by stronger activation within the striatum across different mental tasks. Additional time-resolved single-trial analysis revealed that neurofeedback performance was positively correlated with a delayed brain response in the striatum that reflected the accuracy of self-regulation.

CONCLUSION

Overall, these findings support that neurofeedback contributes to self-regulation through task-general regions involved in feedback and reward processing.

Neurobiological responses in the adolescent striatum to being 'tested'

While emerging research implicates the striatum in adolescents' ability to learn from feedback, little is known about how motivational contexts, such as emphasizing the evaluative nature of learning tasks, modulate adolescents' striatal learning. We used functional magnetic resonance imaging during a feedback-based learning task, in conjunction with a within-subject evaluative threat manipulation, to determine whether evaluation threat influences behavioral and neural responses to feedback in adolescents. On average, adolescents were less sensitive than adults to the evaluation threat. In the adolescents, the effect of evaluation threat on performance was tracked with the striatal response to performance feedback during the evaluation threat condition, such that greater striatal sensitivity correlated with greater gains in learning performance. Our findings suggest that variability in how adolescents respond to a contextual threat of evaluation and associated striatal sensitivity can facilitate enhanced learning.

A neuroscientific perspective on basic psychological needs

OBJECTIVE

Self-determination theory's (SDT) most basic propositions are, first, that all people possess an inherent set of psychological needs and, second, that autonomy, competence, and relatedness are the three critical psychological satisfactions needed to maintain and promote personal growth and well-being. In this article, we identify the neural basis of the psychological needs and, in doing so, seek to advance the integration of SDT and neuroscience.

METHOD

We examine the neural underpinnings of SDT-based motivational states and traits for autonomy, competence, and relatedness. To study motivational states, participants are exposed to situational conditions known to affect their psychological needs, and neuroscience methods (e.g., fMRI) are used to examine changes in their brain activity. To study motivational traits, participants complete self-report trait measures that are then correlated with their brain activity observed during need-satisfying activities.

RESULTS

For both motivational states and traits and across all three needs, intrinsic satisfaction is associated with striatum-based reward processing, anterior insula-based subjective feelings, and the integration of these subjective feelings with reward-based processing.

CONCLUSIONS

Psychological need satisfaction is associated with striatum activity, anterior insula activity, and the functional coactivation between these two brain areas. Given these findings, it is now clear that several opportunities exist to integrate SDT motivational constructs with neuroscientific study, so we suggest eight new questions for future research.

Knowledge acquisition is governed by striatal prediction errors

Discrepancies between expectations and outcomes, or prediction errors, are central to trial-and-error learning based on reward and punishment, and their neurobiological basis is well characterized. It is not known, however, whether the same principles apply to declarative memory systems, such as those supporting semantic learning. Here, we demonstrate with fMRI that the brain parametrically encodes the degree to which new factual information violates expectations based on prior knowledge and beliefs-most prominently in the ventral striatum, and cortical regions supporting declarative memory encoding. These semantic prediction errors determine the extent to which information is incorporated into long-term memory, such that learning is superior when incoming information counters strong incorrect recollections, thereby eliciting large prediction errors. Paradoxically, by the same account, strong accurate recollections are more amenable to being supplanted by misinformation, engendering false memories. These findings highlight a commonality in brain mechanisms and computational rules that govern declarative and nondeclarative learning, traditionally deemed dissociable.

The anticipation and outcome phases of reward and loss processing: A neuroimaging meta-analysis of the monetary incentive delay task

The processing of rewards and losses are crucial to everyday functioning. Considerable interest has been attached to investigating the anticipation and outcome phases of reward and loss processing, but results to date have been inconsistent. It is unclear if anticipation and outcome of a reward or loss recruit similar or distinct brain regions. In particular, while the striatum has widely been found to be active when anticipating a reward, whether it activates in response to the anticipation of losses as well remains ambiguous. Furthermore, concerning the orbitofrontal/ventromedial prefrontal regions, activation is often observed during reward receipt. However, it is unclear if this area is active during reward anticipation as well. We ran an Activation Likelihood Estimation meta‐analysis of 50 fMRI studies, which used the Monetary Incentive Delay Task (MIDT), to identify which brain regions are implicated in the anticipation of rewards, anticipation of losses, and the receipt of reward. Anticipating rewards and losses recruits overlapping areas including the striatum, insula, amygdala and thalamus, suggesting that a generalised neural system initiates motivational processes independent of valence. The orbitofrontal/ventromedial prefrontal regions were recruited only during the reward outcome, likely representing the value of the reward received. Our findings help to clarify the neural substrates of the different phases of reward and loss processing, and advance neurobiological models of these processes.

The Caudate Nucleus Mediates Learning of Stimulus-Control State Associations

A longstanding dichotomy in cognitive psychology and neuroscience pits controlled, top-down driven behavior against associative, bottom-up driven behavior, where cognitive control processes allow us to override well-learned stimulus-response (S-R) associations. By contrast, some previous studies have raised the intriguing possibility of an integration between associative and controlled processing in the form of stimulus-control state (S-C) associations, the learned linkage of specific stimuli to particular control states, such as high attentional selectivity. The neural machinery mediating S-C learning remains poorly understood, however. Here, we combined human functional magnetic resonance imaging (fMRI) with a previously developed Stroop protocol that allowed us to dissociate reductions in Stroop interference based on S-R learning from those based on S-C learning. We modeled subjects' acquisition of S-C and S-R associations using an associative learning model and then used trial-by-trial S-C and S-R prediction error (PE) estimates in model-based behavioral and fMRI analyses. We found that PE estimates derived from S-C and S-R associations accounted for the reductions in behavioral Stroop interference effects in the S-C and S-R learning conditions, respectively. Moreover, model-based fMRI analyses identified the caudate nucleus as the key structure involved in selectively updating stimulus-control state associations. Complementary analyses also revealed a greater reliance on parietal cortex when using the learned S-R versus S-C associations to minimize Stroop interference. These results support the emerging view that generalizable control states can become associated with specific bottom-up cues, and they place the caudate nucleus of the dorsal striatum at the center of the neural stimulus-control learning machinery.

SIGNIFICANCE STATEMENT

Previous behavioral studies have demonstrated that control states, for instance, heightened attentional selectivity, can become directly associated with, and subsequently retrieved by, particular stimuli, thus breaking down the traditional dichotomy between top-down and bottom-up driven behavior. However, the neural mechanisms underlying this type of stimulus-control learning remain poorly understood. We therefore combined noninvasive human neuroimaging with a task that allowed us to dissociate the acquisition of stimulus-control associations from that of stimulus-response associations. The results revealed the caudate nucleus as the key brain structure involved in selectively driving stimulus-control learning. These data represent the first identification of the neural mechanisms of stimulus-specific control associations, and they significantly extend current conceptions of the type of learning processes mediated by the caudate.

Fronto-striatal network activation leads to less fatigue in multiple sclerosis

BACKGROUND

The fronto-striatal network has been implicated in both fatigue, a common multiple sclerosis (MS) symptom, and goal attainment, which has been shown to reduce fatigue in healthy individuals.

OBJECTIVES

To investigate whether stimulation of the fronto-striatal network through goal attainment (potential monetary gain) leads to fatigue reduction in MS and healthy control (HC) participants.

METHODS

In all, 14 healthy and 19 MS participants performed a gambling task during functional magnetic resonance imaging (fMRI). Participants were presented with an opportunity to receive monetary reward during the outcome condition of the task but not during the no outcome condition. Self-reported fatigue measures were obtained after each condition and outside of the scanner. Structural alterations were also examined.

RESULTS

A significant decrease in fatigue was observed after the outcome condition compared to the no outcome condition in both groups. Significantly greater activation was observed in the ventral striatum in association with the outcome condition compared to the no outcome condition in both groups. Ventromedial prefrontal cortex showed significantly greater activation during the no outcome condition compared to the outcome condition with greater difference between conditions in the HC group.

CONCLUSION

This is the first functional neuroimaging study showing that stimulation of the fronto-striatal network through goal attainment leads to decreased on-task fatigue in MS and healthy participants.

Frontostriatal development and probabilistic reinforcement learning during adolescence

Adolescence has traditionally been viewed as a period of vulnerability to increased risk-taking and adverse outcomes, which have been linked to neurobiological maturation of the frontostriatal reward system. However, growing research on the role of developmental changes in the adolescent frontostriatal system in facilitating learning will provide a more nuanced view of adolescence. In this review, we discuss the implications of existing research on this topic for learning during adolescence, and suggest that the very neural changes that render adolescents vulnerable to social pressure and risky decision making may also stand to play a role in scaffolding the ability to learn from rewards and from performance-related feedback.

Modulation of ventral striatal activity by cognitive effort

Effort discounting theory suggests that the value of a reward should be lower if it was effortful to obtain, whereas contrast theory suggests that the contrast between the costly effort and the reward makes the reward seem more valuable. To test these alternative hypotheses, we used functional magnetic resonance imaging (fMRI) as participants engaged in feedback-based learning that required low or high cognitive effort to obtain positive feedback, while the objective amount of information provided by feedback remained constant. In the low effort condition, a single image was presented with four response options. In the high effort condition, two images were presented, each with two response options, and correct feedback was presented only when participants responded correctly to both of the images. Accuracy was significantly lower for the high effort condition, and all participants reported that the high effort condition was more difficult. A region of the ventral striatum selected for sensitivity to feedback value also showed increased activation to feedback presentation associated with the high effort condition relative to the low effort condition, when controlling for activation from corresponding control conditions where feedback was random. These results suggest that increased cognitive effort produces corresponding increases in positive feedback-related ventral striatum activity, in line with the predictions made by contrast theory. The accomplishment of obtaining a hard-earned intrinsic reward, such as positive feedback, may be particularly likely to promote reward-related brain activity.

The Value of Being Wrong: Intermittent Feedback Delivery Alters the Striatal Response to Negative Feedback

Whereas positive feedback is both rewarding and informative, negative feedback can be construed as either punishing (because it is indicative of poor performance) or informative (because it may lead to goal attainment). In this neuroimaging experiment, we highlighted the informational value of negative feedback by intermixing trials with and without feedback. When performance feedback is expected, positive feedback triggers an increase in striatal activity, whereas negative feedback elicits a decrease in striatal activity. We predicted that, in contrast, when feedback receipt is unpredictable, the striatal response to negative feedback would increase. Participants performed a paired-associate learning task during fMRI scanning. In one condition ("blocked feedback"), the receipt of feedback was predictable--participants knew whether or not they would receive feedback for their responses. In another condition ("mixed feedback"), the receipt of feedback was unpredictable--on a random 50% of trials, participants received feedback, and they otherwise received no feedback. Negative feedback in the mixed feedback condition elicited more striatal activity than negative feedback in the blocked feedback condition. In contrast, feedback omission evoked more striatal activity when feedback delivery was expected, compared to when it was unpredictable. This pattern emerged from an increase in caudate activity in response to negative feedback in the mixed feedback condition and a decrease in ventral striatal activity in response to no feedback in this condition. These results suggest that, by emphasizing the informational value of negative feedback, an unpredictable feedback context alters the striatal response to negative feedback and to the omission of feedback.

The power of competition: Effects of social motivation on attention, sustained physical effort, and learning

Competition has often been implicated as a means to improve effort-based learning and attention. Two experiments examined the effects of competition on effort and memory. In Experiment 1, participants completed a physical effort task in which they were rewarded for winning an overall percentage, or for winning a competition they believed was against another player. In Experiment 2, participants completed a memory task in which they were rewarded for remembering an overall percentage of shapes, or more shapes than a “competitor.” We found that, in the physical effort task, participants demonstrated faster reaction times (RTs)—a previous indicator of increased attention—in the competitive environment. Moreover, individual differences predicted the salience of competition’s effect. Furthermore, male participants showed faster RTs and greater sustained effort as a result of a competitive environment, suggesting that males may be more affected by competition in physical effort tasks. However, in Experiment 2, participants remembered fewer shapes when competing, and later recalled less of these shapes during a post-test, suggesting that competition was harmful in our memory task. The different results from these two experiments suggest that competition can improve attention in a physical effort task, yet caution the use of competition in memory tasks.

Effects of intrinsic motivation on feedback processing during learning

Learning commonly requires feedback about the consequences of one's actions, which can drive learners to modify their behavior. Motivation may determine how sensitive an individual might be to such feedback, particularly in educational contexts where some students value academic achievement more than others. Thus, motivation for a task might influence the value placed on performance feedback and how effectively it is used to improve learning. To investigate the interplay between intrinsic motivation and feedback processing, we used functional magnetic resonance imaging (fMRI) during feedback-based learning before and after a novel manipulation based on motivational interviewing, a technique for enhancing treatment motivation in mental health settings. Because of its role in the reinforcement learning system, the striatum is situated to play a significant role in the modulation of learning based on motivation. Consistent with this idea, motivation levels during the task were associated with sensitivity to positive versus negative feedback in the striatum. Additionally, heightened motivation following a brief motivational interview was associated with increases in feedback sensitivity in the left medial temporal lobe. Our results suggest that motivation modulates neural responses to performance-related feedback, and furthermore that changes in motivation facilitate processing in areas that support learning and memory.

The role of prediction and outcomes in adaptive cognitive control

Humans adaptively perform actions to achieve their goals. This flexible behaviour requires two core abilities: the ability to anticipate the outcomes of candidate actions and the ability to select and implement actions in a goal-directed manner. The ability to predict outcomes has been extensively researched in reinforcement learning paradigms, but this work has often focused on simple actions that are not embedded in hierarchical and sequential structures that are characteristic of goal-directed human behaviour. On the other hand, the ability to select actions in accordance with high-level task goals, particularly in the presence of alternative responses and salient distractors, has been widely researched in cognitive control paradigms. Cognitive control research, however, has often paid less attention to the role of action outcomes. The present review attempts to bridge these accounts by proposing an outcome-guided mechanism for selection of extended actions. Our proposal builds on constructs from the hierarchical reinforcement learning literature, which emphasises the concept of reaching and evaluating informative states, i.e., states that constitute subgoals in complex actions. We develop an account of the neural mechanisms that allow outcome-guided action selection to be achieved in a network that relies on projections from cortical areas to the basal ganglia and back-projections from the basal ganglia to the cortex. These cortico-basal ganglia-thalamo-cortical 'loops' allow convergence - and thus integration - of information from non-adjacent cortical areas (for example between sensory and motor representations). This integration is essential in action sequences, for which achieving an anticipated sensory state signals the successful completion of an action. We further describe how projection pathways within the basal ganglia allow selection between representations, which may pertain to movements, actions, or extended action plans. The model lastly envisages a role for hierarchical projections from the striatum to dopaminergic midbrain areas that enable more rostral frontal areas to bias the selection of inputs from more posterior frontal areas via their respective representations in the basal ganglia.