Modulation of ventral striatal activity by cognitive effort

The neuroscience of active learning and direct instruction

Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.

Bypassing Striatal Learning Mechanisms Using Delayed Feedback to Circumvent Learning Deficits in Traumatic Brain Injury

OBJECTIVE

Feedback facilitates learning by guiding and modifying behaviors through an action-outcome contingency. As the majority of existing studies have focused on the immediate presentation of feedback, the impact of delayed feedback on learning is understudied. Prior work demonstrated that learning from immediate and delayed feedback employed distinct brain regions in healthy individuals, and compared to healthy individuals, individuals with traumatic brain injury (TBI) are impaired in learning from immediate feedback. The goal of the current investigation was to assess the effects of delayed vs immediate feedback on learning in individuals with TBI and examine brain networks associated with delayed and immediate feedback processing.

SETTING

Nonprofit research organization.

PARTICIPANTS

Twenty-eight individuals with moderate-to-severe TBI.

DESIGN

Participants completed a paired-associate word learning task while undergoing magnetic resonance imaging. During the task, feedback was presented either immediately, after a delay, or not at all (control condition).

MAIN MEASURES

Learning performance accuracy, confidence ratings, post-task questionnaire, and blood oxygen level-dependent signal.

RESULTS

Behavioral data showed that delayed feedback resulted in better learning performance than immediate feedback and no feedback. In addition, participants reported higher confidence in their performance during delayed feedback trials. During delayed vs immediate feedback processing, greater activation was observed in the superior parietal and angular gyrus. Activation in these areas has been previously associated with successful retrieval and greater memory confidence.

CONCLUSION

The observed results might be explained by delayed feedback processing circumventing the striatal dopaminergic regions responsible for learning from immediate feedback that are impaired in TBI. In addition, delayed feedback evokes less of an affective reaction than immediate feedback, which likely benefited memory performance. Indeed, compared to delayed feedback, positive or negative immediate feedback was more likely to be rated as rewarding or punishing, respectively. The findings have significant implications for TBI rehabilitation and suggest that delaying feedback during rehabilitation might recruit brain regions that lead to better functional outcomes.

Disentangling effort from probability of success: Temporal dynamics of frontal midline theta in effort-based reward processing

The ability to weigh a reward against the effort required to acquire it is critical for decision-making. However, extant experimental paradigms oftentimes confound increased effort demand with decreased reward probability, thereby obscuring neural correlates underlying these cognitive processes. To resolve this issue, we designed novel tasks that disentangled probability of success - and therefore reward probability - from effort demand. In Experiment 1, reward magnitude and effort demand were varied while reward probability was kept constant. In Experiment 2, effort demand and reward probability were varied while reward magnitude remained fixed. Electroencephalogram (EEG) data was recorded to explore how frontal midline theta (FMT; an electrophysiological index of mPFC function) and component P3 (an index of incentive salience) respond to effort demand, and reward magnitude and probability. We found no evidence that FMT tracked effort demands or net value during cue evaluation. At feedback, however, FMT power was enhanced for high compared to low effort trials, but not modulated by reward magnitude or probability. Conversely, P3 was sensitive to reward magnitude and probability at both cue and feedback phases and only integrated expended effort costs at feedback, such that P3 amplitudes continued to scale with reward magnitude and probability but were also increased for high compared to low effort reward feedback. These findings suggest that, when likelihood of success is equal, FMT power does not track net value of prospective effort-based rewards. Instead, expended cognitive effort potentiates FMT power and enhances the saliency of rewards at feedback. SIGNIFICANCE STATEMENT: The way the brain weighs rewards against the effort required to achieve them is critical for understanding motivational disorders. Current paradigms confound increased effort demand with decreased reward probability, making it difficult to disentangle neural activity associated with effort costs from those associated with reward likelihood. Here, we explored the temporal dynamics of effort-based reward (via frontal midline theta (FMT) and component P3) while participants underwent a novel paradigm that kept probability of reward constant between mental effort demand conditions. Our findings suggest that the FMT does not track net value and that expended effort enhances, instead of attenuates, the saliency of rewards.

Altered neural anticipation of reward and loss but not receipt in adolescents with obsessive-compulsive disorder

BACKGROUND

Obsessive-compulsive disorder (OCD) is characterized by persistent, unwanted thoughts and repetitive actions. Such repetitive thoughts and/or behaviors may be reinforced either by reducing anxiety or by avoiding a potential threat or harm, and thus may be rewarding to the individual. The possible involvement of the reward system in the symptomatology of OCD is supported by studies showing altered reward processing in reward-related regions, such as the ventral striatum (VS) and the orbitofrontal cortex (OFC), in adults with OCD. However, it is not clear whether this also applies to adolescents with OCD.

METHODS

Using functional magnetic resonance imaging, two sessions were conducted focusing on the anticipation and receipt of monetary reward (1) or loss (2), each contrasted to a verbal (control) condition. In each session, adolescents with OCD (n1=31/n2=26) were compared with typically developing (TD) controls (n1=33/ n2=31), all aged 10-19 years, during the anticipation and feedback phase of an adapted Monetary Incentive Delay task.

RESULTS

Data revealed a hyperactivation of the VS, but not the OFC, when anticipating both monetary reward and loss in the OCD compared to the TD group.

CONCLUSIONS

These findings suggest that aberrant neural reward and loss processing in OCD is associated with greater motivation to gain or maintain a reward but not with the actual receipt. The greater degree of reward 'wanting' may contribute to adolescents with OCD repeating certain actions more and more frequently, which then become habits (i.e., OCD symptomatology).

Dissociable neural after-effects of cognitive and physical effort expenditure during reward evaluation

The reward after-effect of effort expenditure refers to the phenomenon that previous effort investment changes the subjective value of rewards when obtained. However, the neural mechanisms underlying the after-effects of effort exertion are still not fully understood. We investigated the modulation of reward after-effects by effort type (cognitive vs. physical) through the lens of neural dynamics. Thirty-two participants performed a physically or cognitively demanding task during an effort phase and then played a simple gambling game during a subsequent reward phase to earn monetary rewards while their electroencephalogram (EEG) was recorded. We found that previous effort expenditure decreased electrocortical activity during feedback evaluation. Importantly, this effort effect occurred in a domain-general manner during the early stage (as indexed by the reward positivity) but in a domain-specific manner during the later and more elaborative stage (as indexed by the P3 and delta oscillation) of reward evaluation. Additionally, effort expenditure enhanced P3 sensitivity to feedback valence regardless of effort type. Our findings suggest that cognitive and physical effort, although bearing some surface resemblance to each other, may have dissociable neural influences on the reward after-effects.

Learned industriousness as a translational mechanism in anorexia nervosa

It remains unexplained why some behaviours persist despite being non-hedonic and ostensibly aversive. This phenomenon is especially baffling when such behaviours are taken to excess in the form of psychopathology. Anorexia nervosa is one psychiatric disorder in which effortful behaviours that most people find unpleasant (suchas restrictive eating) are persistently performed. We propose thatthe social psychology theory of learned industriousness providesa novel mechanistic account for such phenomena. This theoryposits that high-effort behaviour can be conditioned to acquire secondary reinforcing properties through repeated pairing with reward. Accordingly, effort sensations become less aversive andmore appetitive, increasing willingness to engage in effortful behaviour. In this Perspective, we review pre-clinical behaviouraland biological data that support learned industriousness, contrast learned industriousness with other models of non-hedonic persistence (such as habit learning), highlight evidence that supports learned industriousness in individuals with anorexia nervosa and consider implications of the model, including translation to other psychiatric presentations.

Motivated for near impossibility: How task type and reward modulate task enjoyment and the striatal activation for extremely difficult task

Economic and decision-making theories suppose that people would disengage from a task with near zero success probability, because this implicates little normative utility values. However, humans often are motivated for an extremely challenging task, even without any extrinsic incentives. The current study aimed to address the nature of this challenge-based motivation and its neural correlates. We found that, when participants played a skill-based task without extrinsic incentives, their task enjoyment increased as the chance of success decreased, even if the task was almost impossible to achieve. However, such challenge-based motivation was not observed when participants were rewarded for the task or the reward was determined in a probabilistic manner. The activation in the ventral striatum/pallidum tracked the pattern of task enjoyment. These results suggest that people are intrinsically motivated to challenge a nearly impossible task but only when the task requires certain skills and extrinsic rewards are unavailable.

Expending effort may share neural responses with reward and evokes high subjective satisfaction

Throughout our daily lives, the levels of effort we invest in various tasks are influenced by reward processing. The subjective expectation after expending effort is a primary factor affecting reward processing. However, recent studies indicate that individual differences in reward anticipation influence this subjective valuation. To better understand the relationship between effort expenditure and the subjective valuation of rewards, in this study, we perform an experiment in which we manipulate effort, control reward expectation implicitly, and measure the subjective valuation of rewards using event-related potentials (ERPs) and physical effort through behavioral measures (number of keystrokes). In the reward-task paradigm, 30 subjects performed effort and control trials, with the reward probability comparable across the effort and control conditions. We also examined the ERPs associated with the valuation of subjective rewards, including reward positivity (RewP) and set reward expectation controlled as the baseline. The results showed that the ERP amplitudes, the number of keystrokes, and explicit satisfaction ratings were all significantly greater in the effort condition than in the control condition. The participants maintained high levels of effort throughout the sessions associated with the experiment. The results of this study suggest that when reward expectations are controlled, effort expenditure evokes neural responses similar to reward feedback being given, which is linked with increased subjective satisfaction.

Altered functional connectivity during performance feedback processing in multiple sclerosis

Effective learning from performance feedback is vital for adaptive behavior regulation necessary for successful cognitive performance. Yet, how this learning operates in clinical groups that experience cognitive dysfunction is not well understood. Multiple sclerosis (MS) is an autoimmune, degenerative disease of the central nervous system characterized by physical and cognitive dysfunction. A highly prevalent impairment in MS is cognitive fatigue (CF). CF is associated with altered functioning within cortico-striatal regions that also facilitate feedback-based learning in neurotypical (NT) individuals. Despite this cortico-striatal overlap, research about feedback-based learning in MS, its associated neural underpinnings, and its sensitivity to CF, are all lacking. The present study investigated feedback-based learning ability in MS, as well as associated cortico-striatal function and connectivity. MS and NT participants completed a functional magnetic resonance imaging (fMRI) paired-word association task during which they received trial-by-trial monetary, non-monetary, and uninformative performance feedback. Despite reporting greater CF throughout the task, MS participants displayed comparable task performance to NTs, suggesting preserved feedback-based learning ability in the MS group. Both groups recruited the ventral striatum (VS), caudate nucleus, and ventromedial prefrontal cortex in response to the receipt of performance feedback, suggesting that people with MS also recruit cortico-striatal regions during feedback-based learning. However, compared to NT participants, MS participants also displayed stronger functional connectivity between the VS and task-relevant regions, including the left angular gyrus and right superior temporal gyrus, in response to feedback receipt. Results indicate that CF may not interfere with feedback-based learning in MS. Nonetheless, people with MS may recruit alternative connections with the striatum to assist with this form of learning. These findings have implications for cognitive rehabilitation treatments that incorporate performance feedback to remediate cognitive dysfunction in clinical populations.

Is loss avoidance differentially rewarding in adolescents versus adults? Differences in ventral striatum and anterior insula activation during the anticipation of potential monetary losses

Avoiding loss is a crucial, adaptive guide to human behavior. While previous developmental research has primarily focused on gaining rewards, less attention has been paid to loss processing and its avoidance. In daily life, it is often unknown how likely an action will result in a loss, making the role of uncertainty in loss processing particularly important. By using functional magnetic resonance imaging, we investigated the influence of varying outcome probabilities (12%, 34%, and 67%) on brain regions implicated in loss processing (ventral striatum (VS), anterior insula (AI)) by comparing 28 adolescents (10-18 years) and 24 adults (22-32 years) during the anticipation of potential monetary loss.Overall, results revealed slower RTs in adolescents compared to adults with both groups being faster in the experimental (monetary condition) vs. control trials (verbal condition). Fastest RTs were observed for the 67% outcome probability in both age groups. An age group × outcome probability interaction effect revealed the greatest differences between the groups for the 12% vs. the 67% outcome probability. Neurally, both age groups demonstrated a higher percent signal change in the VS and AI during the anticipation of potential monetary loss versus the verbal condition. However, adults demonstrated an even greater activation of VS and AI than adolescents during the anticipation of potential monetary loss, but not during the verbal condition. This may indicate that adolescents differ from adults regarding their experience of avoiding losing monetary rewards.

Reward enhances connectivity between the ventral striatum and the default mode network

The default mode network (DMN) has been theorized to participate in a range of social, cognitive, and affective functions. Yet, previous accounts do not consider how the DMN contributes to other brain regions depending on psychological context, thus rendering our understanding of DMN function incomplete. We addressed this gap by applying a novel network-based psychophysiological interaction (nPPI) analysis to the reward task within the Human Connectome Project. We first focused on the task-evoked responses of the DMN and other networks involving the prefrontal cortex, including the executive control network (salience network) and the left and right frontoparietal networks. Consistent with a host of prior studies, the DMN exhibited a relative decrease in activation during the task, while the other networks exhibited a relative increase during the task. Next, we used nPPI analyses to assess whether these networks exhibit task-dependent changes in connectivity with other brain regions. Strikingly, we found that the experience of reward enhances task-dependent connectivity between the DMN and the ventral striatum, an effect that was specific to the DMN. Surprisingly, the strength of DMN-VS connectivity was correlated with personality characteristics relating to openness. Taken together, these results advance models of DMN by demonstrating how it contributes to other brain systems during task performance and how those contributions relate to individual differences.

Neural dynamics of effort-modulated reward processing

Previous studies have established that effort can either decrease reward value as a cost or increase reward value as an augmenter, which is referred to as the effort paradox. The present event-related potential study investigated the neural dynamics underlying the modulation of reward processing by effort expenditure. Thirty-two participants completed a modified monetary incentive delay task in a high-effort context and a low-effort context to earn monetary rewards while their EEG was recorded during effort anticipation, effort execution, and effort completion, respectively. Results revealed the multiphase nature of the effort-reward relationship, which unfolds over time. During effort anticipation, effort discounts reward value as reflected by a reduced contingent negative variation for high- versus low-effort trials. During effort exertion, the target-P3 reward effect was accrued by effort expenditure. Finally, effort also provides its own value following effort completion, as revealed by an enhanced stimulus-preceding negativity for high-effort relative to low-effort trials. These findings provide new insights into the relationship between effort expenditure and reward processing and are helpful to mitigate the effort paradox.

Signal Detection Theory as a Novel Tool to Understand Cognitive Fatigue in Individuals With Multiple Sclerosis

Multiple Sclerosis (MS) affects 2.8 million persons worldwide. One of the most persistent, pervasive, and debilitating symptoms of MS is cognitive fatigue. While this has been known for over a century, cognitive fatigue has been difficult to study because patients' subjective (self-reported) cognitive fatigue has consistently failed to correlate with more objective measures, such as reaction time (RT) and accuracy. Here, we investigated whether more nuanced metrics of performance, specifically the metrics of Signal Detection Theory (SDT), would show a relationship to cognitive fatigue even if RT and accuracy did not. We also measured brain activation to see whether SDT metrics were related to activation in brain areas that have been shown to be sensitive to cognitive fatigue. Fifty participants (30 MS, 20 controls) took part in this study and cognitive fatigue was induced using four blocks of a demanding working memory paradigm. Participants reported their fatigue before and after each block, and their performance was used to calculate SDT metrics (Perceptual Certainty and Criterion) and RT and accuracy. The results showed that the SDT metric of Criterion (i.e., response bias) was positively correlated with subjective cognitive fatigue. Moreover, the activation in brain areas previously shown to be related to cognitive fatigue, such as the striatum, was also related to Criterion. These results suggest that the metrics of SDT may represent a novel tool with which to study cognitive fatigue in MS and other neurological populations. These results hold promise for characterizing cognitive fatigue in MS and developing effective interventions in the future.

Cognitive Effort Exertion Enhances Electrophysiological Responses to Rewarding Outcomes

Recent work has highlighted neural mechanisms underlying cognitive effort-related discounting of anticipated rewards. However, findings on whether effort exertion alters the subjective value of obtained rewards are inconsistent. Here, we provide a more nuanced account of how cognitive effort affects subsequent reward processing in a novel task designed to assess effort-induced modulations of the Reward Positivity, an event-related potential indexing reward-related neural activity. We found that neural responses to both gains and losses were significantly elevated in trials requiring more versus less cognitive effort. Moreover, time-frequency analysis revealed that these effects were mirrored in gain-related delta, but not in loss-related theta band activity, suggesting that people ascribed more value to high-effort outcomes. In addition, we also explored whether individual differences in behavioral effort discounting rates and reward sensitivity in the absence of effort may affect the relationship between effort exertion and subsequent reward processing. Together, our findings provide evidence that cognitive effort exertion can increase the subjective value of subsequent outcomes and that this effect may primarily rely on modulations of delta band activity.

Rewarding cognitive effort increases the intrinsic value of mental labor

Many extraordinary human skills like reading, mastering an instrument, or programming require thousands of hours of practice and continued exertion of mental effort. However, the importance of mental effort often contrasts with currently dominant theories suggesting that effort is aversive and something people avoid whenever possible. Here, we show that rewarding participants for the exertion of effort in a cognitive task increased their preference for more demanding tasks in a transfer phase. This provides evidence that people can learn to positively value effort and demanding tasks in the absence of extrinsic reward. These findings challenge currently dominant theories of mental effort and point to the role of learning environments for the development of effort-related motivation.

Current models of mental effort in psychology, behavioral economics, and cognitive neuroscience typically suggest that exerting cognitive effort is aversive, and people avoid it whenever possible. The aim of this research was to challenge this view and show that people can learn to value and seek effort intrinsically. Our experiments tested the hypothesis that effort-contingent reward in a working-memory task will induce a preference for more demanding math tasks in a transfer phase, even though participants were aware that they would no longer receive any reward for task performance. In laboratory Experiment 1 (n = 121), we made reward directly contingent on mobilized cognitive effort as assessed via cardiovascular measures (β-adrenergic sympathetic activity) during the training task. Experiments 2a to 2e (n = 1,457) were conducted online to examine whether the effects of effort-contingent reward on subsequent demand seeking replicate and generalize to community samples. Taken together, the studies yielded reliable evidence that effort-contingent reward increased participants’ demand seeking and preference for the exertion of cognitive effort on the transfer task. Our findings provide evidence that people can learn to assign positive value to mental effort. The results challenge currently dominant theories of mental effort and provide evidence and an explanation for the positive effects of environments appreciating effort and individual growth on people’s evaluation of effort and their willingness to mobilize effort and approach challenging tasks.

Relationship of ventral striatum activation during effort discounting to clinical amotivation severity in schizophrenia

Motivational deficits play a central role in disability due to negative symptoms of schizophrenia (SZ), but limited pathophysiological understanding impedes critically needed therapeutic development. We applied an fMRI Effort Discounting Task (EDT) that quantifies motivation using a neuroeconomic decision-making approach, capturing the degree to which effort requirements produce reductions in the subjective value (SV) of monetary reward. An analyzed sample of 21 individuals with SZ and 23 group-matched controls performed the EDT during fMRI. We hypothesized that ventral striatum (VS) as well as extended brain motivation circuitry would encode SV, integrating reward and effort costs. We also hypothesized that VS hypoactivation during EDT decisions would demonstrate a dimensional relationship with clinical amotivation severity, reflecting greater suppression by effort costs. As hypothesized, VS as well as a broader cortico-limbic network were activated during the EDT and this activation correlated positively with SV. In SZ, activation to task decisions was reduced selectively in VS. Greater VS reductions correlated with more severe clinical amotivation in SZ and across all participants. However, these diagnosis and amotivation effects could not be explained by the response to parametric variation in reward, effort, or model-based SV. Our findings demonstrate that VS hypofunction in schizophrenia is manifested during effort-based decisions and reflects dimensional motivation impairment. Dysfunction of VS impacting effort-based decision-making can provide a target for biomarker development to guide novel efforts to assess and treat disabling amotivation.

Ventral Striatal Activation During Reward Anticipation of Different Reward Probabilities in Adolescents and Adults

Adolescence has been linked to an enhanced tolerance of uncertainty and risky behavior and is possibly connected to an increased response toward rewards. However, previous research has produced inconsistent findings. To investigate whether these findings are due to different reward probabilities used in the experimental design, we extended a monetary incentive delay (MID) task by including three different reward probabilities. Using functional magnetic resonance imaging, 25 healthy adolescents and 22 adults were studied during anticipation of rewards in the VS. Differently colored cue stimuli indicated either a monetary or verbal trial and symbolized different reward probabilities, to which the participants were blinded. Results demonstrated faster reaction times for lower reward probabilities (33%) in both age groups. Adolescents were slower through all conditions and had less activation on a neural level. Imaging results showed a three-way interaction between age group x condition x reward probability with differences in percent signal change between adolescents and adults for the high reward probabilities (66%, 88%) while adolescents demonstrated differences for the lowest (33%). Therefore, previous inconsistent findings could be due to different reward probabilities, which makes examining these crucial for a better understanding of adolescent and adult behavior.

Dissociable Motivational Deficits in Pre-manifest Huntington's Disease

Motivation is characterized by a willingness to overcome both cognitive and physical effort costs. Impairments in motivation are common in striatal disorders, such as Huntington’s disease (HD), but whether these impairments are isolated to particular domains of behavior is controversial. We ask whether HD differentially affects the willingness of individuals to overcome cognitive versus physical effort. We tested 20 individuals with pre-manifest HD and compared their behavior to 20 controls. Across separate trials, participants made choices about how much cognitive or physical effort they were willing to invest for reward. Our key results were that individuals with pre-manifest HD were less willing than controls to invest cognitive effort but were no different in their overall preference for physical effort. These results cannot be explained by group differences in neuropsychological or psychiatric profiles. This dissociation of cognitive- and physical-effort-based decisions provides important evidence for separable, domain-specific mechanisms of motivation.

We examine cognitive and physical effort discounting in pre-manifest HD

Individuals with pre-manifest HD are less cognitively motivated than controls

There are no differences in physical motivation between the two groups

This dissociation is not confounded by neuropsychological or psychiatric factors

Moving towards specificity: A systematic review of cue features associated with reward and punishment in anorexia nervosa

Models of anorexia nervosa (AN) posit that symptoms are maintained through deficient reward and enhanced punishment processing. However, theoretical and empirical inconsistencies highlight the need for a more nuanced conceptualization of this literature. Our goal was to comprehensively review the research on reward and punishment responding in AN from a cue-specific lens to determine which stimuli evoke or discourage reward and punishment responses in this population, and, ultimately, what properties these rewarding and punishing cues might share. A systematic review interrogating reward and punishment responses to specific cues yielded articles (n = 92) that examined responses to disorder relevant (e.g., food) and irrelevant (e.g., money) stimuli across self-report, behavioral, and biological indices. Overall, in most studies individuals with AN exhibited aversive responses to cues signaling higher body weights, social contexts, and monetary losses, and appetitive responses to cues for weight loss behaviors and thinness. Findings were more mixed on responses to palatable food and monetary gains. Results highlight that reward and punishment responding in AN are context specific and may be affected by varied stimulus qualities (e.g., predictability, controllability, delay, effort). Increasing specificity in future research on reward and punishment mechanisms in AN will better inform development of precisely-targeted interventions.

Decomposing the effort paradox in reward processing: Time matters

Effort expenditure not only discounts reward value prospectively but also accrues reward value retrospectively. To decompose the effort paradox in reward processing, the current event-related potential study investigated the neural dynamics underlying effects of effort expenditure on subsequent reward processing. Participants exerted one of two levels of effort to obtain an opportunity of winning a high or low amount of monetary reward, and we focused on electrophysiological activity during the anticipatory and consummatory phases of reward processing. During the anticipatory phase, the stimulus-preceding negativity was enhanced when potential high rewards were anticipated, but this reward effect disappeared following high-effort expenditure. During the consummatory phase, feedback-related ERPs were increased for high relative to low rewards, and this reward effect was enlarged following effort expenditure during the early stage (200-300 ms) as indexed by the reward positivity but not the late stage (400-600 ms) as indexed by the P3. Our findings provide a strong support for the psychological contrast theory and indicate that time matters in decomposing the effort paradox for reward processing such that effort expenditure reduces reward sensitivity during the anticipatory phase but enhances reward sensitivity during the consummatory phase.

Reward presentation reduces on-task fatigue in traumatic brain injury

While cognitive fatigue is experienced by up to 80% of individuals with traumatic brain injury (TBI), little is known about its neural underpinnings. We previously hypothesized that presentation of rewarding outcomes leads to cognitive fatigue reduction and activation of the striatum, a brain region shown to be associated with cognitive fatigue in clinical populations and processing of rewarding outcomes. We have demonstrated this in individuals with multiple sclerosis. Here, we tested this hypothesis in individuals with TBI. Twenty-one individuals with TBI and 24 healthy participants underwent functional magnetic resonance imaging. Participants performed a task during which they were presented with 1) the Outcome condition where they were exposed to monetary rewards, and 2) the No Outcome condition that served as the control condition and was not associated with monetary rewards. In accordance with our hypothesis, results showed that attainment of rewarding outcomes leads to cognitive fatigue reduction in individuals with TBI, as well as activation of the striatum. Specifically, we observed a significant group by condition interaction on fatigue scores driven by the TBI group reporting lower levels of fatigue after the Outcome condition. fMRI data revealed a significant main-effect of condition in regions previously implicated in outcome processing, while a significant group by condition interaction was observed in the left ventral striatum as revealed by a priori region of interest analysis. Results suggest that a salient motivator can significantly reduce fatigue and that outcome presentation leads to increased activation of the ventral striatum in TBI. These findings can inform the development of future non-pharmacological cognitive fatigue treatment methods and contribute to the growing body of evidence showing the association between cognitive fatigue and the striatum.

Intact Reinforcement Learning But Impaired Attentional Control During Multidimensional Probabilistic Learning in Older Adults

To efficiently learn optimal behavior in complex environments, humans rely on an interplay of learning and attention. Healthy aging has been shown to independently affect both of these functions. Here, we investigate how reinforcement learning and selective attention interact during learning from trial and error across age groups. We acquired behavioral and fMRI data from older and younger adults (male and female) performing two probabilistic learning tasks with varying attention demands. Although learning in the unidimensional task did not differ across age groups, older adults performed worse than younger adults in the multidimensional task, which required high levels of selective attention. Computational modeling showed that choices of older adults are better predicted by reinforcement learning than Bayesian inference, and that older adults rely more on reinforcement learning-based predictions than younger adults. Conversely, a higher proportion of younger adults' choices was predicted by a computationally demanding Bayesian approach. In line with the behavioral findings, we observed no group differences in reinforcement-learning related fMRI activation. Specifically, prediction-error activation in the nucleus accumbens was similar across age groups, and numerically higher in older adults. However, activation in the default mode was less suppressed in older adults in for higher attentional task demands, and the level of suppression correlated with behavioral performance. Our results indicate that healthy aging does not significantly impair simple reinforcement learning. However, in complex environments, older adults rely more heavily on suboptimal reinforcement-learning strategies supported by the ventral striatum, whereas younger adults use attention processes supported by cortical networks.SIGNIFICANCE STATEMENT Changes in the way that healthy human aging affects how we learn to optimally behave are not well understood; it has been suggested that age-related declines in dopaminergic function may impair older adult's ability to learn from reinforcement. In the present fMRI experiment, we show that learning and nucleus accumbens activation in a simple unidimensional reinforcement-learning task was not significantly affected by age. However, in a more complex multidimensional task, older adults showed worse performance and relied more on reinforcement-learning strategies than younger adults, while failing to disengage their default-mode network during learning. These results imply that older adults are only impaired in reinforcement learning if they additionally need to learn which dimensions of the environment are currently important.

Cognitive and behavioral training interventions to promote self-control

This review article discusses various cognitive and behavioral interventions that have been developed with the goal of promoting self-controlled responding. Self-control can exert a significant impact on human health and impulsive behaviors are associated with a wide range of diseases and disorders, leading to the suggestion that impulsivity is a trans-disease process. The self-control interventions include effort exposure, reward discrimination, reward bundling, interval schedules of reinforcement, impulse control training, and mindfulness training. Most of the interventions have been consistently shown to increase self-control, except for mindfulness training. Some of the successful interventions are long-lasting, whereas others may be transient. Most interventions are domain-specific, targeting specific cognitive and behavioral processes that relate to self-control rather than targeting overall self-control. For example, effort exposure appears to primarily increase effort tolerance, which in turn can improve self-control. Similarly, interval schedules primarily target interval timing, which promotes self-controlled responses. A diagram outlining a proposed set of intervention effects on self-control is introduced to motivate further research in this area. The diagram suggests that the individual target processes of the interventions may potentially summate to produce general self-control, or perhaps even produce synergistic effects. In addition, it is suggested that developing a self-control profile may be advantageous for aligning specific interventions to mitigate specific deficits. Overall, the results indicate that interventions are a promising avenue for promoting self-control and may help to contribute to changing health outcomes associated with a wide variety of diseases and disorders. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The Subjective Value of Cognitive Effort is Encoded by a Domain-General Valuation Network

Cognitive control is necessary for goal-directed behavior, yet people treat cognitive control demand as a cost, which discounts the value of rewards in a similar manner as other costs, such as delay or risk. It is unclear, however, whether the subjective value (SV) of cognitive effort is encoded in the same putatively domain-general brain valuation network implicated in other cost domains, or instead engages a distinct frontoparietal network, as implied by recent studies. Here, we provide rigorous evidence that the valuation network, with core foci in the ventromedial prefrontal cortex and ventral striatum, also encodes SV during cognitive effort-based decision-making in healthy, male and female adult humans. We doubly dissociate this network from frontoparietal regions that are instead recruited as a function of decision difficulty. We show that the domain-general valuation network jointly and independently encodes both reward benefits and cognitive effort costs. We also demonstrate that cognitive effort SV signals predict choice and are influenced by state and trait motivation, including sensitivity to reward and anticipated task performance. These findings unify cognitive effort with other cost domains, and suggest candidate neural mechanisms underlying state and trait variation in willingness to expend cognitive effort.SIGNIFICANCE STATEMENT Subjective effort costs are increasingly understood to diminish cognitive control over task performance and can thus undermine functioning across health and disease. Yet, we are only beginning to understand how decisions about cognitive effort are made. A key question is how subjective values are computed. Recent work suggests that the value of cognitive effort might be computed by networks that are distinct from those involved in other domains like intertemporal and risky decision-making, implying distinct mechanisms. Here we demonstrate that the domain-general network also encodes effort-discounted value, linking cognitive effort closely with other domains. Our results thus elucidate key mechanisms supporting decisions about cognitive effort, and point to candidate neural targets for intervention in disorders involving impaired cognitive motivation.

Reward circuitry activation reflects social preferences in the face of cognitive effort

Research at the intersection of social neuroscience and cognitive effort is an interesting new area for exploration. There is great potential to broaden our understanding of how social context and cognitive effort processes, currently addressed in disparate literatures, interact with one another. In this paper, we briefly review the literature on cognitive effort, focusing on effort-linked valuation and the gap in the literature regarding cognitive effort in the social domain. Next, we present a study designed to explore valuation processes linked to cognitive effort within the social context of an inequality manipulation. More specifically, we created monetary inequality among the participant (SELF, endowed with $50) and two confederates: one also endowed with $50 (OTHER HIGH) and another with only $5 (OTHER LOW). We then scanned participants using fMRI as they attempted to earn bonus payments for themselves and others through a cognitively effortful feedback-based learning task. Positive feedback produced significantly greater activation than negative feedback in key valuation regions, the ventral striatum (VS) and ventromedial prefrontal cortex (vmPFC), both when participants were performing the task on their own behalf and when earning rewards for others. While reward-related activity in the VS was exaggerated for SELF compared to OTHER HIGH for both positive and negative feedback, activity in the vmPFC did not distinguish between recipients in the group-level results. Furthermore, participants naturally fell into two groups: those most engaged when playing for themselves and those who reported engagement for others. While Self-Engaged participants showed differences between the SELF and both OTHER conditions in the VS and vmPFC, Other-Engaged participants only showed an attenuated response to negative feedback for OTHER HIGH compared to SELF in the VS and no differences between recipient conditions in the vmPFC. Together, this work shows the importance of individual differences and the fragility of advantageous inequality aversion in the face of cognitive effort, highlighting the need to study cognitive effort in the social domain.