Dopamine Manipulation Affects Response Vigor Independently of Opportunity Cost

Valence-dependent dopaminergic modulation during reversal learning in Parkinson's disease: A neurocomputational approach

Reinforcement learning, crucial for behavior in dynamic environments, is driven by rewards and punishments, modulated by dopamine (DA) changes. This study explores the dopaminergic system's influence on learning, particularly in Parkinson's disease (PD), where medication leads to impaired adaptability. Highlighting the role of tonic DA in signaling the valence of actions, this research investigates how DA affects response vigor and decision-making in PD. DA not only influences reward and punishment learning but also indicates the cognitive effort level and risk propensity in actions, which are essential for understanding and managing PD symptoms. In this work, we adapt our existing neurocomputational model of basal ganglia (BG) to simulate two reversal learning tasks proposed by Cools et al. We first optimized a Hebb rule for both probabilistic and deterministic reversal learning, conducted a sensitivity analysis (SA) on parameters related to DA effect, and compared performances between three groups: PD-ON, PD-OFF, and control subjects. In our deterministic task simulation, we explored switch error rates after unexpected task switches and found a U-shaped relationship between tonic DA levels and switch error frequency. Through SA, we classify these three groups. Then, assuming that the valence of the stimulus affects the tonic levels of DA, we were able to reproduce the results by Cools et al. As for the probabilistic task simulation, our results are in line with clinical data, showing similar trends with PD-ON, characterized by higher tonic DA levels that are correlated with increased difficulty in both acquisition and reversal tasks. Our study proposes a new hypothesis: valence, signaled by tonic DA levels, influences learning in PD, confirming the uncorrelation between phasic and tonic DA changes. This hypothesis challenges existing paradigms and opens new avenues for understanding cognitive processes in PD, particularly in reversal learning tasks.

Sex- and estrous-related response patterns for alcohol depend critically on the level of compulsion-like challenge

Alcohol use disorder is a substantial social and economic burden. During the last years, the number of women with drinking problems has been increasing, and one main concern is that they are particularly more vulnerable to negative consequences of alcohol. However, little is known about female-specific response patterns for alcohol, and potential underlying differences in brain mechanisms, including for compulsion-like alcohol drinking (when intake persists despite adverse consequences). We used lickometry to assess behavioral microstructure in adult Wistar male and female rats (n = 28-30) during alcohol-only drinking or moderate- or higher-challenge alcohol compulsion (10 or 60 mg/l quinine in alcohol, respectively). Estrous stages were determined and related to drinking levels and patterns of responding to alcohol, as was ovariectomy. Our findings showed that females (where we didn't determine estrus stage) had similar total licks in a session as males, but significantly longer licking bouts under alcohol-only and moderate-challenge, suggesting greater persistence. Further, greater intake under alcohol-only and moderate-challenge was related to faster licking in males, while female consumption was not related to licking speed. Thus, females could have increased persistence without greater vigor, unlike males. However, under higher-challenge, faster licking did predict higher intake in females, similar to males. To better understand female higher-challenge responding, we examined drinking in relation to phases of the estrous cycle. Higher-challenge had longer bouts only in late diestrus. In addition, ovariectomy led to longer bouts only under higher-challenge, suggesting that conditions with reduced hormone levels could increase female persistence for alcohol under higher-challenge. However, ovariectomy also reduced alcohol-only and moderate-challenge drinking but did not reduce bout length. Thus, intake level and response strategy could be regulated somewhat differently by ovarian hormones. Finally, moderate-challenge licking speed was less variable during early diestrus, and we previously showed more stereotyped responding specifically under moderate-challenge in males. By combining behavioral microstructure and sex- and estrus-related changes in drinking patterns, our results suggest that females have greater persistence for alcohol under lower-challenge drinking, while late diestrus and ovariectomy unmasked greater persistence under higher-challenge. Together, our novel insights could help develop more effective and personalized treatments for problematic alcohol use.

Non-invasive vagus nerve stimulation conditions increased invigoration and wanting in depression

BACKGROUND

Major depressive disorder (MDD) is often marked by impaired motivation and reward processing, known as anhedonia. Many patients do not respond to first-line treatments, and improvements in motivation can be slow, creating an urgent need for rapid interventions. Recently, we demonstrated that transcutaneous auricular vagus nerve stimulation (taVNS) acutely boosts effort invigoration in healthy participants, but its effects on depression remain unclear.

OBJECTIVE

To assess the impact of taVNS on effort invigoration and maintenance in a sample that includes patients with MDD, evaluating the generalizability of our findings.

METHODS

We used a single-blind, randomized crossover design in 30 patients with MDD and 29 matched (age, sex, and BMI) healthy control participants (HCP).

RESULTS

Consistent with prior findings, taVNS increased effort invigoration for rewards in both groups during Session 1 (p = .040), particularly for less wanted rewards in HCP (pboot < 0.001). However, invigoration remained elevated in all participants, and no acute changes were observed in Session 2 (Δinvigoration = 3.3, p = .12). Crucially, throughout Session 1, we found taVNS-induced increases in effort invigoration (pboot = 0.008) and wanting (pboot = 0.010) in patients with MDD, with gains in wanting maintained across sessions (Δwanting = 0.06, p = .97).

CONCLUSIONS

Our study replicates the invigorating effects of taVNS in Session 1 and reveals its generalizability to depression. Furthermore, we expand upon previous research by showing taVNS-induced conditioning effects on invigoration and wanting within Session 1 in patients that were largely sustained. While enduring motivational improvements present challenges for crossover designs, they are highly desirable in interventions and warrant further follow-up research.

Effects of dopamine and opioid receptor antagonism on the neural processing of social and nonsocial rewards

Rewards are a broad category of stimuli inducing approach behavior to aid survival. Extensive evidence from animal research has shown that wanting (the motivation to pursue a reward) and liking (the pleasure associated with its consumption) are mostly regulated by dopaminergic and opioidergic activity in dedicated brain areas. However, less is known about the neuroanatomy of dopaminergic and opioidergic regulation of reward processing in humans, especially when considering different types of rewards (i.e., social and nonsocial). To fill this gap of knowledge, we combined dopaminergic and opioidergic antagonism (via amisulpride and naltrexone administration) with functional neuroimaging to investigate the neurochemical and neuroanatomical bases of wanting and liking of matched nonsocial (food) and social (interpersonal touch) rewards, using a randomized, between-subject, placebo-controlled, double-blind design. While no drug effect was observed at the behavioral level, brain activity was modulated by the administered compounds. In particular, opioid antagonism, compared to placebo, reduced activity in the medial orbitofrontal cortex during consumption of the most valued social and nonsocial rewards. Dopamine antagonism, however, had no clear effects on brain activity in response to reward anticipation. These findings provide insights into the neurobiology of human reward processing and suggest a similar opioidergic regulation of the neural responses to social and nonsocial reward consumption.

The validity of a therapeutic invigoration task in avolitional schizophrenia outpatients

BACKGROUND AND OBJECTIVES

Avolition is associated with much morbidity and functional impairment in schizophrenia patients. Vigor may be taken as, in part, the inverse of avolition, but it has not been investigated as a therapeutic pursuit before. To this end, a therapeutic invigoration task was developed drawing on cognitive-behavioral and guided imagery therapies. This study investigated the validity and reliability of a therapeutic invigoration task in avolitional residual phase schizophrenia outpatients.

METHODS

In a proof-of-concept quasi-experimental one-group sequentially repeated pretest/posttest study design, patients (n = 76) participated in a structured invigoration task that was repeated after 1 month (n = 70).

RESULTS

Patients' vigor during the preceding 7 days measured on the Vigor Assessment Scale increased highly significantly in anticipation of the subsequent 7 days on both occasions with respectively very large (Cohen's δ with Hedges' correction [δ] = 1.46) and large (δ = 1.04) effect sizes. The anticipated vigor after the first occasion was partially consummated during the subsequent month in that vigor during the 7 days preceding the second occasion was lower than participants had anticipated but still significantly higher than at baseline (p < 0.001; δ = 0.70). Repeating the task a month later, together with homework, had a cumulative effect as indicated by a very large effect size (δ = 1.61).

CONCLUSION

Results suggest that the invigoration task did what it was supposed do, and did so consistently, in patients with avolitional residual schizophrenia. These results warrant a subsequent randomized controlled trial to establish the efficacy of the invigoration task.

Inhibitory Pedunculopontine Neurons Gate Dopamine-Mediated Motor Actions of Unsigned Valence

BACKGROUND

The pedunculopontine nucleus (PPN) maintains a bidirectional connectivity with the basal ganglia that supports their shared roles in the selection and execution of motor actions. Previous studies identified a role for PPN neurons in goal-directed behavior, but the cellular substrates underlying this function have not been elucidated. We recently revealed the existence of a monosynaptic GABAergic input from the PPN that inhibits dopamine neurons of the substantia nigra. Activation of this pathway interferes with the execution of learned motor sequences when the actions are rewarded, even though the inhibition of dopamine neurons did not shift the value of the action, hence suggesting executive control over the gating of behavior.

OBJECTIVE

To test the attributes of the inhibition of dopamine neurons by the PPN in the context of goal-directed behavior regardless of whether the outcome is positively or negatively reinforced.

METHODS

We delivered optogenetic stimulation to PPN GABAergic axon terminals in the substantia nigra during a battery of behavioral tasks with positive and negative valence.

RESULTS

Inhibition of dopamine neurons by PPN optogenetic activation during an appetitive task impaired the initiation and overall execution of the behavioral sequence without affecting the consumption of reward. During an active avoidance task, the same activation impaired the ability of mice to avoid a foot shock, but their escape response was unaffected. In addition, responses to potential threats were significantly attenuated.

CONCLUSION

Our results show that PPN GABAergic neurons modulate learned, goal-directed behavior of unsigned valence without affecting overall motor behavior.

Dopamine facilitates the translation of physical exertion into assessments of effort

Our assessments of effort are critically shaped by experiences of exertion. However, it is unclear how the nervous system transforms physical exertion into assessments of effort. Availability of the neuromodulator dopamine influences features of motor performance and effort-based decision-making. To test dopamine's role in the translation of effortful exertion into assessments of effort, we had participants with Parkinson's disease, in dopamine depleted (OFF dopaminergic medication) and elevated (ON dopaminergic medication) states, exert levels of physical exertion and retrospectively assess how much effort they exerted. In a dopamine-depleted state, participants exhibited increased exertion variability and over-reported their levels of exertion, compared to the dopamine-supplemented state. Increased exertion variability was associated with less accurate effort assessment and dopamine had a protective influence on this effect, reducing the extent to which exertion variability corrupted assessments of effort. Our findings provide an account of dopamine's role in the translation of features of motor performance into judgments of effort, and a potential therapeutic target for the increased sense of effort observed across a range of neurologic and psychiatric conditions.

Pharmacological investigations of effort-based decision-making in humans: Naltrexone and nicotine

Many mental health disorders are characterized by an impaired ability, or willingness, to exert effort to obtain rewards. This impairment is modeled in effort-based decision tasks, and neuropharmacological studies implicate dopamine in this process. However, other transmitter systems such as opioidergic and cholinergic systems have received less attention. Here, in two separate studies we tested the acute effects of naltrexone and nicotine on effort-based decision-making in healthy adults. In Study 1, we compared naltrexone (50mg and 25mg) to placebo, and in Study 2, a pilot study, we compared nicotine (7mg) to placebo. In both studies, participants completed the Effort Expenditure for Rewards Task (EEfRT), which measured effort-based decision-making related to monetary rewards. Although subjects expended greater effort for larger reward magnitude and when there was a higher probability of receiving the reward, neither naltrexone nor nicotine affected willingness to exert effort for monetary rewards. Although the drugs produced significant and typical drug effects on measures of mood and behavior, they did not alter effort-based decision-making. This has implications both for the clinical use of these drugs, as well as for understanding the neuropharmacology of effort-related behavior.

Vigor, Effort-Related Aspects of Motivation and Anhedonia

In this chapter we provide an overview of the pharmacological and circuit mechanisms that determine the willingness to expend effort in pursuit of rewards. A particular focus will be on the role of the mesolimbic dopamine system, as well the contributing roles of limbic and cortical brains areas involved in the evaluation, selection, and invigoration of goal-directed actions. We begin with a review of preclinical studies, which have provided key insights into the brain systems that are necessary and sufficient for effort-based decision-making and have characterized novel compounds that enhance selection of high-effort activities. Next, we summarize translational studies identifying and expanding this circuitry in humans. Finally, we discuss the relevance of this work for understanding common motivational impairments as part of the broader anhedonia symptom domain associated with mental illness, and the identification of new treatment targets within this circuitry to improve motivation and effort-expenditure.

Dopaminergic medication increases motivation to exert cognitive control by reducing subjective effort costs in Parkinson's patients

Engaging in demanding mental activities requires the allocation of cognitive control, which can be effortful and aversive. Individuals thus tend to avoid exerting cognitive effort if less demanding behavioral options are available. Recent accounts propose a key role for dopamine in motivating behavior by increasing the sensitivity to rewards associated with effort exertion. Whether dopamine additionally plays a specific role in modulating the sensitivity to the costs of cognitive effort, even in the absence of any incentives, is much less clear. To address this question, we assessed cognitive effort avoidance in patients (n = 38) with Parkinson's disease, a condition characterized by loss of midbrain dopaminergic neurons, both ON and OFF dopaminergic medication and compared them to healthy controls (n = 24). Effort avoidance was assessed using the Demand Selection Task (DST), in which participants could freely choose between performing a high-demand or a low-demand version of a task-switching paradigm. Critically, participants were not offered any incentives to choose the more effortful option, nor for good performance. While healthy controls and patients OFF their dopaminergic medications consistently preferred the low-demand option, effort avoidance in patients ON dopaminergic medications was reduced compared to patients OFF, a difference which seems to lessen over trials. These differences in preference could not be explained by altered task-switching performance. Although patients ON were less accurate at detecting the different effort levels, as measured during instructed forced-choice blocks, their detection ability was not associated with effort avoidance, unlike in the healthy controls and the patients OFF. Our findings provide evidence that dopamine replacement in Parkinson's patients increases the willingness to engage in cognitively demanding behavior, and that this cannot be explained by possible effects of dopamine replacement on performance nor on the ability to detect effort demands. These results suggest that dopamine plays a role in reducing the sensitivity to effort costs that is independent of its role in enhancing the sensitivity to the benefits of effort exertion.

Towards a Unifying Account of Dopamine’s Role in Cost-Benefit Decision Making

Dopamine is thought to play a crucial role in cost-benefit decision making, but so far there is no consensus on the precise role of dopamine in decision making. Here, we review the literature on dopaminergic manipulations of cost-benefit decision making in humans and evaluate how well different theoretical accounts explain the existing body of evidence. Reduced D₂ stimulation tends to increase the willingness to bear delay and risk costs (i.e., wait for later rewards, take riskier options), while increased D₁ and D₂ receptor stimulation increases willingness to bear effort costs. We argue that the empirical findings can best be explained by combining the strengths of two theoretical accounts: in cost-benefit decision making, dopamine may play a dual role both in promoting the pursuit of psychologically close options (e.g., sooner and safer rewards) and in computing which costs are acceptable for a reward at stake. Moreover, we identify several limiting factors in the study designs of previous investigations that prevented a fuller understanding of dopamine's role in value-based choice. Together, the proposed theoretical framework and the methodological suggestions for future studies may bring us closer to a unifying account of dopamine in healthy and impaired cost-benefit decision making.

Thunderstruck: The ACDC model of flexible sequences and rhythms in recurrent neural circuits

Adaptive sequential behavior is a hallmark of human cognition. In particular, humans can learn to produce precise spatiotemporal sequences given a certain context. For instance, musicians can not only reproduce learned action sequences in a context-dependent manner, they can also quickly and flexibly reapply them in any desired tempo or rhythm without overwriting previous learning. Existing neural network models fail to account for these properties. We argue that this limitation emerges from the fact that sequence information (i.e., the position of the action) and timing (i.e., the moment of response execution) are typically stored in the same neural network weights. Here, we augment a biologically plausible recurrent neural network of cortical dynamics to include a basal ganglia-thalamic module which uses reinforcement learning to dynamically modulate action. This “associative cluster-dependent chain” (ACDC) model modularly stores sequence and timing information in distinct loci of the network. This feature increases computational power and allows ACDC to display a wide range of temporal properties (e.g., multiple sequences, temporal shifting, rescaling, and compositionality), while still accounting for several behavioral and neurophysiological empirical observations. Finally, we apply this ACDC network to show how it can learn the famous “Thunderstruck” song intro and then flexibly play it in a “bossa nova” rhythm without further training.

How do humans flexibly adapt action sequences? For instance, musicians can learn a song and quickly speed up or slow down the tempo, or even play the song following a completely different rhythm (e.g., a rock song using a bossa nova rhythm). In this work, we build a biologically plausible network of cortico-basal ganglia interactions that explains how this temporal flexibility may emerge in the brain. Crucially, our model factorizes sequence order and action timing, respectively represented in cortical and basal ganglia dynamics. This factorization allows full temporal flexibility, i.e. the timing of a learned action sequence can be recomposed without interfering with the order of the sequence. As such, our model is capable of learning asynchronous action sequences, and flexibly shift, rescale, and recompose them, while accounting for biological data.

Effects of average reward rate on vigor as a function of individual variation in striatal dopamine

RATIONALE

We constantly need to decide not only which actions to perform, but also how vigorously to perform them. In agreement with an earlier theoretical model, it has been shown that a significant portion of the variance in our action vigor can be explained by the average rate of rewards received for that action. Moreover, this invigorating effect of average reward rate was shown to vary with within-subject changes in dopamine, both in human individuals and experimental rodents.

OBJECTIVES

Here, we assessed whether individual differences in the effect of average reward rate on vigor are related to individual variation in a stable measure of striatal dopamine function in healthy, unmedicated participants.

METHODS

Forty-four participants performed a discrimination task to test the effect of average reward rate on response times to index vigor and completed an [¹⁸F]-DOPA PET scan to index striatal dopamine synthesis capacity.

RESULTS

We did not find an interaction between dopamine synthesis capacity and average reward rate across the entire group. However, a post hoc analysis revealed that participants with higher striatal dopamine synthesis capacity, particularly in the nucleus accumbens, exhibited a stronger invigorating effect of average reward rate among the 30 slowest participants.

CONCLUSIONS

Our findings provide converging evidence for a role of striatal dopamine in average reward rate signaling, thereby extending the current literature on the mechanistic link between average reward rate, vigor, and dopamine.

Opioid antagonism modulates wanting-related frontostriatal connectivity

Theoretical accounts distinguish between motivational (‘wanting’) and hedonic (‘liking’) dimensions of rewards. Previous animal and human research linked wanting and liking to anatomically and neurochemically distinct brain mechanisms, but it remains unknown how the different brain regions and neurotransmitter systems interact in processing distinct reward dimensions. Here, we assessed how pharmacological manipulations of opioid and dopamine receptor activation modulate the neural processing of wanting and liking in humans in a randomized, placebo-controlled, double-blind clinical trial. Reducing opioid receptor activation with naltrexone selectively reduced wanting of rewards, which on a neural level was reflected by stronger coupling between dorsolateral prefrontal cortex and the striatum under naltrexone compared with placebo. In contrast, reducing dopaminergic neurotransmission with amisulpride revealed no robust effects on behavior or neural activity. Our findings thus provide insights into how opioid receptors mediate neural connectivity related to specifically motivational, not hedonic, aspects of rewards.

Subthalamic stimulation breaks the balance between distal and axial signs in Parkinson's disease

In Parkinson’s disease (PD), the effects of both L_dopa and subthalamic deep brain stimulation (STN-DBS) are known to change cost-valuation. However, this was mostly studied through reward-effort task involving distal movements, while axial effort, less responsive to treatments, have been barely studied. Thus, our objective was to compare the influence of both L_dopa and STN-DBS on cost-valuation between two efforts modalities: vowel production (as an example of axial movement) and hand squeezing (as an example of distal movement). Twelve PD patients were recruited to participate in this study. The task consisted in deciding whether to accept or reject trials based on a reward-effort trade-off. Participants performed two blocks with hand squeezing, and two with vowel production, in the four treatment conditions (L_dopaOn/Off; STN-DBS On/Off). We found that STN-DBS changed the ratio difference between hand and phonation efforts. Vowel production effort was estimated easier to perform with STN-DBS alone, and harder when associated with L_dopa. The difference between hand and phonation efforts was correlated with quality of life in Off/Off and On L_dopa alone conditions, and with impulsive assessment On STN-DBS alone. We highlighted that STN-DBS could introduce an imbalance between the actual motor impairments and their subjective costs. With this finding, we also suggest paying particular attention to the different treatment effects that should be expected for axial and distal movement dysfunctions.

Reducing behavioral dimensions to study brain-environment interactions

Movement vigor provides a window on action valuation. But what is vigor, and how to measure it in the first place? Strikingly, many different co-varying vigor-related metrics can be found in the literature. I believe this is because vigor, just like the neural circuits that determine it, is an integrated, low-dimensional parameter. As such, it can only be roughly estimated.

Effort Mobilization and Healthy Aging

Healthy aging is in part dependent upon people's willingness and ability to mobilize the effort necessary to support behaviors that promote health and well-being. People may have the best information relating to health along with the best intentions to stay healthy (e.g., health-related goals), but positive outcomes will ultimately be dependent upon them actually investing the necessary effort toward using this information to achieve their goals. In addition, the influences on effort mobilization may vary as a function of physical, psychological, and social changes experienced by the individual across the life span. Building on the overall theme of this special issue, we explore the relationships between motivation, effort mobilization, and healthy aging. We begin by characterizing the relationship between motivation and effort, and identify the factors that influence effort mobilization. We then consider the factors associated specifically with aging that may influence effort mobilization (e.g., changes in cardiovascular and neural mechanisms) and, ultimately, the health and well-being of older adults. Finally, distinguishing between those influential factors that are modifiable versus intractable, we identify ways to structure situations and beliefs to optimize mobilization in support of healthy aging.

A mosaic of cost-benefit control over cortico-striatal circuitry

Dopamine contributes to cognitive control through well-established effects in both the striatum and cortex. Although earlier work suggests that dopamine affects cognitive control capacity, more recent work suggests that striatal dopamine may also impact on cognitive motivation. We consider the emerging perspective that striatal dopamine boosts control by making people more sensitive to the benefits versus the costs of cognitive effort, and we discuss how this sensitivity shapes competition between controlled and prepotent actions. We propose that dopamine signaling in distinct cortico-striatal subregions mediates different types of cost-benefit tradeoffs, and also discuss mechanisms for the local control of dopamine release, enabling selectivity among cortico-striatal circuits. In so doing, we show how this cost-benefit mosaic can reconcile seemingly conflicting findings about the impact of dopamine signaling on cognitive control.

Dose-response effects of d-amphetamine on effort-based decision-making and reinforcement learning

Effort-related decision-making and reward learning are both dopamine-dependent, but preclinical research suggests they depend on different dopamine signaling dynamics. Therefore, the same dose of a dopaminergic medication could have differential effects on effort for reward vs. reward learning. However, no study has tested how effort and reward learning respond to the same dopaminergic medication within subjects. The current study aimed to test the effect of therapeutic doses of d-amphetamine on effort for reward and reward learning in the same healthy volunteers. Participants (n = 30) completed the Effort Expenditure for Reward Task (EEfRT) measure of effort-related decision-making, and the Probabilistic Reward Task (PRT) measure of reward learning, under placebo and two doses of d-amphetamine (10 mg, and 20 mg). Secondarily, we examined whether the individual characteristics of baseline working memory and willingness to exert effort for reward moderated the effects of d-amphetamine. d-Amphetamine increased willingness to exert effort, particularly at low to intermediate expected values of reward. Computational modeling analyses suggested this was due to decreased effort discounting rather than probability discounting or decision consistency. Both baseline effort and working memory emerged as moderators of this effect, such that d-amphetamine increased effort more in individuals with lower working memory and lower baseline effort, also primarily at low to intermediate expected values of reward. In contrast, d-amphetamine had no significant effect on reward learning. These results have implications for treatment of neuropsychiatric disorders, which may be characterized by multiple underlying reward dysfunctions.

Methylphenidate boosts choices of mental labor over leisure depending on striatal dopamine synthesis capacity

The cognitive enhancing effects of methylphenidate are well established, but the mechanisms remain unclear. We recently demonstrated that methylphenidate boosts cognitive motivation by enhancing the weight on the benefits of a cognitive task in a manner that depended on striatal dopamine. Here, we considered the complementary hypothesis that methylphenidate might also act by changing the weight on the opportunity cost of a cognitive task, that is, the cost of foregoing alternative opportunity. To this end, 50 healthy participants (25 women) completed a novel cognitive effort-discounting task that required choices between task and leisure. They were tested on methylphenidate, placebo, as well as the selective D2-receptor agent sulpiride, the latter to strengthen inference about dopamine receptor selectivity of methylphenidate's effects. Furthermore, they also underwent an [18F]DOPA PET scan to quantify striatal dopamine synthesis capacity. Methylphenidate boosted choices of cognitive effort over leisure across the group, and this effect was greatest in participants with more striatal dopamine synthesis capacity. The effects of sulpiride did not reach significance. This study strengthens the motivational account of methylphenidate's effects on cognition, and suggests that methylphenidate reduces the cost of mental labor by increasing striatal dopamine.

Using pharmacological manipulations to study the role of dopamine in human reward functioning: A review of studies in healthy adults

Dopamine (DA) plays a key role in reward processing and is implicated in psychological disorders such as depression, substance use, and schizophrenia. The role of DA in reward processing is an area of highly active research. One approach to this question is drug challenge studies with drugs known to alter DA function. These studies provide good experimental control and can be performed in parallel in laboratory animals and humans. This review aimed to summarize results of studies using pharmacological manipulations of DA in healthy adults. 'Reward' is a complex process, so we separated 'phases' of reward, including anticipation, evaluation of cost and benefits of upcoming reward, execution of actions to obtain reward, pleasure in response to receiving a reward, and reward learning. Results indicated that i) DAergic drugs have different effects on different phases of reward; ii) the relationship between DA and reward functioning appears unlikely to be linear; iii) our ability to detect the effects of DAergic drugs varies depending on whether subjective, behavioral, imaging measures are used.

Pharmacological evidence for the implication of noradrenaline in effort

The trade-off between effort and reward is one of the main determinants of behavior, and its alteration is at the heart of major disorders such as depression or Parkinson's disease. Monoaminergic neuromodulators are thought to play a key role in this trade-off, but their relative contribution remains unclear. Rhesus monkeys (Macaca mulatta) performed a choice task requiring a trade-off between the volume of fluid reward and the amount of force to be exerted on a grip. In line with a causal role of noradrenaline in effort, decreasing noradrenaline levels with systemic clonidine injections (0.01 mg/kg) decreased exerted force and enhanced the weight of upcoming force on choices, without any effect on reward sensitivity. Using computational modeling, we showed that a single variable ("effort") could capture the amount of resources necessary for action and control both choices (as a variable for decision) and force production (as a driving force). Critically, the multiple effects of noradrenaline manipulation on behavior could be captured by a specific modulation of this single variable. Thus, our data strongly support noradrenaline's implication in effort processing.

Pupillary Dilations of Mice Performing a Vibrotactile Discrimination Task Reflect Task Engagement and Response Confidence

Pupillometry, the measure of pupil size and reactivity, has been widely used to assess cognitive processes. Changes in pupil size have been shown to correlate with various behavioral states, both externally and internally induced such as locomotion, arousal, cortical state, and decision-making processes. Besides, these pupillary responses have also been linked to the activity of neuromodulatory systems that modulate attention and perception such as the noradrenergic and cholinergic systems. Due to the extent of processes the pupil reflects, we aimed at further resolving pupillary responses in the context of behavioral state and task performance while recording pupillary transients of mice performing a vibrotactile two-alternative forced-choice task (2-AFC). We show that before the presentation of task-relevant information, pre-stimulus, pupil size differentiates between states of disengagement from task performance vs. engagement. Also, when subjects have to attend to task stimuli to attain a reward, post-stimulus, pupillary dilations exhibit a difference between correct and error responses with this difference reflecting an internal decision variable. We hypothesize that this internal decision variable relates to response confidence, the internal perception of the confidence the subject has in its choice. As opposed to this, we show that in a condition of passive performance, when the stimulus has no more task relevance due to reward being provided automatically, pupillary dilations reflect the occurrence of stimulation and reward provision but not decisional variables as under active performance. Our results provide evidence that in addition to reflecting attentiveness under task performance rather than arousal per se, pupil dilations also reflect the confidence of the subject in his ensuing response. This confidence coding is overlaid within a more pronounced pupil dilation that reflects post-decision components that are related to the response itself but not to the decision. We also provide evidence as to how different behavioral states, imposed by task demands, modulate what the pupil is reflecting, presumably showing what the underlying cognitive network is coding for.

The role of dopamine in dynamic effort-reward integration

When deciding to act, the neurotransmitter dopamine is implicated in a valuation of prospective effort and reward. However, its role in dynamic effort-reward integration during action, a process central to everyday behaviour, remains unclear. In a placebo-controlled, within-subject, study, we probed the impact of increasing brain dopamine levels (150 mg of levodopa) and blocking dopamine receptors (1.5 mg of haloperidol) in the context of a novel dynamic effort task in healthy human subjects. We show that modulating homoeostatic dopamine balance distinctly alters implicit and explicit effort allocation as a function of instantaneous reward. Pharmacologically boosting dopamine enhanced motor vigour, reflected in an implicit increase in effort allocation for high rewards. Conversely, pharmacological blockade of dopamine attenuated sensitivity to differences in reward context, reflected in reduced strategic effort discounting. These findings implicate dopamine in an integration of momentary physical experience and instantaneous reward, suggesting a key role of dopamine in acting to maximise reward on the fly.

Vagus nerve stimulation boosts the drive to work for rewards

Interoceptive feedback transmitted via the vagus nerve plays a vital role in motivation by tuning actions according to physiological needs. Whereas vagus nerve stimulation (VNS) reinforces actions in animals, motivational effects elicited by VNS in humans are still largely elusive. Here, we applied non-invasive transcutaneous auricular VNS (taVNS) on the left or right ear while participants exerted effort to earn rewards using a randomized cross-over design (vs. sham). In line with preclinical studies, acute taVNS enhances invigoration of effort, and stimulation on the left side primarily facilitates invigoration for food rewards. In contrast, we do not find conclusive evidence that acute taVNS affects effort maintenance or wanting ratings. Collectively, our results suggest that taVNS enhances reward-seeking by boosting invigoration, not effort maintenance and that the stimulation side affects generalization beyond food reward. Thus, taVNS may enhance the pursuit of prospective rewards which may pave avenues to treat motivational deficiencies.

The vagus nerve transmits signals between the gut and the brain thereby tuning motivated behavior to physiological needs. Here, the authors show that acute non-invasive stimulation of the vagus nerve via the ear enhances the invigoration of effort for rewards.

Distinct signals in medial and lateral VTA dopamine neurons modulate fear extinction at different times

Dopamine (DA) neurons are thought to encode reward prediction error (RPE), in addition to other signals, such as salience. While RPE is known to support learning, the role of salience in learning remains less clear. To address this, we recorded and manipulated VTA DA neurons in mice during fear extinction. We applied deep learning to classify mouse freezing behavior, eliminating the need for human scoring. Our fiber photometry recordings showed DA neurons in medial and lateral VTA have distinct activity profiles during fear extinction: medial VTA activity more closely reflected RPE, while lateral VTA activity more closely reflected a salience-like signal. Optogenetic inhibition of DA neurons in either region slowed fear extinction, with the relevant time period for inhibition differing across regions. Our results indicate salience-like signals can have similar downstream consequences to RPE-like signals, although with different temporal dependencies.

Dissociating self-generated volition from externally-generated motivation

Insight into motivational processes may be gained by examining measures of willingness to exert effort for rewards, which have been linked to neuropsychiatric symptoms of anhedonia and apathy. However, while much work has focused on the development of models of motivation based on classic tasks of externally-generated levels of effort for reward, there has been less focus on the question of self-generated motivation or volition. We developed a task that aims to capture separate measures of self-generated and externally-generated motivation, with two task variants for physical and cognitive effort, and sought to test and validate this measure in two populations of healthy volunteers (N = 27 and N = 28). Similar to previous reports, a sigmoid function represented a better overall fit to the effort-reward data than a linear or Weibull model. Individual sigmoid function shapes were governed by two free parameters: bias (the amount of reward needed for effort initiation) and reward insensitivity (the amount of increase in reward needed to accelerate effort expenditure). For both physical and cognitive effort, bias was higher in the self-generated condition, indicating reduced self-generated volitional effort initiation, compared to externally-generated effort initiation, across effort domains. Bias against initial effort initiation in the self-generated condition was related to a specific dimensional measure of anticipatory anhedonia. For physical effort only, reward insensitivity was also higher in the self-generated condition compared to the externally-generated motivation condition, indicating lower self-generated effort acceleration. This work provides a novel objective measure of self-generated motivation that may provide insights into mechanisms of anhedonia and related symptoms.

Noradrenergic But Not Dopaminergic Neurons Signal Task State Changes and Predict Reengagement After a Failure

The two catecholamines, noradrenaline and dopamine, have been shown to play comparable roles in behavior. Both noradrenergic and dopaminergic neurons respond to cues predicting reward availability and novelty. However, even though both are thought to be involved in motivating actions, their roles in motivation have seldom been directly compared. We therefore examined the activity of putative noradrenergic neurons in the locus coeruleus and putative midbrain dopaminergic neurons in monkeys cued to perform effortful actions for rewards. The activity in both regions correlated with engagement with a presented option. By contrast, only noradrenaline neurons were also (i) predictive of engagement in a subsequent trial following a failure to engage and (ii) more strongly activated in nonrepeated trials, when cues indicated a new task condition. This suggests that while both catecholaminergic neurons are involved in promoting action, noradrenergic neurons are sensitive to task state changes, and their influence on behavior extends beyond the immediately rewarded action.

Motor cortical thickness is related to effort-based decision-making in humans

Although motor cortex is integral in driving physical exertion, how its inherent properties influence decisions to exert is unknown. In this study, we examined how anatomical properties of motor cortex are related to participants' subjective valuations of effort and their decisions to exert effort. We used computational modeling to characterize participants' subjective valuation of physical effort during an effort-based decision-making task in which they made choices about exerting different levels of hand-grip exertion. We also acquired structural MRI data from these participants and extracted anatomical measures of each individual's hand knob, the region of motor cortex recruited during hand-grip exertion. We found that individual participants' cortical thickness of hand knob was associated with their effort-based decisions regarding hand exertion. These data provide evidence that the anatomy of an individual's motor cortex is an important factor in decisions to engage in physical activity.NEW & NOTEWORTHY How effortful a task feels is an integral aspect of human decision-making that influences choices to engage in physical activity. We show that properties of motor cortex (the brain region responsible for physical exertion) are related to assessments of effort and decisions to exert. These findings provide a link between the anatomical properties of motor cortex and the cognitive function of effort-based choice.

Interplay between midbrain and dorsal anterior cingulate regions arbitrates lingering reward effects on memory encoding

Rewarding events enhance memory encoding via dopaminergic influences on hippocampal plasticity. Phasic dopamine release depends on immediate reward magnitude, but presumably also on tonic dopamine levels, which may vary as a function of the average accumulation of reward over time. Using model-based fMRI in combination with a novel associative memory task, we show that immediate reward magnitude exerts a monotonically increasing influence on the nucleus accumbens, ventral tegmental area (VTA), and hippocampal activity during encoding, and enhances memory. By contrast, average reward levels modulate feedback-related responses in the VTA and hippocampus in a non-linear (inverted U-shape) fashion, with similar effects on memory performance. Additionally, the dorsal anterior cingulate cortex (dACC) monotonically tracks average reward levels, while VTA-dACC functional connectivity is non-linearly modulated (inverted U-shape) by average reward. We propose that the dACC computes the net behavioral impact of average reward and relays this information to memory circuitry via the VTA.

Dopaminergic D1 Receptor Stimulation Affects Effort and Risk Preferences

BACKGROUND

Activation of D₁ receptors has been related to successful goal-directed behavior, but it remains unclear whether D₁ receptor activation causally tips the balance of weighing costs and benefits in humans. Here, we tested the impact of pharmacologically stimulated D₁ receptors on sensitivity to risk, delay, and effort costs in economic choice and investigated whether D₁ receptor stimulation would bias preferences toward options with increased costs in a cost-specific manner.

METHODS

In a randomized, double-blind, placebo-controlled, parallel-group phase 1 study, 120 healthy young volunteers received either placebo or 1 of 3 doses (6 mg, 15 mg, or 30 mg) of a novel, selective D₁ agonist (PF-06412562). After drug administration, participants performed decision tasks measuring their preferences for risky, delayed, and effortful outcomes.

RESULTS

Higher doses of the D₁ agonist increased the willingness to exert physical effort for reward as well as reduced the preference for risky outcomes. We observed no effects on preferences for delayed rewards.

CONCLUSIONS

The current results provide evidence that D₁ receptor stimulation causally affects core aspects of cost-benefit decision making in humans.

Why not try harder? Computational approach to motivation deficits in neuro-psychiatric diseases

Motivation deficits, such as apathy, are pervasive in both neurological and psychiatric diseases. Even when they are not the core symptom, they reduce quality of life, compromise functional outcome and increase the burden for caregivers. They are currently assessed with clinical scales that do not give any mechanistic insight susceptible to guide therapeutic intervention. Here, we present another approach that consists of phenotyping the behaviour of patients in motivation tests, using computational models. These formal models impose a precise and operational definition of motivation that is embedded in decision theory. Motivation can be defined as the function that orients and activates the behaviour according to two attributes: a content (the goal) and a quantity (the goal value). Decision theory offers a way to quantify motivation, as the cost that patients would accept to endure in order to get the benefit of achieving their goal. We then review basic and clinical studies that have investigated the trade-off between the expected cost entailed by potential actions and the expected benefit associated with potential rewards. These studies have shown that the trade-off between effort and reward involves specific cortical, subcortical and neuromodulatory systems, such that it may be shifted in particular clinical conditions, and reinstated by appropriate treatments. Finally, we emphasize the promises of computational phenotyping for clinical purposes. Ideally, there would be a one-to-one mapping between specific neural components and distinct computational variables and processes of the decision model. Thus, fitting computational models to patients' behaviour would allow inferring of the dysfunctional mechanism in both cognitive terms (e.g. hyposensitivity to reward) and neural terms (e.g. lack of dopamine). This computational approach may therefore not only give insight into the motivation deficit but also help personalize treatment.

A Pavlovian account for paradoxical effects of motivation on controlling response vigour

In high stakes situations, people sometimes choke under pressure, performing below their abilities. Here, we suggest a novel mechanism to account for this paradoxical effect of motivation: the automatic adjustment of action vigour to potential reward. Although adaptive on average, this mechanism may impede fine motor control. Such detrimental effect was observed in three studies (n = 74 in total), using behavioural tasks where payoff depended on the precision of handgrip squeezing or golf putting. Participants produced more force for higher incentives, which aggravated their systematic overshooting of low-force targets. This reward bias was specific to action vigour, as reward did not alter action timing, direction or variability across trials. Although participants could report their reward bias, they somehow failed to limit their produced force. Such an automatic link between incentive and force level might correspond to a Pavlovian response that is counterproductive when action vigour is not instrumental for maximizing reward.

Effects of subthalamic nucleus stimulation and levodopa on decision-making in Parkinson's disease

BACKGROUND

Parkinson's disease (PD) is frequently associated with behavioral disorders, particularly within the spectrum of motivated behaviors such as apathy or impulsivity. Both pharmacological and neurosurgical treatments have an impact on these impairments. However, there still is controversy as to whether subthalamic nucleus deep brain stimulation (STN-DBS) can cause or reduce impulsive behaviors.

OBJECTIVES

We aimed to identify the influence of functional surgery on decision-making processes in PD.

METHODS

We studied 13 PD patients and 13 healthy controls. The experimental task involved squeezing a dynamometer with variable force to obtain rewards of various values under four conditions: without treatment, with l-dopa or subthalamic stimulation alone, and with both l-dopa and subthalamic stimulation. Statistical analyses consisted of generalized linear mixed models including treatment condition, reward value, level of effort, and their interactions. We analyzed acceptance rate (the percentage of accepted trials), decision time, and force applied.

RESULTS

Comparatively to controls, patients without treatment exhibited lower acceptance rate and force applied. Patients under l-dopa alone did not exhibit increased acceptance rate. With subthalamic stimulation, either with or without added l-dopa, all measures were improved so that patients' behaviors were undistinguishable from healthy controls'.

CONCLUSIONS

Our study shows that l-dopa administration does not fully restore cost-benefit decision-making processes, whereas STN-DBS fully normalizes patients' behaviors. These findings suggest that dopamine is partly involved in cost-benefit valuation, and that STN-DBS can have a beneficial effect on motivated behaviors in PD and may improve certain forms of impulsive behaviors. © 2019 International Parkinson and Movement Disorder Society.

An information-theoretic perspective on the costs of cognition

In statistics and machine learning, model accuracy is traded off with complexity, which can be viewed as the amount of information extracted from the data. Here, we discuss how cognitive costs can be expressed in terms of similar information costs, i.e. as a function of the amount of information required to update a person's prior knowledge (or internal model) to effectively solve a task. We then examine the theoretical consequences that ensue from this assumption. This framework naturally explains why some tasks - for example, unfamiliar or dual tasks - are costly and permits to quantify these costs using information-theoretic measures. Finally, we discuss brain implementation of this principle and show that subjective cognitive costs can originate either from local or global capacity limitations on information processing or from increased rate of metabolic alterations. These views shed light on the potential adaptive value of cost-avoidance mechanisms.

Neuroscience of apathy and anhedonia: a transdiagnostic approach

Apathy and anhedonia are common syndromes of motivation that are associated with a wide range of brain disorders and have no established therapies. Research using animal models suggests that a useful framework for understanding motivated behaviour lies in effort-based decision making for reward. The neurobiological mechanisms underpinning such decisions have now begun to be determined in individuals with apathy or anhedonia, providing an important foundation for developing new treatments. The findings suggest that there might be some shared mechanisms between both syndromes. A transdiagnostic approach that cuts across traditional disease boundaries provides a potentially useful means for understanding these conditions.

Brain Stimulation Over the Frontopolar Cortex Enhances Motivation to Exert Effort for Reward

BACKGROUND

Loss of motivation is a characteristic feature of several psychiatric and neurological disorders. However, the neural mechanisms underlying human motivation are far from being understood. Here, we investigate the role that the frontopolar cortex (FPC) plays in motivating cognitive and physical effort exertion by computing subjective effort equivalents.

METHODS

We manipulated neural processing with transcranial direct current stimulation targeting the FPC while 141 healthy participants decided whether or not to engage in cognitive or physical effort to obtain rewards.

RESULTS

We found that brain stimulation targeting the FPC increased the amount of both types of effort participants were willing to exert for rewards.

CONCLUSIONS

Our findings provide important insights into the neural mechanisms involved in motivating effortful behavior. Moreover, they suggest that considering the motivation-related activity of the FPC could facilitate the development of treatments for the loss of motivation commonly seen in psychiatric and other neurological disorders.

Dopamine, Effort-Based Choice, and Behavioral Economics: Basic and Translational Research

Operant behavior is not only regulated by factors related to the quality or quantity of reinforcement, but also by the work requirements inherent in performing instrumental actions. Moreover, organisms often make effort-related decisions involving economic choices such as cost/benefit analyses. Effort-based decision making is studied using behavioral procedures that offer choices between high-effort options leading to relatively preferred reinforcers vs. low effort/low reward choices. Several neural systems, including the mesolimbic dopamine (DA) system and other brain circuits, are involved in regulating effort-related aspects of motivation. Considerable evidence indicates that mesolimbic DA transmission exerts a bi-directional control over exertion of effort on instrumental behavior tasks. Interference with DA transmission produces a low-effort bias in animals tested on effort-based choice tasks, while increasing DA transmission with drugs such as DA transport blockers tends to enhance selection of high-effort options. The results from these pharmacology studies are corroborated by the findings from recent articles using optogenetic, chemogenetic and physiological techniques. In addition to providing important information about the neural regulation of motivated behavior, effort-based choice tasks are useful for developing animal models of some of the motivational symptoms that are seen in people with various psychiatric and neurological disorders (e.g., depression, schizophrenia, Parkinson’s disease). Studies of effort-based decision making may ultimately contribute to the development of novel drug treatments for motivational dysfunction.

Dopamine and Proximity in Motivation and Cognitive Control

Cognitive control - the ability to override a salient or prepotent action to execute a more deliberate one - is required for flexible, goal-directed behavior, and yet it is subjectively costly: decision-makers avoid allocating control resources, even when doing so affords more valuable outcomes. Dopamine likely offsets effort costs just as it does for physical effort. And yet, dopamine can also promote impulsive action, undermining control. We propose a novel hypothesis that reconciles opposing effects of dopamine on cognitive control: during action selection, striatal dopamine biases benefits relative to costs, but does so preferentially for "proximal" motor and cognitive actions. Considering the nature of instrumental affordances and their dynamics during action selection facilitates a parsimonious interpretation and conserved corticostriatal mechanisms across physical and cognitive domains.

Dual contributions of noradrenaline to behavioural flexibility and motivation

INTRODUCTION

While several theories have highlighted the importance of the noradrenergic system for behavioral flexibility, a number of recent studies have also shown a role for noradrenaline in motivation, particularly in effort processing. Here, we designed a novel sequential cost/benefit decision task to test the causal influence of noradrenaline on these two functions in rhesus monkeys.

METHODS

We manipulated noradrenaline using clonidine, an alpha-2 noradrenergic receptor agonist, which reduces central noradrenaline levels and examined how this manipulation influenced performance on the task.

RESULTS

Clonidine had two specific and distinct effects: first, it decreased choice variability, without affecting the cost/benefit trade-off; and second, it reduced force production, without modulating the willingness to work.

CONCLUSIONS

Together, these results support an overarching role for noradrenaline in facing challenging situations in two complementary ways: by modulating behavioral volatility, which would facilitate adaptation depending on the lability of the environment, and by modulating the mobilization of resources to face immediate challenges.