What Is the Relationship between Dopamine and Effort?

Pupil dilation reflects effortful action invigoration in overcoming aversive Pavlovian biases

"Pavlovian" or "motivational" biases describe the phenomenon that the valence of prospective outcomes modulates action invigoration: Reward prospect invigorates action, whereas punishment prospect suppresses it. The adaptive role of these biases in decision-making is still unclear. One idea is that they constitute a fast-and-frugal decision strategy in situations characterized by high arousal, e.g., in presence of a predator, which demand a quick response. In this pre-registered study (N = 35), we tested whether such a situation-induced via subliminally presented angry versus neutral faces-leads to increased reliance on Pavlovian biases. We measured trial-by-trial arousal by tracking pupil diameter while participants performed an orthogonalized Motivational Go/NoGo Task. Pavlovian biases were present in responses, reaction times, and even gaze, with lower gaze dispersion under aversive cues reflecting "freezing of gaze." The subliminally presented faces did not affect responses, reaction times, or pupil diameter, suggesting that the arousal manipulation was ineffective. However, pupil dilations reflected facets of bias suppression, specifically the physical (but not cognitive) effort needed to overcome aversive inhibition: Particularly strong and sustained dilations occurred when participants managed to perform Go responses to aversive cues. Conversely, no such dilations occurred when they managed to inhibit responses to Win cues. These results suggest that pupil diameter does not reflect response conflict per se nor the inhibition of prepotent responses, but specifically effortful action invigoration as needed to overcome aversive inhibition. We discuss our results in the context of the "value of work" theory of striatal dopamine.

Mild exogenous inflammation blunts neural signatures of bounded evidence accumulation and reward prediction error processing in healthy male participants

BACKGROUND

Altered neural haemodynamic activity during decision making and learning has been linked to the effects of inflammation on mood and motivated behaviours. So far, it has been reported that blunted mesolimbic dopamine reward signals are associated with inflammation-induced anhedonia and apathy. Nonetheless, it is still unclear whether inflammation impacts neural activity underpinning decision dynamics. The process of decision making involves integration of noisy evidence from the environment until a critical threshold of evidence is reached. There is growing empirical evidence that such process, which is usually referred to as bounded accumulation of decision evidence, is affected in the context of mental illness.

METHODS

In a randomised, placebo-controlled, crossover study, 19 healthy male participants were allocated to placebo and typhoid vaccination. Three to four hours post-injection, participants performed a probabilistic reversal-learning task during functional magnetic resonance imaging. To capture the hidden neurocognitive operations underpinning decision-making, we devised a hybrid sequential sampling and reinforcement learning computational model. We conducted whole brain analyses informed by the modelling results to investigate the effects of inflammation on the efficiency of decision dynamics and reward learning.

RESULTS

We found that during the decision phase of the task, typhoid vaccination attenuated neural signatures of bounded evidence accumulation in the dorsomedial prefrontal cortex, only for decisions requiring short integration time. Consistent with prior work, we showed that, in the outcome phase, mild acute inflammation blunted the reward prediction error in the bilateral ventral striatum and amygdala.

CONCLUSIONS

Our study extends current insights into the effects of inflammation on the neural mechanisms of decision making and shows that exogenous inflammation alters neural activity indexing efficiency of evidence integration, as a function of choice discriminability. Moreover, we replicate previous findings that inflammation blunts striatal reward prediction error signals.

Brain stimulation over the left DLPFC enhances motivation for effortful rewards in patients with major depressive disorder

BACKGROUND

Amotivation is a typical feature in major depressive disorder (MDD), which produces reduced willingness to exert effort. The dorsolateral prefrontal cortex (DLPFC) is a crucial structure in goal-directed actions and therefore is a potential target in modulating effortful motivation. However, it remains unclear whether the intervention is effective for patients with MDD.

METHODS

We employed transcranial magnetic stimulation (TMS), computational modelling and event-related potentials (ERPs) to reveal the causal relationship between the left DLPFC and motivation for effortful rewards in MDD. Fifty patients underwent both active and sham TMS sessions, each followed by performing an Effort-Expenditure for Rewards Task, during which participants chose and implemented between low-effort/low-reward and high-effort/high-reward options.

RESULTS

The patients showed increased willingness to exert effort for rewards during the DLPFC facilitated session, compared with the sham session. They also had a trend in larger P3 amplitude for motivated attention toward chosen options, larger CNV during preparing for effort exertion, and larger SPN during anticipating a high reward. Besides, while behavior indexes for effortful choices were negatively related to depression severity in the sham session, this correlation was weakened in the active stimulation session.

CONCLUSIONS

These findings provide behavioral, computational, and neural evidence for the left DLPFC on effortful motivation for rewards. Facilitated DLPFC improves motor preparation and value anticipation after making decisions especially for highly effortful rewards in MDD. Facilitated DLPFC also has a potential function in enhancing motivated attention during cost-benefit trade-off. This neuromodulation effect provides a potential treatment for improving motivation in clinics.

General mechanisms of task engagement in the primate frontal cortex

Staying engaged is necessary to maintain goal-directed behaviors. Despite this, engagement exhibits continuous, intrinsic fluctuations. Even in experimental settings, animals, unlike most humans, repeatedly and spontaneously move between periods of complete task engagement and disengagement. We, therefore, looked at behavior in male macaques (macaca mulatta) in four tasks while recording fMRI signals. We identified consistent autocorrelation in task disengagement. This made it possible to build models capturing task-independent engagement. We identified task general patterns of neural activity linked to impending sudden task disengagement in mid-cingulate gyrus. By contrast, activity centered in perigenual anterior cingulate cortex (pgACC) was associated with maintenance of performance across tasks. Importantly, we carefully controlled for task-specific factors such as the reward history and other motivational effects, such as response vigor, in our analyses. Moreover, we showed pgACC activity had a causal link to task engagement: transcranial ultrasound stimulation of pgACC changed task engagement patterns.

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.

Effort-based decision making and motivational deficits in stroke patients

Motivational deficits in patients recovering from stroke are common and can reduce active participation in rehabilitation and thereby impede functional recovery. We investigated whether stroke patients with clinically reduced drive, initiation, and endurance during functional rehabilitative training (n = 30) display systematic alterations in effort-based decision making compared to age, sex, and severity-matched stroke patients (n = 30) whose drive appeared unaffected. Notably, the two groups did not differ in self-reported ratings of apathy and depression. However, on an effort-based decision-making task, stroke patients with clinically apparent drive impairment showed intact willingness to accept effort for reward, but were more likely to fail to execute the required effort compared to patients without apparent drive impairments. In other words, the decision behavioural assessment revealed that stroke patients that displayed reduced drive, initiation, and endurance during inpatient neurorehabilitation failed to persist in goal-directed effort production, even over very short periods. These findings indicate that reduced drive during rehabilitative therapy in post-stroke patients is not due to a diminished motivation to invest physical effort, but instead is related to a reduced persistence with effortful behaviour.

Striatal dopamine integrates cost, benefit, and motivation

Striatal dopamine (DA) release has long been linked to reward processing, but it remains controversial whether DA release reflects costs or benefits and how these signals vary with motivation. Here, we measure DA release in the nucleus accumbens (NAc) and dorsolateral striatum (DLS) while independently varying costs and benefits and apply behavioral economic principles to determine a mouse's level of motivation. We reveal that DA release in both structures incorporates both reward magnitude and sunk cost. Surprisingly, motivation was inversely correlated with reward-evoked DA release. Furthermore, optogenetically evoked DA release was also heavily dependent on sunk cost. Our results reconcile previous disparate findings by demonstrating that striatal DA release simultaneously encodes cost, benefit, and motivation but in distinct manners over different timescales. Future work will be necessary to determine whether the reduction in phasic DA release in highly motivated animals is due to changes in tonic DA levels.

What Is Mental Effort: A Clinical Perspective

Although mental effort is a frequently used term, it is poorly defined and understood. Consequently, its usage is frequently loose and potentially misleading. In neuroscience research, the term is used to mean both the cognitive work that is done to meet task demands and the subjective experience of performing that work. We argue that conflating these two meanings hampers progress in understanding cognitive impairments in neuropsychiatric conditions because cognitive work and the subjective experience of it have distinct underlying mechanisms. We suggest that the most coherent and clinically useful perspective on mental effort is that it is a subjective experience. This makes a clear distinction between cognitive impairments that arise from changes in the cognitive apparatus, as in dementia and brain injury, and those that arise from subjective difficulties in carrying out the cognitive work, as in attention-deficit/hyperactivity disorder, depression, and other motivational disorders. We review recent advances in neuroscience research that suggests that the experience of effort has emerged to control task switches so as to minimize costs relative to benefits. We consider how these advances can contribute to our understanding of the experience of increased effort perception in clinical populations. This more specific framing of mental effort will offer a deeper understanding of the mechanisms of cognitive impairments in differing clinical groups and will ultimately facilitate better therapeutic interventions.

Atomoxetine and reward size equally improve task engagement and perceptual decisions but differently affect movement execution

Our ability to engage and perform daily activities relies on balancing the associated benefits and costs. Rewards, as benefits, act as powerful motivators that help us stay focused for longer durations. The noradrenergic (NA) system is thought to play a significant role in optimizing our performance. Yet, the interplay between reward and the NA system in shaping performance remains unclear, particularly when actions are driven by external incentives (reward). To explore this interaction, we tested four female rhesus monkeys performing a sustained Go/NoGo task under two reward sizes (low/high) and three pharmacological conditions (saline and two doses of atomoxetine, a NA reuptake inhibitor: ATX-0.5 mg/kg and ATX-1 mg/kg). We found that increasing either reward or NA levels equally enhanced the animal's engagement in the task compared to low reward saline; the animals also responded faster and more consistently under these circumstances. Notably, we identified differences between reward size and ATX. When combined with ATX, high reward further reduced the occurrence of false alarms (i.e., incorrect go trials on distractors), implying that it helped further suppress impulsive responses. In addition, ATX (but not reward size) consistently increased movement duration dose-dependently, while high reward did not affect movement duration but decreased its variability. We conclude that noradrenaline and reward modulate performance, but their effects are not identical, suggesting differential underlying mechanisms. Reward might energize/invigorate decisions and action, while ATX might help regulate energy expenditure, depending on the context, through the NA system.

Hyperdopaminergia in rats is associated with reverse effort-cost dependent performance

BACKGROUND

Dopamine is implicated in the effort-based control of motivational processes; however, whether tonic dopamine regulates the effort-cost impact on motivation, is still debated.

AIMS

The rats lacking the dopamine transporter (DAT), which have dramatically increased levels of the synaptic dopamine, were used in the present study to elucidate the role of the synaptic dopamine in motivational processes.

METHODS

To study the reward-related processes, the progressive ratio 3 (PR3) operant schedule of food reinforcement (the ratio increases by 3 after each earned reinforcer) was performed in adult male rats (DAT knockouts (DAT-KO), heterozygotes (DAT-HT) and wild-types (DAT-WT)).

RESULTS

During the PR3 session, the response rate of DAT-KO rats was gradually increased following the augmented required number of responses. In contrast, the local response rate of DAT-WT and DAT-HT decreased. d-Amphetamine sulfate salt (3 mg/kg, i.p.) altered the local response rate dynamics in DAT-WT, which became similar to that of DAT-KO. Interestingly, the reduction in response rate at low effort demands was associated with decreased rate of entries into the magazine tray in DAT-WT rats treated with amphetamine (3 mg/kg) but not in DAT-KO rats.

CONCLUSIONS

Our results suggest that the elevated tonic synaptic dopamine can strongly affect motivation/effort-cost relation in rodents.

Dual credit assignment processes underlie dopamine signals in a complex spatial environment

Animals frequently make decisions based on expectations of future reward ("values"). Values are updated by ongoing experience: places and choices that result in reward are assigned greater value. Yet, the specific algorithms used by the brain for such credit assignment remain unclear. We monitored accumbens dopamine as rats foraged for rewards in a complex, changing environment. We observed brief dopamine pulses both at reward receipt (scaling with prediction error) and at novel path opportunities. Dopamine also ramped up as rats ran toward reward ports, in proportion to the value at each location. By examining the evolution of these dopamine place-value signals, we found evidence for two distinct update processes: progressive propagation of value along taken paths, as in temporal difference learning, and inference of value throughout the maze, using internal models. Our results demonstrate that within rich, naturalistic environments dopamine conveys place values that are updated via multiple, complementary learning algorithms.

Amphetamines Improve the Motivation to Invest Effort in Attention-Deficit/Hyperactivity Disorder

Prevailing frameworks propose that a key feature of attention-deficit/hyperactivity disorder (ADHD) is lower motivation. An important component of motivation is the willingness to engage in cognitively or physically effortful behavior. However, the degree to which effort sensitivity is impaired in ADHD has rarely been tested, and the efficacy of stimulant medication in ameliorating any such impairments is unclear. Here, we tested 20 individuals with ADHD (11 males, 9 females) who were managed with amphetamine-based medication (dexamfetamine, lisdexamfetamine), and 24 controls (8 males, 16 females). Individuals with ADHD were tested over two counterbalanced sessions, ON and OFF their usual amphetamine-based medication. In each session, participants performed an effort-based decision-making task, in which they were required to choose how much cognitive or physical effort they were willing to engage in return for reward. Our results revealed three main findings. First, individuals with ADHD had lower motivation relative to controls to invest effort in both the cognitive and physical domains. Second, amphetamine increased motivation uniformly across both domains. Finally, the net effect of amphetamine treatment was to mostly restore motivation across both domains of effort relative to healthy controls. These data provide clear evidence for a heightened sensitivity to both cognitive and physical effort in ADHD, and reveal the efficacy of amphetamine-based drugs in restoring effort sensitivity to levels similar to controls. These findings confirm the existence of reduced motivational drive in ADHD, and more broadly provide direct causal evidence for a domain-general role of catecholamines in motivating effortful behavior.SIGNIFICANCE STATEMENT A core feature of attention-deficit/hyperactivity disorder (ADHD) is thought to be a heightened aversion to effort. Surprisingly, however, the degree to which effort sensitivity is impaired in ADHD has rarely been tested. More broadly, the relative efficacy of catecholamines in motivating the investment of cognitive and physical effort is unclear. We tested 20 individuals with ADHD ON and OFF amphetamines, and compared their behavior on an effort-based decision-making task to 24 controls. When tested OFF medication, the ADHD group was less cognitively and physically motivated than controls. However, amphetamines led to a comparable increase in motivation across both domains. This demonstrates the efficacy of catecholamines in facilitating domain-general effort, and highlights the broader potential of such drugs to treat disorders of motivation.

Reframing dopamine: A controlled controller at the limbic-motor interface

Pavlovian influences notoriously interfere with operant behaviour. Evidence suggests this interference sometimes coincides with the release of the neuromodulator dopamine in the nucleus accumbens. Suppressing such interference is one of the targets of cognitive control. Here, using the examples of active avoidance and omission behaviour, we examine the possibility that direct manipulation of the dopamine signal is an instrument of control itself. In particular, when instrumental and Pavlovian influences come into conflict, dopamine levels might be affected by the controlled deployment of a reframing mechanism that recasts the prospect of possible punishment as an opportunity to approach safety, and the prospect of future reward in terms of a possible loss of that reward. We operationalize this reframing mechanism and fit the resulting model to rodent behaviour from two paradigmatic experiments in which accumbens dopamine release was also measured. We show that in addition to matching animals' behaviour, the model predicts dopamine transients that capture some key features of observed dopamine release at the time of discriminative cues, supporting the idea that modulation of this neuromodulator is amongst the repertoire of cognitive control strategies.

Stress to inflammation and anhedonia: Mechanistic insights from preclinical and clinical models

Anhedonia, as evidenced by impaired pleasurable response to reward, reduced reward motivation, and/or deficits in reward-related learning, is a common feature of depression. Such deficits in reward processing are also an important clinical target as a risk factor for depression onset. Unfortunately, reward-related deficits remain difficult to treat. To address this gap and inform the development of effective prevention and treatment strategies, it is critical to understand the mechanisms that drive impairments in reward function. Stress-induced inflammation is a plausible mechanism of reward deficits. The purpose of this paper is to review evidence for two components of this psychobiological pathway: 1) the effects of stress on reward function; and 2) the effects of inflammation on reward function. Within these two areas, we draw upon preclinical and clinical models, distinguish between acute and chronic effects of stress and inflammation, and address specific domains of reward dysregulation. By addressing these contextual factors, the review reveals a nuanced literature which might be targeted for additional scientific inquiry to inform the development of precise interventions.

EngAge - A metacognitive intervention to supplement working memory training: A feasibility study in older adults

Working Memory (WM) training has shown promise in supporting cognitive functioning in older adult populations, but effects that generalize beyond the trained task have been inconsistent. Targeting cognitive processes in isolation might be a limiting factor given that metacognitive and motivational factors have been shown to impact older adults' engagement with challenging cognitive activities, such as WM training. The current feasibility study implemented a novel metacognitive intervention in conjunction with WM training in older adults and examined its potential amplifying short- and long-term effects on cognitive and self-report outcomes as compared to WM or active control training alone. One-hundred and nineteen older adults completed a cognitive training over the course of 20 sessions at home. The cognitive training targeted either WM or general knowledge. In addition, one of the WM training groups completed a metacognitive program via group seminars. We tested for group differences in WM, inhibitory control, and episodic memory, and we assessed participants' perceived self-efficacy and everyday memory failures. At post-test, we replicated earlier work by demonstrating that participants who completed the WM intervention outperformed the active control group in non-trained WM measures, and to some extent, in inhibitory control. However, we found no evidence that the supplemental metacognitive program led to benefits over and above the WM intervention. Nonetheless, we conclude that our metacognitive program is a step in the right direction given the tentative long-term effects and participants' positive feedback, but more longitudinal data with larger sample sizes are needed to confirm these early findings.

Not so smart? "Smart" drugs increase the level but decrease the quality of cognitive effort

The efficacy of pharmaceutical cognitive enhancers in everyday complex tasks remains to be established. Using the knapsack optimization problem as a stylized representation of difficulty in tasks encountered in daily life, we discover that methylphenidate, dextroamphetamine, and modafinil cause knapsack value attained in the task to diminish significantly compared to placebo, even if the chance of finding the optimal solution (~50%) is not reduced significantly. Effort (decision time and number of steps taken to find a solution) increases significantly, but productivity (quality of effort) decreases significantly. At the same time, productivity differences across participants decrease, even reverse, to the extent that above-average performers end up below average and vice versa. The latter can be attributed to increased randomness of solution strategies. Our findings suggest that "smart drugs" increase motivation, but a reduction in quality of effort, crucial to solve complex problems, annuls this effect.

Apathy in Parkinson's Disease: Clinical Patterns and Neurobiological Basis

Apathy is commonly defined as a loss of motivation leading to a reduction in goal-directed behaviors. This multidimensional syndrome, which includes cognitive, emotional and behavioral components, is one of the most prevalent neuropsychiatric features of Parkinson's disease (PD). It has been established that the prevalence of apathy increases as PD progresses. However, the pathophysiology and anatomic substrate of this syndrome remain unclear. Apathy seems to be underpinned by impaired anatomical structures that link the prefrontal cortex with the limbic system. It can be encountered in the prodromal stage of the disease and in fluctuating PD patients receiving bilateral chronic subthalamic nucleus stimulation. In these stages, apathy may be considered as a disorder of motivation that embodies amotivational behavioral syndrome, is underpinned by combined dopaminergic and serotonergic denervation and is dopa-responsive. In contrast, in advanced PD patients, apathy may be considered as cognitive apathy that announces cognitive decline and PD dementia, is underpinned by diffuse neurotransmitter system dysfunction and Lewy pathology spreading and is no longer dopa-responsive. In this review, we discuss the clinical patterns of apathy and their treatment, the neurobiological basis of apathy, the potential role of the anatomical structures involved and the pathways in motivational and cognitive apathy.

Integrated control of non-motor and motor efforts during perceptual decision-making and action execution: a pilot study

Humans daily life is characterized by a succession of voluntary actions. Since energy resources are limited, the ability to invest the appropriate amount of effort for selecting and executing these actions is a hallmark of adapted behavior. Recent studies indicate that decisions and actions share important principles, including the optimization of their duration when the context requires it. In the present pilot study, we test the hypothesis that the management of effort-related energy resources is shared between decision and action too. Healthy human subjects performed a perceptual decision task where they had to choose between two levels of effort to invest in making the decision (i.e. two levels of perceptual difficulty), and report it with a reaching movement. Crucially, the movement accuracy requirement gradually increased from trial to trial depending on participants' decision performance. Results indicate an overall moderate and non-significant impact of the increasing motor difficulty on the choice of the non-motor (decision) effort to invest in each trial and on decision performance. By contrast, motor performance strongly decreased depending on both the motor and decisional difficulties. Together, the results support the hypothesis of an integrated management of the effort-related energy resources between decision and action. They also suggest that in the present task, the mutualized resources are primarily allocated to the decision-making process to the detriment of movements.

G protein-coupled receptor modulation of striatal dopamine transmission: Implications for psychoactive drug effects

Dopamine transmission in the striatum is a critical mediator of the rewarding and reinforcing effects of commonly misused psychoactive drugs. G protein-coupled receptors (GPCRs) that bind a variety of neuromodulators including dopamine, endocannabinoids, acetylcholine and endogenous opioid peptides regulate dopamine release by acting on several components of dopaminergic circuitry. Striatal dopamine release can be driven by both somatic action potential firing and local mechanisms that depend on acetylcholine released from striatal cholinergic interneurons. GPCRs that primarily regulate somatic firing of dopamine neurons via direct effects or modulation of synaptic inputs are likely to affect distinct aspects of behaviour and psychoactive drug actions compared with those GPCRs that primarily regulate local acetylcholine-dependent dopamine release in striatal regions. This review will highlight mechanisms by which GPCRs modulate dopaminergic transmission and the relevance of these findings to psychoactive drug effects on physiology and behaviour.

Dopamine facilitates the translation of physical exertion into assessments of effort

Our assessments of effort are critically shaped by experiences of exertion. However, it is unclear how the nervous system transforms physical exertion into assessments of effort. Availability of the neuromodulator dopamine influences features of motor performance and effort-based decision-making. To test dopamine's role in the translation of effortful exertion into assessments of effort, we had participants with Parkinson's disease, in dopamine depleted (OFF dopaminergic medication) and elevated (ON dopaminergic medication) states, exert levels of physical exertion and retrospectively assess how much effort they exerted. In a dopamine-depleted state, participants exhibited increased exertion variability and over-reported their levels of exertion, compared to the dopamine-supplemented state. Increased exertion variability was associated with less accurate effort assessment and dopamine had a protective influence on this effect, reducing the extent to which exertion variability corrupted assessments of effort. Our findings provide an account of dopamine's role in the translation of features of motor performance into judgments of effort, and a potential therapeutic target for the increased sense of effort observed across a range of neurologic and psychiatric conditions.

Dual credit assignment processes underlie dopamine signals in a complex spatial environment

Dopamine in the nucleus accumbens helps motivate behavior based on expectations of future reward ("values"). These values need to be updated by experience: after receiving reward, the choices that led to reward should be assigned greater value. There are multiple theoretical proposals for how this credit assignment could be achieved, but the specific algorithms that generate updated dopamine signals remain uncertain. We monitored accumbens dopamine as freely behaving rats foraged for rewards in a complex, changing environment. We observed brief pulses of dopamine both when rats received reward (scaling with prediction error), and when they encountered novel path opportunities. Furthermore, dopamine ramped up as rats ran towards reward ports, in proportion to the value at each location. By examining the evolution of these dopamine place-value signals, we found evidence for two distinct update processes: progressive propagation along taken paths, as in temporal-difference learning, and inference of value throughout the maze, using internal models. Our results demonstrate that within rich, naturalistic environments dopamine conveys place values that are updated via multiple, complementary learning algorithms.

Using Task-Evoked Pupillary Response to Predict Clinical Performance during a Simulation Training

Training in healthcare skills can be affected by trainees' workload when completing a task. Due to cognitive processing demands being negatively correlated to clinical performance, assessing mental workload through objective measures is crucial. This study aimed to investigate task-evoked changes in pupil size as reliable markers of mental workload and clinical performance. A sample of 49 nursing students participated in a cardiac arrest simulation-based practice. Measurements of cognitive demands (NASA-Task Load Index), physiological parameters (blood pressure, oxygen saturation, and heart rate), and pupil responses (minimum, maximum, and difference diameters) throughout revealed statistically significant differences according to performance scores. The analysis of a multiple regression model produced a statistically significant pattern between pupil diameter differences and heart rate, systolic blood pressure, workload, and performance (R² = 0.280; F (6, 41) = 2.660; p < 0.028; d = 2.042). Findings suggest that pupil variations are promising markers to complement physiological metrics for predicting mental workload and clinical performance in medical practice.

Motivational and myopic mechanisms underlying dopamine medication-induced impulsive-compulsive behaviors in Parkinson's disease

Introduction

Dopaminergic medications can trigger impulsive-compulsive behaviors (ICBs) in pre-disposed patients with Parkinson's disease (PD), but what this implies on a neurocognitive level is unclear. Previous findings highlighted potentially exacerbated incentive motivation (willingness to work for rewards) and choice impulsivity (preferring smaller, immediate rewards over larger, delayed rewards) in PD patients with ICBs (PD + ICBs).

Methods

To deeply understand this evidence, we studied 24 PD + ICBs and 28 PD patients without ICBs (PD-ICBs). First of all, patients underwent the assessment of impulsivity traits, mood, anxiety, and addiction condition. We further administered robust objective and subjective measures of specific aspects of motivation. Finally, we explored whether these processes might link to any heightened antisocial behavior (aggression and risky driving) in PD + ICBs.

Results

High levels of positive urgency trait characterized PD + ICBs. They choose to exert more effort for rewards under the conditions of low and medium reward probability and as reward magnitude increases. Findings on choice impulsivity show a great tendency to delay discounting in PD + ICBs, other than a high correlation between delay and probability discounting. In addition, we found what appears to be the first evidence of heightened reactive aggression in PD patients with ICBs. Exacerbated incentive motivation and delay discounting trended toward positively predicting reactive aggression in PD + ICBs.

Discussion

Our promising results suggest that there might be immense value in future large-scale studies adopting a transdiagnostic neurocognitive endophenotype approach to understanding and predicting the addictive and aggressive behaviors that can arise from dopaminergic medication in PD.

Dopamine D1 receptor and effort-based decision making in rats: The moderating effect of sex

Dopamine is a modulating factor in effort-based decision-making, and emerging evidence from pharmacological research suggests that the dopamine D1 receptor is the primary regulator. Given the limited selectivity of pharmacological tools, we further explored this hypothesis using dopamine D1 mutant (DAD1^-/-) rats which have a specific genetic reduction in functional D1 receptors. Moreover, given the strong focus on males in neuroscience research in general and in the role of D1 receptors in effort-based learning, we compared both sexes in the present study. Adult male and female DAD1^-/- mutant rats and wild type controls were trained to press a lever for a reinforcer. Once trained, subjects completed multiple fixed ratio, progressive ratio, and operant effort-choice (concurrent progressive ratio/chow feeding task [PROG/chow]) experiments. We predicted that DAD1^-/- mutant rats would press the lever significantly less than controls across all experiments, have lower breakpoints, and consume more freely available food. As predicted, DAD1^-/- mutant rats (regardless of sex) pressed the lever significantly less than controls and had lower breakpoints. Interestingly, there was a sex * genotype interaction for acquisition rates of lever pressing and change in breakpoints with free food available. Only 31% of DAD1^-/- mutant males acquired lever pressing while 73% of DAD1^-/- mutant females acquired lever pressing. Additionally, DAD1^-/- mutant males had significantly larger decreases in breakpoints when free food was available. These findings extend the pharmacological research suggesting that the dopamine D1 receptor modulates decisions based on effort, which has implications for the development of treatment targeting amotivation in neuropsychiatric disorders. The sex * genotype interaction highlights the importance of including both sexes in future research, especially when there are sex differences in incidences and severity of neuropsychiatric disorders.

Plasticity of synapses and reward circuit function in the genesis and treatment of depression

What changes in brain function cause the debilitating symptoms of depression? Can we use the answers to this question to invent more effective, faster acting antidepressant drug therapies? This review provides an overview and update of the converging human and preclinical evidence supporting the hypothesis that changes in the function of excitatory synapses impair the function of the circuits they are embedded in to give rise to the pathological changes in mood, hedonic state, and thought processes that characterize depression. The review also highlights complementary human and preclinical findings that classical and novel antidepressant drugs relieve the symptoms of depression by restoring the functions of these same synapses and circuits. These findings offer a useful path forward for designing better antidepressant compounds.

Memory-Guided Reaching: Is It Effortful?

We previously showed that perceived effort during visually guided reaching was altered as task demand varied. Further, self-reported subjective fatigue correlated with perceived effort and reach performance under visually guided conditions. Memory-guided reaching often leads to performance deterioration and can provide insights about the planning and control of reach actions. It is unclear how perceived effort changes during memory-guided reaching and whether self-reported subjective fatigue is associated with perceived effort of memory-guided reaching. Twenty-three young adults performed reach actions under visually- and memory-guided conditions. Perceived effort, reaction time, and endpoint error increased significantly from the visually- to the memory-guided condition. Self-reported subjective fatigue was associated with perceived effort and reach distance error during memory-guided reaching; those with higher levels of fatigue reported greater perceived effort and tended to reach farther when visual information was not available. These findings establish a foundation to examine relationships between subjective fatigue, perceived effort, and reach control.

Gains and Losses Affect Learning Differentially at Low and High Attentional Load

Prospective gains and losses influence cognitive processing, but it is unresolved how they modulate flexible learning in changing environments. The prospect of gains might enhance flexible learning through prioritized processing of reward-predicting stimuli, but it is unclear how far this learning benefit extends when task demands increase. Similarly, experiencing losses might facilitate learning when they trigger attentional reorienting away from loss-inducing stimuli, but losses may also impair learning by increasing motivational costs or when negative outcomes are overgeneralized. To clarify these divergent views, we tested how varying magnitudes of gains and losses affect the flexible learning of feature values in environments that varied attentional load by increasing the number of interfering object features. With this task design, we found that larger prospective gains improved learning efficacy and learning speed, but only when attentional load was low. In contrast, expecting losses impaired learning efficacy, and this impairment was larger at higher attentional load. These findings functionally dissociate the contributions of gains and losses on flexible learning, suggesting they operate via separate control mechanisms. One mechanism is triggered by experiencing loss and reduces the ability to reduce distractor interference, impairs assigning credit to specific loss-inducing features, and decreases efficient exploration during learning. The second mechanism is triggered by experiencing gains, which enhances prioritizing reward-predicting stimulus features as long as the interference of distracting features is limited. Taken together, these results support a rational theory of cognitive control during learning, suggesting that experiencing losses and experiencing distractor interference impose costs for learning.

Regulation of cocaine seeking behavior by locus coeruleus noradrenergic activity in the ventral tegmental area is time- and contingency-dependent

Substance use disorder is linked to impairments in the ventral tegmental area (VTA) dopamine (DA) reward system. Noradrenergic (NA) inputs from locus coeruleus (LC) into VTA have been shown to modulate VTA neuronal activity, and are implicated in psychostimulant effects. Phasic LC activity controls time- and context-sensitive processes: decision making, cognitive flexibility, motivation and attention. However, it is not yet known how such temporally-distinct LC activity contributes to cocaine seeking. In a previous study we demonstrated that pharmacological inhibition of NA signaling in VTA specifically attenuates cocaine-seeking. Here, we used virally-delivered opsins to target LC neurons for inhibition or excitation, delivered onto afferents in VTA of male rats seeking cocaine under extinction conditions. Optogenetic stimulation or inhibition was delivered in distinct conditions: upon active lever press, contingently with discreet cues; or non-contingently, i.e., throughout the cocaine seeking session. Non-contingent inhibition of LC noradrenergic terminals in VTA attenuated cocaine seeking under extinction conditions. In contrast, contingent inhibition increased, while contingent stimulation reduced cocaine seeking. These findings were specific for cocaine, but not natural reward (food) seeking. Our results show that NA release in VTA drives behavior depending on timing and contingency between stimuli – context, discreet conditioned cues and reinforcer availability. We show that, depending on those factors, noradrenergic signaling in VTA has opposing roles, either driving CS-induced drug seeking, or contributing to behavioral flexibility and thus extinction.

Neural signals implicated in the processing of appetitive and aversive events in social and non-social contexts

In 2014, we participated in a special issue of Frontiers examining the neural processing of appetitive and aversive events. Specifically, we reviewed brain areas that contribute to the encoding of prediction errors and value versus salience, attention and motivation. Further, we described how we disambiguated these cognitive processes and their neural substrates by using paradigms that incorporate both appetitive and aversive stimuli. We described a circuit in which the orbitofrontal cortex (OFC) signals expected value and the basolateral amygdala (BLA) encodes the salience and valence of both appetitive and aversive events. This information is integrated by the nucleus accumbens (NAc) and dopaminergic (DA) signaling in order to generate prediction and prediction error signals, which guide decision-making and learning via the dorsal striatum (DS). Lastly, the anterior cingulate cortex (ACC) is monitoring actions and outcomes, and signals the need to engage attentional control in order to optimize behavioral output. Here, we expand upon this framework, and review our recent work in which within-task manipulations of both appetitive and aversive stimuli allow us to uncover the neural processes that contribute to the detection of outcomes delivered to a conspecific and behaviors in social contexts. Specifically, we discuss the involvement of single-unit firing in the ACC and DA signals in the NAc during the processing of appetitive and aversive events in both social and non-social contexts.

Nucleus accumbens D1-receptors regulate and focus transitions to reward-seeking action

It is well established that dopamine transmission is integral in mediating the influence of reward expectations on reward-seeking actions. However, the precise causal role of dopamine transmission in moment-to-moment reward-motivated behavioral control remains contentious, particularly in contexts where it is necessary to refrain from responding to achieve a beneficial outcome. To examine this, we manipulated dopamine transmission pharmacologically as rats performed a Go/No-Go task that required them to either make or withhold action to gain either a small or large reward. D1R Stimulation potentiated cue-driven action initiation, including fast impulsive actions on No-Go trials. By contrast, D1R blockade primarily disrupted the successful completion of Go trial sequences. Surprisingly, while after global D1R blockade this was characterized by a general retardation of reward-seeking actions, nucleus accumbens core (NAcC) D1R blockade had no effect on the speed of action initiation or impulsive actions. Instead, fine-grained analyses showed that this manipulation decreased the precision of animals' goal-directed actions, even though they usually still followed the appropriate response sequence. Strikingly, such "unfocused" responding could also be observed off-drug, particularly when only a small reward was on offer. These findings suggest that the balance of activity at NAcC D1Rs plays a key role in enabling the rapid activation of a focused, reward-seeking state to enable animals to efficiently and accurately achieve their goal.

Dopaminergic medication increases motivation to exert cognitive control by reducing subjective effort costs in Parkinson's patients

Engaging in demanding mental activities requires the allocation of cognitive control, which can be effortful and aversive. Individuals thus tend to avoid exerting cognitive effort if less demanding behavioral options are available. Recent accounts propose a key role for dopamine in motivating behavior by increasing the sensitivity to rewards associated with effort exertion. Whether dopamine additionally plays a specific role in modulating the sensitivity to the costs of cognitive effort, even in the absence of any incentives, is much less clear. To address this question, we assessed cognitive effort avoidance in patients (n = 38) with Parkinson's disease, a condition characterized by loss of midbrain dopaminergic neurons, both ON and OFF dopaminergic medication and compared them to healthy controls (n = 24). Effort avoidance was assessed using the Demand Selection Task (DST), in which participants could freely choose between performing a high-demand or a low-demand version of a task-switching paradigm. Critically, participants were not offered any incentives to choose the more effortful option, nor for good performance. While healthy controls and patients OFF their dopaminergic medications consistently preferred the low-demand option, effort avoidance in patients ON dopaminergic medications was reduced compared to patients OFF, a difference which seems to lessen over trials. These differences in preference could not be explained by altered task-switching performance. Although patients ON were less accurate at detecting the different effort levels, as measured during instructed forced-choice blocks, their detection ability was not associated with effort avoidance, unlike in the healthy controls and the patients OFF. Our findings provide evidence that dopamine replacement in Parkinson's patients increases the willingness to engage in cognitively demanding behavior, and that this cannot be explained by possible effects of dopamine replacement on performance nor on the ability to detect effort demands. These results suggest that dopamine plays a role in reducing the sensitivity to effort costs that is independent of its role in enhancing the sensitivity to the benefits of effort exertion.

Cognitive control, motivation and fatigue: A cognitive neuroscience perspective

The present article provides a unified systematic account of the role of cognitive control, motivation and dopamine pathways in relation to the development of fatigue. Since cognitive fatigue is considered to be one aspect of the general control system that manages goal activity in the service of motivational requirements (Hockey, 2011), our focus is also broader than fatigue itself. The paper shall therefore first focus on the motivation-control interactions at the level of networks of the brain. A motivational control network is argued to play a critical role in shaping goal-directed behavior, in conjunction with dopamine systems that energize the network. Furthermore, motivation-control interactions as implemented in networks of the brain provide an important element to elucidate how decision making weighs both the anticipated benefits and costs of control operations, in optimal and suboptimal conditions such as mental fatigue. The paper further sketches how fatigue affects the connectivity of large-scale networks in the brain during effortful exercition, in particular the high-cost long striatal-cortical pathways, leading to a global reduction of integration in the brain's network architecture. The resulting neural state within these networks then enters as interoceptive information to systems in the brain that perform cost-benefit calculations. Based on these notions we propose a unifying cost-benefit model, inspired by influential insights from the current neuroscience literature of how fatigue changes the motivation to perform. The model specifies how the reward value, effort costs and fatigue aspects of task performance converge in the medial prefrontal cortex to calculate the net motivation value of stimuli and select the appropriate actions.

Optogenetic stimulation of lateral hypothalamic orexin/dynorphin inputs in the ventral tegmental area potentiates mesolimbic dopamine neurotransmission and promotes reward-seeking behaviours

Reward and reinforcement processes are critical for survival and propagation of genes. While numerous brain systems underlie these processes, a cardinal role is ascribed to mesolimbic dopamine. However, ventral tegmental area (VTA) dopamine neurons receive complex innervation and various neuromodulatory factors, including input from lateral hypothalamic (LH) orexin/hypocretin neurons which also express and co-release the neuropeptide, dynorphin. Dynorphin in the VTA induces aversive conditioning through the Kappa opioid receptor (KOR) and decreases dopamine when administered intra-VTA. Exogenous application of orexin or orexin 1 receptor (oxR1) antagonists in the VTA bidirectionally modulates dopamine-driven motivation and reward-seeking behaviours, including the attribution of motivational value to primary rewards and associated conditioned stimuli. However, the effect of endogenous stimulation of LH orexin/dynorphin-containing projections to the VTA and the potential contribution of co-released dynorphin on mesolimbic dopamine and reward related processes remains uncharacterised. We combined optogenetic, electrochemical, and behavioural approaches to examine this. We found that optical stimulation of LH orexin/dynorphin inputs in the VTA potentiates mesolimbic dopamine neurotransmission in the nucleus accumbens (NAc) core, produces real time and conditioned place preference, and increases the food cue-directed orientation in a Pavlovian conditioning procedure. LH orexin/dynorphin potentiation of NAc dopamine release and real time place preference was blocked by an oxR1, but not KOR antagonist. Thus, rewarding effects associated with optical stimulation of LH orexin/dynorphin inputs in the VTA are predominantly driven by orexin rather than dynorphin.

Aversive motivation and cognitive control

Aversive motivation plays a prominent role in driving individuals to exert cognitive control. However, the complexity of behavioral responses attributed to aversive incentives creates significant challenges for developing a clear understanding of the neural mechanisms of this motivation-control interaction. We review the animal learning, systems neuroscience, and computational literatures to highlight the importance of experimental paradigms that incorporate both motivational context manipulations and mixed motivational components (e.g., bundling of appetitive and aversive incentives). Specifically, we postulate that to understand aversive incentive effects on cognitive control allocation, a critical contextual factor is whether such incentives are associated with negative reinforcement or punishment. We further illustrate how the inclusion of mixed motivational components in experimental paradigms enables increased precision in the measurement of aversive influences on cognitive control. A sharpened experimental and theoretical focus regarding the manipulation and assessment of distinct motivational dimensions promises to advance understanding of the neural, monoaminergic, and computational mechanisms that underlie the interaction of motivation and cognitive control.

Influences of dopaminergic system dysfunction on late-life depression

Deficits in cognition, reward processing, and motor function are clinical features relevant to both aging and depression. Individuals with late-life depression often show impairment across these domains, all of which are moderated by the functioning of dopaminergic circuits. As dopaminergic function declines with normal aging and increased inflammatory burden, the role of dopamine may be particularly salient for late-life depression. We review the literature examining the role of dopamine in the pathogenesis of depression, as well as how dopamine function changes with aging and is influenced by inflammation. Applying a Research Domain Criteria (RDoC) Initiative perspective, we then review work examining how dopaminergic signaling affects these domains, specifically focusing on Cognitive, Positive Valence, and Sensorimotor Systems. We propose a unified model incorporating the effects of aging and low-grade inflammation on dopaminergic functioning, with a resulting negative effect on cognition, reward processing, and motor function. Interplay between these systems may influence development of a depressive phenotype, with an initial deficit in one domain reinforcing decline in others. This model extends RDoC concepts into late-life depression while also providing opportunities for novel and personalized interventions.

Neural Circuitry of Salience and Reward Processing in Psychosis

The processing of salient and rewarding stimuli is integral to engaging our attention, stimulating anticipation for future events, and driving goal-directed behaviors. Widespread impairments in these processes are observed in psychosis, which may be associated with worse functional outcomes or mechanistically linked to the development of symptoms. Here, we summarize the current knowledge of behavioral and functional neuroimaging in salience, prediction error, and reward. Although each is a specific process, they are situated in multiple feedback and feedforward systems integral to decision making and cognition more generally. We argue that the origin of salience and reward processing dysfunctions may be centered in the subcortex during the earliest stages of psychosis, with cortical abnormalities being initially more spared but becoming more prominent in established psychotic illness/schizophrenia. The neural circuits underpinning salience and reward processing may provide targets for delaying or preventing progressive behavioral and neurobiological decline.

Locus coeruleus neurons encode the subjective difficulty of triggering and executing actions

The brain stem noradrenergic nucleus locus coeruleus (LC) is involved in various costly processes: arousal, stress, and attention. Recent work has pointed toward an implication in physical effort, and indirect evidence suggests that the LC could be also involved in cognitive effort. To assess the dynamic relation between LC activity, effort production, and difficulty, we recorded the activity of 193 LC single units in 5 monkeys performing 2 discounting tasks (a delay discounting task and a force discounting task), as well as a simpler target detection task where conditions were matched for difficulty and only differed in terms of sensory-motor processes. First, LC neurons displayed a transient activation both when monkeys initiated an action and when exerting force. Second, the magnitude of the activation scaled with the associated difficulty, and, potentially, the corresponding amount of effort produced, both for decision and force production. Indeed, at action initiation in both discounting tasks, LC activation increased in conditions associated with lower average engagement rate, i.e., those requiring more cognitive control to trigger the response. Decision-related activation also scaled with response time (RT), over and above task parameters, in line with the idea that it reflects the amount of resources (here time) spent on the decision process. During force production, LC activation only scaled with the amount of force produced in the force discounting task, but not in the control target detection task, where subjective difficulty was equivalent across conditions. Our data show that LC neurons dynamically track the amount of effort produced to face both cognitive and physical challenges with a subsecond precision. This works provides key insight into effort processing and the contribution of the noradrenergic system, which is affected in several pathologies where effort is impaired, including Parkinson disease and depression.

Local and distributed cortical markers of effort expenditure during sustained goal pursuit

The adaptive adjustment of behavior in pursuit of desired goals is critical for survival. To accomplish this complex feat, individuals must weigh the potential benefits of a given action against time, energy, and resource costs. Here, we examine brain responses associated with willingness to exert physical effort during the sustained pursuit of desired goals. Our analyses reveal a distributed pattern of brain activity in aspects of ventral medial prefrontal cortex that tracks with trial-level variability in effort expenditure. Indicating the brain represents echoes of effort at the point of feedback, whole-brain searchlights identified signals reflecting past effort expenditure in medial and lateral prefrontal cortices, encompassing broad swaths of frontoparietal and dorsal attention networks. These data have important implications for our understanding of how the brain's valuation mechanisms contend with the complexity of real-world dynamic environments with relevance for the study of behavior across health and disease.

Relationship of ventral striatum activation during effort discounting to clinical amotivation severity in schizophrenia

Motivational deficits play a central role in disability due to negative symptoms of schizophrenia (SZ), but limited pathophysiological understanding impedes critically needed therapeutic development. We applied an fMRI Effort Discounting Task (EDT) that quantifies motivation using a neuroeconomic decision-making approach, capturing the degree to which effort requirements produce reductions in the subjective value (SV) of monetary reward. An analyzed sample of 21 individuals with SZ and 23 group-matched controls performed the EDT during fMRI. We hypothesized that ventral striatum (VS) as well as extended brain motivation circuitry would encode SV, integrating reward and effort costs. We also hypothesized that VS hypoactivation during EDT decisions would demonstrate a dimensional relationship with clinical amotivation severity, reflecting greater suppression by effort costs. As hypothesized, VS as well as a broader cortico-limbic network were activated during the EDT and this activation correlated positively with SV. In SZ, activation to task decisions was reduced selectively in VS. Greater VS reductions correlated with more severe clinical amotivation in SZ and across all participants. However, these diagnosis and amotivation effects could not be explained by the response to parametric variation in reward, effort, or model-based SV. Our findings demonstrate that VS hypofunction in schizophrenia is manifested during effort-based decisions and reflects dimensional motivation impairment. Dysfunction of VS impacting effort-based decision-making can provide a target for biomarker development to guide novel efforts to assess and treat disabling amotivation.

Neural Representation of Costs and Rewards in Decision Making

Decision making is crucial for animal survival because the choices they make based on their current situation could influence their future rewards and could have potential costs. This review summarises recent developments in decision making, discusses how rewards and costs could be encoded in the brain, and how different options are compared such that the most optimal one is chosen. The reward and cost are mainly encoded by the forebrain structures (e.g., anterior cingulate cortex, orbitofrontal cortex), and their value is updated through learning. The recent development on dopamine and the lateral habenula's role in reporting prediction errors and instructing learning will be emphasised. The importance of dopamine in powering the choice and accounting for the internal state will also be discussed. While the orbitofrontal cortex is the place where the state values are stored, the anterior cingulate cortex is more important when the environment is volatile. All of these structures compare different attributes of the task simultaneously, and the local competition of different neuronal networks allows for the selection of the most appropriate one. Therefore, the total value of the task is not encoded as a scalar quantity in the brain but, instead, as an emergent phenomenon, arising from the computation at different brain regions.

Chronic Augmentation of Endocannabinoid Levels Persistently Increases Dopaminergic Encoding of Reward Cost and Motivation

Motivational deficits characterized by an unwillingness to overcome effortful costs are a common feature of neuropsychiatric and neurologic disorders that are insufficiently understood and treated. Dopamine (DA) signaling in the nucleus accumbens (NAc) facilitates goal-seeking, but how NAc DA release encodes motivationally salient stimuli to influence effortful investment is not clear. Using fast-scan cyclic voltammetry in male and female mice, we find that NAc DA release diametrically responds to cues signaling increasing cost of reward, while DA release to the reward itself is unaffected by its cost. Because endocannabinoid (eCB) signaling facilitates goal seeking and NAc DA release, we further investigated whether repeated augmentation of the eCB 2-arachidonoylglycerol with a low dose of a monoacylglycerol lipase (MAGL) inhibitor facilitates motivation and DA signaling without the development of tolerance. We find that chronic MAGL treatment stably facilitates goal seeking and DA encoding of prior reward cost, providing critical insight into the neurobiological mechanisms of a viable treatment for motivational deficits.SIGNIFICANCE STATEMENT Decades of work has established a fundamental role for dopamine neurotransmission in motivated behavior and cue-reward learning, but how dopaminergic encoding of cues associates with motivated action has remained unclear. Specifically, how dopamine neurons signal future and prior reward cost, and whether this can be modified to influence motivational set points is not known. The current study provides important insight into how dopamine neurons encode motivationally relevant stimuli to influence goal-directed action and supports cannabinoid-based therapies for treatment of motivational disorders.

Action Costs Rapidly and Automatically Interfere with Reward-Based Decision-Making in a Reaching Task

It is widely assumed that we select actions we value the most. While the influence of rewards on decision-making has been extensively studied, evidence regarding the influence of motor costs is scarce. Specifically, how and when motor costs are integrated in the decision process is unclear. Twenty-two right-handed human participants performed a reward-based target selection task by reaching with their right arm toward one of two visual targets. Targets were positioned in different directions according to biomechanical preference, such that one target was systematically associated with a lower motor cost than the other. Only one of the two targets was rewarded, either in a congruent or incongruent manner with respect to the associated motor cost. A timed-response paradigm was used to manipulate participants' reaction times (RT). Results showed that when the rewarded target carried the highest motor cost, movements produced at short RT (<350 ms) were deviated toward the other (i.e., non-rewarded, low-cost (LC) target). In this context participants needed an additional 150-ms delay to reach the same percentage of rewarded trials as when the LC target was rewarded. Crucially, motor costs affected the total earnings of participants. These results demonstrate a robust interference of motor costs in a simple reward-based decision-making task. They point to the rapid and automatic integration of motor costs at an early stage of processing, potentially through the direct modulation of competing action representations in parieto-frontal regions. The progressive overcoming of this bias with increasing RT is likely achieved through top-down signaling pertaining to expected rewards.

Cocaine-induced increases in motivation require 2-arachidonoylglycerol mobilization and CB1 receptor activation in the ventral tegmental area

A wide body of evidence supports an integral role for mesolimbic dopamine (DA) in motivated behavior. In brief, drugs that increase DA in mesolimbic terminal regions, like cocaine, enhance motivation, while drugs that decrease DA concentration reduce motivation. Data from our laboratory and others shows that phasic activation of mesolimbic DA requires signaling at cannabinoid type-1 (CB1) receptors in the ventral tegmental area (VTA), and systemic delivery of CB1 receptor antagonists reduces DA cell activity and attenuates motivated behaviors. Recent findings demonstrate that cocaine mobilizes the endocannabinoid 2-arachidonoylglycerol (2-AG) in the VTA to cause phasic activation of DA neurons and terminal DA release. It remains unclear, however, if cocaine-induced midbrain 2-AG signaling contributes to the motivation-enhancing effects of cocaine. To examine this, we trained male and female rats on a progressive ratio (PR) task for a food reinforcer. Each rat underwent a series of tests in which they were pretreated with cocaine alone or in combination with systemic or intra-VTA administration of the CB1 receptor antagonist rimonabant or the 2-AG synthesis inhibitor tetrahydrolipstatin (THL). Cocaine increased motivation, measured by augmented PR breakpoints, while rimonabant dose-dependently decreased motivation. Importantly, intra-VTA administration of rimonabant or THL, at doses that did not decrease breakpoints on their own, blocked systemic cocaine administration from increasing breakpoints in male and female rats. These data suggest that cocaine-induced increases in motivation require 2-AG signaling at CB1 receptors in the VTA and may provide critical insight into cannabinoid-based pharmacotherapeutic targets for the successful treatment of substance abuse.

Dopamine, vocalization, and astrocytes

Dopamine, the main catecholamine neurotransmitter in the brain, is predominately produced in the basal ganglia and released to various brain regions including the frontal cortex, midbrain and brainstem. Dopamine's effects are widespread and include modulation of a number of voluntary and innate behaviors. Vigilant regulation and modulation of dopamine levels throughout the brain is imperative for proper execution of motor behaviors, in particular speech and other types of vocalizations. While dopamine's role in motor circuitry is widely accepted, its unique function in normal and abnormal speech production is not fully understood. In this perspective, we first review the role of dopaminergic circuits in vocal production. We then discuss and propose the conceivable involvement of astrocytes, the numerous star-shaped glia cells of the brain, in the dopaminergic network modulating normal and abnormal vocal productions.

Anticipatory Energization Revealed by Pupil and Brain Activity Guides Human Effort-Based Decision Making

An organism's fitness is determined by how it chooses to adapt to effort in response to challenges. Exertion of effort correlates with activity in dorsomedial prefrontal cortex (dmPFC) and noradrenergic pupil dilation, but little is known about the role of these neurophysiological processes for decisions about future efforts, they may provide anticipatory energization to help us accept the challenge or a cost representation that is weighted against the expected rewards. Here, we provide evidence for the former, by measuring pupil and functional magnetic resonance imaging (fMRI) brain responses while 52 human participants (29 females) chose whether to exert efforts to obtain rewards. Both pupil-dilation rate and dmPFC fMRI activity increased with anticipated effort level, and these increases differ depending on the choice outcome: they were stronger when participants chose to accept the challenge compared with when the challenge was declined. Crucially, the choice-dependent modulation of pupil and brain-activity effort representations were stronger in participants whose behavioral choices were more sensitive to effort. Our results identify a process involving the peripheral and central human nervous system that simulates the required energization before overt response, suggesting a role in guiding effort-based decisions.SIGNIFICANCE STATEMENT The brain's arousal system tracks the effort we engage in during strenuous activity. But much less is known about what role this effort signaling may play when we decide whether to exert effort in the future. Here, we characterize pupil-linked arousal and brain signals that guide decisions whether to engage in effort to gain money. During such choices, increases in brain activity and pupil dilation correlated with the effort involved in the chosen option, and these increases were stronger when people decided to accept the effort compared with when they rejected it. These results suggest that the brain arousal system guides decisions by energizing the organism for the prospective challenge.

Pupil dilation predicts individual self-regulation success across domains

Multiple theories have proposed that increasing central arousal through the brain's locus coeruleus-norepinephrine system may facilitate cognitive control and memory. However, the role of the arousal system in emotion regulation is less well understood. Pupil diameter is a proxy to infer upon the central arousal state. We employed an emotion regulation paradigm with a combination of design features that allowed us to dissociate regulation from emotional arousal in the pupil diameter time course of 34 healthy adults. Pupil diameter increase during regulation predicted individual differences in emotion regulation success beyond task difficulty. Moreover, the extent of this individual regulatory arousal boost predicted performance in another self-control task, dietary health challenges. Participants who harnessed more regulation-associated arousal during emotion regulation were also more successful in choosing healthier foods. These results suggest that a common arousal-based facilitation mechanism may support an individual's self-control across domains.

D1- and D2-like receptors differentially mediate the effects of dopaminergic transmission on cost-benefit evaluation and motivation in monkeys

It has been widely accepted that dopamine (DA) plays a major role in motivation, yet the specific contribution of DA signaling at D₁-like receptor (D₁R) and D₂-like receptor (D₂R) to cost–benefit trade-off remains unclear. Here, by combining pharmacological manipulation of DA receptors (DARs) and positron emission tomography (PET) imaging, we assessed the relationship between the degree of D₁R/D₂R blockade and changes in benefit- and cost-based motivation for goal-directed behavior of macaque monkeys. We found that the degree of blockade of either D₁R or D₂R was associated with a reduction of the positive impact of reward amount and increasing delay discounting. Workload discounting was selectively increased by D₂R antagonism. In addition, blocking both D₁R and D₂R had a synergistic effect on delay discounting but an antagonist effect on workload discounting. These results provide fundamental insight into the distinct mechanisms of DA action in the regulation of the benefit- and cost-based motivation, which have important implications for motivational alterations in both neurological and psychiatric disorders.

Using quantitatively controlled pharmacological manipulations, this study teases apart the role of D1- and D2-like dopamine receptors in motivation and goal-directed behavior in monkeys, revealing complementary roles of two dopamine receptor subtypes in the computation of the cost/benefit trade-off to guide action.

A mosaic of cost-benefit control over cortico-striatal circuitry

Dopamine contributes to cognitive control through well-established effects in both the striatum and cortex. Although earlier work suggests that dopamine affects cognitive control capacity, more recent work suggests that striatal dopamine may also impact on cognitive motivation. We consider the emerging perspective that striatal dopamine boosts control by making people more sensitive to the benefits versus the costs of cognitive effort, and we discuss how this sensitivity shapes competition between controlled and prepotent actions. We propose that dopamine signaling in distinct cortico-striatal subregions mediates different types of cost-benefit tradeoffs, and also discuss mechanisms for the local control of dopamine release, enabling selectivity among cortico-striatal circuits. In so doing, we show how this cost-benefit mosaic can reconcile seemingly conflicting findings about the impact of dopamine signaling on cognitive control.

Divergent Whole Brain Projections from the Ventral Midbrain in Macaques

To understand the connectome of the axonal arborizations of dopaminergic midbrain neurons, we investigated the anterograde spread of highly sensitive viral tracers injected into the ventral tegmental area (VTA) and adjacent areas in 3 macaques. In 2 monkeys, injections were centered on the lateral VTA with some spread into the substantia nigra, while in one animal the injection targeted the medial VTA with partial spread into the ventro-medial thalamus. Double-labeling with antibodies against transduced fluorescent proteins (FPs) and tyrosine hydroxylase indicated that substantial portions of transduced midbrain neurons were dopaminergic. Interestingly, cortical terminals were found either homogeneously in molecular layer I, or more heterogeneously, sometimes forming patches, in the deeper laminae II-VI. In the animals with injections in lateral VTA, terminals were most dense in somatomotor cortex and the striatum. In contrast, when the medial VTA was transduced, dense terminals were found in dorsal prefrontal and temporal cortices, while projections to striatum were sparse. In all monkeys, orbitofrontal and occipito-parietal cortex received strong and weak innervation, respectively. Thus, the dopaminergic ventral midbrain sends heterogeneous projections throughout the brain. Furthermore, our results suggest the existence of subgroups in meso-dopaminergic neurons depending on their location in the primate ventral midbrain.