Dopamine and effort-based decision making

Treatment of Apathy in Parkinson's Disease and Implications for Underlying Pathophysiology

Apathy is a prevalent and highly debilitating non-motor symptom of Parkinson's disease (PD) that is often overlooked in clinical practice due to its subtle nature. This review aims to provide a comprehensive overview of the current evidence for the treatment of apathy in PD, highlighting recent advancements and emerging therapeutic avenues. In this review, we analyse a diverse array of treatment strategies for apathy in PD, including pharmacological interventions, non-pharmacological approaches, and emerging neuromodulation techniques. We evaluate the efficacy, safety, and limitations of established pharmacotherapies, such as dopaminergic agents, antidepressants, and cognitive enhancers. Additionally, we examine the promising role of non-pharmacological interventions, encompassing psychotherapies and behavioural interventions, in ameliorating apathetic symptoms. Furthermore, this review explores the effects of neuromodulation techniques on apathy, including the modulation of apathy via deep brain stimulation and emerging data on the potential influence of transcranial magnetic stimulation (TMS) on apathy in PD. Ultimately, a deeper understanding of effective treatment strategies for apathy has the potential to significantly improve the quality of life and overall well-being of individuals living with PD.

The Calmodulin-interacting peptide Pcp4a regulates feeding state-dependent behavioral choice in zebrafish

Animals constantly need to judge the valence of an object in their environment: is it potential food or a threat? The brain makes fundamental decisions on the appropriate behavioral strategy by integrating external information from sensory organs and internal signals related to physiological needs. For example, a hungry animal may take more risks than a satiated one when deciding to approach or avoid an object. Using a proteomic profiling approach, we identified the Calmodulin-interacting peptide Pcp4a as a key regulator of foraging-related decisions. Food intake reduced abundance of protein and mRNA of pcp4a via dopamine D2-like receptor-mediated repression of adenylate cyclase. Accordingly, deleting the pcp4a gene made zebrafish larvae more risk averse in a binary decision assay. Strikingly, neurons in the tectum became less responsive to prey-like visual stimuli in pcp4a mutants, thus biasing the behavior toward avoidance. This study pinpoints a molecular mechanism modulating behavioral choice according to internal state.

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.

Apathy-like behaviour in tau mouse models of Alzheimer's disease and frontotemporal dementia

Apathy is the most common behavioural and psychological symptom in Alzheimer's disease (AD) and other neurodegenerative diseases including frontotemporal dementia (FTD) and Parkinson's disease (PD). In patients, apathy can include symptoms of loss of motivation, initiative, and interest, listlessness, and indifference, flattening of emotions, absence of drive and passion. Researchers have later refined this to a reduction in goal direct behaviours. In animals, specific symptoms of apathy-like behaviour have been modelled including goal directed or nest-building behaviour which are seen as indicative of proxies for motivation and daily activities. In the present study a nest-building protocol was established using four different inbred mouse strains (CD1, BALB/c, C57Bl/6J, C3H) before assessing AD and FTD tau transgenic mice of Line 1 (L1) and Line 66 (L66) in this paradigm. Female mice aged 5 - 6 months were assessed in the home cage over a period of 7 days with nest-building behaviour scored by three independent experimenters at intervals of 1-, 2- and 7-days post nestlet introduction. Inbred mouse strains displayed different levels of nesting behaviour. BALB/c mice were more proficient than CD1 and C3H mice, while all strains displayed similar nest-building behaviour by day 7. In the tau mouse models, L66 presented with impaired nesting compared to wild-type on days 1 and 2 (not day 7), whereas L1 performed like wild-type on all days. Anhedonia measured in a sucrose preference test was only observed in L66. Anhedonia and low nesting scores in L66 mice are indicative of apathy-like phenotypes. Differences evident between the L1 and L66 tau transgenic mouse models are likely due to the different human tau species expressed in these mice.

Fatigue during acute systemic inflammation is associated with reduced mental effort expenditure while task accuracy is preserved

BACKGROUND

Earlier work within the physical domain showed that acute inflammation changes motivational prioritization and effort allocation rather than physical abilities. It is currently unclear whether a similar motivational framework accounts for the mental fatigue and cognitive symptoms of acute sickness. Accordingly, this study aimed to assess the relationship between fatigue, cytokines and mental effort-based decision making during acute systemic inflammation.

METHODS

Eighty-five participants (41 males; 18-30 years (M=23.0, SD=2.4)) performed a mental effort-based decision-making task before, 2 hours after, and 5 hours after intravenous administration of 1 ng/kg bacterial lipopolysaccharide (LPS) to induce systemic inflammation. Plasma concentrations of cytokines (interleukin (IL)-6, IL-8 and tumor necrosis factor (TNF)) and fatigue levels were assessed at similar timepoints. In the task, participants decided whether they wanted to perform (i.e., 'accepted') arithmetic calculations of varying difficulty (3 levels: easy, medium, hard) in order to obtain rewards (3 levels: 5, 6 or 7 points). Acceptance rates were analyzed using a binomial generalized estimated equation (GEE) approach with effort, reward and time as independent variables. Arithmetic performance was measured per effort level prior to the decisions and included as a covariate. Associations between acceptance rates, fatigue (self-reported) and cytokine concentrations levels were analyzed using partial correlation analyses.

RESULTS

Plasma cytokine concentrations and fatigue were increased at 2 hours post-LPS compared to baseline and 5 hours post-LPS administration. Acceptance rates decreased for medium, but not for easy or hard effort levels at 2 hours post-LPS versus baseline and 5 hours post-LPS administration, irrespective of reward level. This reduction in acceptance rates occurred despite improved accuracy on the arithmetic calculations itself. Reduced acceptance rates for medium effort were associated with increased fatigue, but not with increased cytokines.

CONCLUSION

Fatigue during acute systemic inflammation is associated with alterations in mental effort allocation, similarly as observed previously for physical effort-based choice. Specifically, willingness to exert mental effort depended on effort and not reward information, while task accuracy was preserved. These results extend the motivational account of inflammation to the mental domain and suggest that inflammation may not necessarily affect domain-specific mental abilities, but rather affects domain-general effort-allocation processes.

The persistence of stress-induced physical inactivity in rats: an investigation of central monoamine neurotransmitters and skeletal muscle oxidative stress

Introduction

Sedentary lifestyles have reached epidemic proportions world-wide. A growing body of literature suggests that exposures to adverse experiences (e.g., psychological traumas) are a significant risk factor for the development of physically inactive lifestyles. However, the biological mechanisms linking prior stress exposure and persistent deficits in physical activity engagement remains poorly understood.

Methods

The purpose of this study was twofold. First, to identify acute stress intensity thresholds that elicit long-term wheel running deficits in rats. To that end, young adult male rats were exposed to a single episode of 0, 50, or 100 uncontrollable tail shocks and then given free access to running wheels for 9 weeks. Second, to identify stress-induced changes to central monoamine neurotransmitters and peripheral muscle physiology that may be maladaptive to exercise output. For this study, rats were either exposed to a single episode of uncontrollable tail shocks (stress) or left undisturbed in home cages (unstressed). Eight days later, monoamine-related neurochemicals were quantified by ultra-high performance liquid chromatography (UHPLC) across brain reward, motor, and emotion structures immediately following a bout of graded treadmill exercise controlled for duration and intensity. Additionally, protein markers of oxidative stress, inflammation, and metabolic activity were assessed in the gastrocnemius muscle by Western blot.

Results

For experiment 1, stress exposure caused a shock number-dependent two to fourfold decrease in wheel running distance across the entire duration of the study. For experiment 2, stress exposure curbed an exercise-induced increase of dopamine (DA) turnover measures in the prefrontal cortex and hippocampus, and augmented serotonin (5HT) turnover in the hypothalamus and remaining cortical area. However, stress exposure also caused several monoaminergic changes independent of exercise that could underlie impaired motivation for physical activity, including a mild dopamine deficiency in the striatal area. Finally, stress potently increased HSP70 and lowered SOD2 protein concentrations in the gastrocnemius muscle, which may indicate prolonged oxidative stress.

Discussion

These data support some of the possible central and peripheral mechanisms by which exposure to adverse experiences may chronically impair physical activity engagement.

Evaluating the neural substrates of effort-expenditure for reward in adults with major depressive disorder and obesity

Converging evidence has suggested that disturbances in monetary reward processing may subserve the shared biosignature between major depressive disorder (MDD) and obesity. However, there remains a paucity of studies that have evaluated the deficits in specific subcomponents of reward functioning in populations with MDD and obesity comorbidity. We evaluated the association between effort-expenditure for monetary reward and neural activation in regions associated with reward-based decision making (i.e., the caudate nucleus, anterior cingulate cortex (ACC) and hippocampus) in people with MDD and obesity comorbidity. We acquired structural and functional magnetic resonance imaging (fMRI) in 12 participants and performed a spherical region-of-interest analysis (ROI) using previously defined peak MNI coordinates. A one-sample t-test was employed to compare ROI-specific blood-oxygen-level-dependent (BOLD) signal change during the task choice selection window (i.e., high-effort vs. low-effort task) of the effort-expenditure for reward task (EEfRT). We observed no change in activation of the caudate nucleus, ACC or hippocampus in participants with increased BMI when contrasting the high effort > low effort reward magnitude condition for the EEfRT. The findings from our exploratory study evaluated the disturbances in fundamental reward processes, including cost-benefit decision making, in people MDD and obesity. Future studies should further investigate this relationship with a larger sample size.

Neural correlates of overvaluation and the effort to save possessions in a novel decision task: An exploratory fMRI study

Introduction

People exhibit a strong attachment to possessions, observed in behavioral economics through loss aversion using new items in the Endowment or IKEA effects and in clinical psychology through pathological trouble discarding domestic items in Hoarding Disorder. These fields rarely intersect, but both document a reticence to relinquish a possessed item, even at a cost, which is associated with feelings of loss but can include enhanced positive states as well.

Methods

To demonstrate the shared properties of these loss-related ownership effects, we developed the Pretzel Decorating Task (PDT), which concurrently measures overvaluation of one's own over others' items and feelings of loss associated with losing a possession, alongside enhanced positive appraisals of one's items and an effort to save them. The PDT was piloted with 31 participants who decorated pretzels and responded to their own or others' items during functional neuroimaging (fMRI). Participants observed one item per trial (self or other) and could work to save it (high or low probability loss) before learning the fate of the item (trashed or saved). Finally, participants rated items and completed hoarding tendency scales.

Results

The hypotheses were supported, as even non-clinical participants overvalued, viewed as nicer, feared losing, and worked harder to save their items over others'-a response that correlated with hoarding tendencies and motor-motivational brain activation. Our region of interest in the nucleus accumbens (NAcc) was engaged when viewing one's own items to the extent that people worked harder to save them and was more active when their items were saved when they felt emotionally attached to possessions in real life. When their items were trashed, NAcc activity negatively correlated with trouble discarding and emotional attachments to possessions. Right anterior insula was more active when working to save one's own over others' items. Extensive motor-motivational areas were engaged when working to save one's own over others' items, including cerebellum, primary motor and somatosensory regions, and retrosplenial/parahippocampal regions-even after controlling for tapping.

Discussion

Our attachments to items are emotional, continuous across typical and pathological populations, and drive us to save possessions that we value.

Delayed effects of neonatal immune activation on brain neurochemistry and hypothalamic-pituitary-adrenal axis functioning

During the postnatal period, the brain is highly sensitive to stress and inflammation, which are hazardous to normal growth and development. There is increasing evidence that inflammatory processes in the early postnatal period increase the risk of psychopathologies and cognitive impairment later in life. On the other hand, there are few studies on the ability of infectious agents to cause long-term neuroinflammation, leading to changes in the hypothalamic-pituitary-adrenal axis functioning and an imbalance in the neurotransmitter system. In this review, we examine short- and long-term effects of neonatal-induced inflammation in rodents on glutamatergic, GABAergic and monoaminergic systems and on hypothalamic-pituitary-adrenal axis activity.

Aging, testosterone, and neuroplasticity: friend or foe?

Neuroplasticity or neural plasticity implicates the adaptive potential of the brain in response to extrinsic and intrinsic stimuli. The concept has been utilized in different contexts such as injury and neurological disease. Neuroplasticity mechanisms have been classified into neuroregenerative and function-restoring processes. In the context of injury, neuroplasticity has been defined in three post-injury epochs. Testosterone plays a key yet double-edged role in the regulation of several neuroplasticity alterations. Research has shown that testosterone levels are affected by numerous factors such as age, stress, surgical procedures on gonads, and pharmacological treatments. There is an ongoing debate for testosterone replacement therapy (TRT) in aging men; however, TRT is more useful in young individuals with testosterone deficit and more specific subgroups with cognitive dysfunction. Therefore, it is important to pay early attention to testosterone profile and precisely uncover its harms and benefits. In the present review, we discuss the influence of environmental factors, aging, and gender on testosterone-associated alterations in neuroplasticity, as well as the two-sided actions of testosterone in the nervous system. Finally, we provide practical insights for further study of pharmacological treatments for hormonal disorders focusing on restoring neuroplasticity.

Responses and functions of dopamine in nucleus accumbens core during social behaviors

Social behaviors are among the most important motivated behaviors. How dopamine (DA), a "reward" signal, releases during social behaviors has been a topic of interest for decades. Here, we use a genetically encoded DA sensor, GRABDA2m, to record DA activity in the nucleus accumbens (NAc) core during various social behaviors in male and female mice. We find that DA releases during approach, investigation and consummation phases of social behaviors signal animals' motivation, familiarity of the social target, and valence of the experience, respectively. Positive and negative social experiences evoke opposite DA patterns. Furthermore, DA releases during mating and fighting are sexually dimorphic with a higher level in males than in females. At the functional level, increasing DA in NAc enhances social interest toward a familiar conspecific and alleviates defeat-induced social avoidance. Altogether, our results reveal complex information encoded by NAc DA activity during social behaviors and their multistage functional roles.

The Neurobiology of Pathological Fatigue: New Models, New Questions

The last decade has seen the emergence of new theoretical frameworks to explain pathological fatigue, a much neglected, yet highly significant symptom across a wide range of diseases. While the new models of fatigue provide new hypotheses to test, they also raise a number of questions. The primary purpose of this essay is to examine the predictions of three recently proposed models of fatigue, the overlap and differences between them, and the evidence from diseases that may lend support to the models of fatigue. I also present expansions for the sensory attenuation model of fatigue. Further questions examined here are the following: What are the neural substrates of fatigue? How can sensory attenuation, which underpins agency also explain fatigue? Are fatigue and agency related?

Covariations between pupil diameter and supplementary eye field activity suggest a role in cognitive effort implementation

In both human and nonhuman primates (NHP), the medial prefrontal region, defined as the supplementary eye field (SEF), can indirectly influence behavior selection through modulation of the primary selection process in the oculomotor structures. To perform this oculomotor control, SEF integrates multiple cognitive signals such as attention, memory, reward, and error. As changes in pupil responses can assess these cognitive efforts, a better understanding of the precise dynamics by which pupil diameter and medial prefrontal cortex activity interact requires thorough investigations before, during, and after changes in pupil diameter. We tested whether SEF activity is related to pupil dynamics during a mixed pro/antisaccade oculomotor task in 2 macaque monkeys. We used functional ultrasound (fUS) imaging to examine temporal changes in brain activity at the 0.1-s time scale and 0.1-mm spatial resolution concerning behavioral performance and pupil dynamics. By combining the pupil signals and real-time imaging of NHP during cognitive tasks, we were able to infer localized cerebral blood volume (CBV) responses within a restricted part of the dorsomedial prefrontal cortex, referred to as the SEF, an area in which antisaccade preparation activity is also recorded. Inversely, SEF neurovascular activity measured by fUS imaging was found to be a robust predictor of specific variations in pupil diameter over short and long-time scales. Furthermore, we directly manipulated pupil diameter and CBV in the SEF using reward modulations. These results bring a novel understanding of the physiological links between pupil and SEF, but it also raises questions about the role of anterior cingulate cortex (ACC), as CBV variations in the ACC seems to be negligible compared to CBV variations in the SEF.

Ultrafast functional imaging reveals short- and long-term covariations between pupil diameter and activity in the Supplementary Eye Field (SEF) of awake behaving non-human primates, yielding a novel understanding of the physiological links between the pupil and SEF.

Disrupted reward processing in Parkinson's disease and its relationship with dopamine state and neuropsychiatric syndromes: a systematic review and meta-analysis

BACKGROUND

Neuropsychiatric symptoms are common in Parkinson's disease (PD) and predict poorer outcomes. Reward processing dysfunction is a candidate mechanism for the development of psychiatric symptoms including depression and impulse control disorders (ICDs). We aimed to determine whether reward processing is impaired in PD and its relationship with neuropsychiatric syndromes and dopamine replacement therapy.

METHODS

The Ovid MEDLINE/PubMed, Embase and PsycInfo databases were searched for articles published up to 5 November 2020. Studies reporting reward processing task performance by patients with PD and healthy controls were included. Summary statistics comparing reward processing between groups were converted to standardised mean difference (SMD) scores and meta-analysed using a random effects model.

RESULTS

We identified 55 studies containing 2578 participants (1638 PD and 940 healthy controls). Studies assessing three subcomponent categories of reward processing tasks were included: option valuation (n=12), reinforcement learning (n=37) and reward response vigour (n=6). Across all studies, patients with PD on medication exhibited a small-to-medium impairment versus healthy controls (SMD=0.34; 95% CI 0.14 to 0.53), with greater impairments observed off dopaminergic medication in within-subjects designs (SMD=0.43, 95% CI 0.29 to 0.57). Within-subjects subcomponent analysis revealed impaired processing off medication on option valuation (SMD=0.57, 95% CI 0.39 to 0.75) and reward response vigour (SMD=0.36, 95% CI 0.13 to 0.59) tasks. However, the opposite applied for reinforcement learning, which relative to healthy controls was impaired on-medication (SMD=0.45, 95% CI 0.25 to 0.65) but not off-medication (SMD=0.28, 95% CI -0.03 to 0.59). ICD was the only neuropsychiatric syndrome with sufficient studies (n=13) for meta-analysis, but no significant impairment was identified compared tonon-ICD patients (SMD=-0.02, 95% CI -0.43 to 0.39).

CONCLUSION

Reward processing disruption in PD differs according to subcomponent and dopamine medication state, and warrants further study as a potential treatment target and mechanism underlying associated neuropsychiatric syndromes.

Chronic Fatigue in Cancer, Brain Connectivity and Reluctance to Engage in Physical Activity: A Mini-Review

A large amount of evidence shows that after a cancer diagnosis, patients significantly reduce their level of physical activity. Usually, this reduction is attributed to cancer-related fatigue. However, to our knowledge, no study has clearly demonstrated that fatigue alters effort-based decision-making in cancer. This mini-review aimed to provide evidence that chronic fatigue in cancer patients causes changes in brain connectivity that impact effort-based decision-making. Indeed, three patterns of activation to compensate for dysfunctional networks have been reported: greater variability in the executive network and hyperactivation in the executive network, which account for less efficient and costly processes in the frontal cortex, and reduced deactivation in the default mode network. Nevertheless, these activation patterns are also observed with other factors, such as anticipatory stressors (worry, rumination or sleep loss), that might also cause reluctance to engage in physical activity. Effort-based decision-making involving weighing costs against benefits and physical activity interventions should increase immediate benefits to facilitate engagement in effortful activities.

The neural basis of effort valuation: A meta-analysis of functional magnetic resonance imaging studies

Choosing how much effort to expend is critical for everyday decisions. While several neuroimaging studies have examined effort-based decision-making, results have been highly heterogeneous, leaving unclear which brain regions process effort-related costs and integrate them with rewards. We conducted two meta-analyses of functional magnetic resonance imaging data to examine consistent neural correlates of effort demands (23 studies, 15 maps, 549 participants) and net value (15 studies, 11 maps, 428 participants). The pre-supplementary motor area (pre-SMA) scaled positively with pure effort demand, whereas the ventromedial prefrontal cortex (vmPFC) showed the opposite effect. Moreover, regions that have been previously implicated in value integration in other cost domains, such as the vmPFC and ventral striatum, were consistently involved in signaling net value. The opposite response patterns of the pre-SMA and vmPFC imply that they are differentially involved in the representation of effort costs and value integration. These findings provide conclusive evidence that the vmPFC is a central node for net value computation and reveal potential brain targets to treat motivation-related disorders.

Deriving Mental Energy From Task Completion

Many tasks in everyday life (e.g., making an accurate decision, completing job tasks, and searching for product information) are extrinsically motivated (i.e., the task is performed to gain a benefit) and require mental effort. Prior research shows that the cognitive resources needed to perform an extrinsically motivated task are allocated pre-task. The pre-task allocation of mental resources tends to be conservative, because mental effort is costly. Consequently, there are mental energy deficits when the use of mental resources exceeds the allocated amount. This research provides evidence for post-task mental energy replenishment. The amount of resource replenishment is a function of the size of the mental energy deficit and the favorability of the cost-benefit trade-off experienced at the completion of the task (i.e., the value of the reward given the energy investment). The findings have implications for how cognitive resources management influences the availability of mental energy on a moment-to-moment basis.

A mosaic of cost-benefit control over cortico-striatal circuitry

Dopamine contributes to cognitive control through well-established effects in both the striatum and cortex. Although earlier work suggests that dopamine affects cognitive control capacity, more recent work suggests that striatal dopamine may also impact on cognitive motivation. We consider the emerging perspective that striatal dopamine boosts control by making people more sensitive to the benefits versus the costs of cognitive effort, and we discuss how this sensitivity shapes competition between controlled and prepotent actions. We propose that dopamine signaling in distinct cortico-striatal subregions mediates different types of cost-benefit tradeoffs, and also discuss mechanisms for the local control of dopamine release, enabling selectivity among cortico-striatal circuits. In so doing, we show how this cost-benefit mosaic can reconcile seemingly conflicting findings about the impact of dopamine signaling on cognitive control.

Rewards that are near increase impulsive action

In modern society, the natural drive to behave impulsively in order to obtain rewards must often be curbed. A continued failure to do so is associated with a range of outcomes including drug abuse, pathological gambling, and obesity. Here, we used virtual reality technology to investigate whether spatial proximity to rewards has the power to exacerbate the drive to behave impulsively toward them. We embedded two behavioral tasks measuring distinct forms of impulsive behavior, impulsive action, and impulsive choice, within an environment rendered in virtual reality. Participants responded to three-dimensional cues representing food rewards located in either near or far space. Bayesian analyses revealed that participants were significantly less able to stop motor actions when rewarding cues were near compared with when they were far. Since factors normally associated with proximity were controlled for, these results suggest that proximity plays a distinctive role in driving impulsive actions for rewards.

The effort-doors task: Examining the temporal dynamics of effort-based reward processing using ERPs

Aberrant reward processing is a cardinal feature of various forms of psychopathology. However, recent research indicates that aberrant reward processing may manifest at temporally distinct substages and involve interdependent subcomponents of reward processing. To improve our understanding of both the temporal dynamics and distinct subcomponents of reward processing, we added an effort manipulation to the "doors" reward-task paradigm, to derive behavioral and event-related potential (ERP) measures of effort-based reward processing. Behavioral measures consisting of reaction time, response rate, and response rate change were used to index effort expenditure, and ERP measures were used to index attention allocated toward effort-completion cues, anticipation of reward, valuation of reward, and attention toward monetary feedback. Reduced response rate and slowing of response were evident during the high effort versus the low effort condition. ERP findings indicated increased attention to signals of high- compared to low-effort completion cues-as well as reduced anticipation of rewards, and reduced attention toward feedback information following high effort expenditure. Participants showing the most response-rate slowing evidenced the greatest reward devaluation following high versus low effort. Findings demonstrate that the addition of an effort expenditure manipulation to the doors reward paradigm produced reliable ERP and behavioral measures of effort-based reward processing, providing opportunities for future researchers to utilize the effort-doors task to parse the temporal dynamics of both anticipatory and consummatory reward processing components.

Using pharmacological manipulations to study the role of dopamine in human reward functioning: A review of studies in healthy adults

Dopamine (DA) plays a key role in reward processing and is implicated in psychological disorders such as depression, substance use, and schizophrenia. The role of DA in reward processing is an area of highly active research. One approach to this question is drug challenge studies with drugs known to alter DA function. These studies provide good experimental control and can be performed in parallel in laboratory animals and humans. This review aimed to summarize results of studies using pharmacological manipulations of DA in healthy adults. 'Reward' is a complex process, so we separated 'phases' of reward, including anticipation, evaluation of cost and benefits of upcoming reward, execution of actions to obtain reward, pleasure in response to receiving a reward, and reward learning. Results indicated that i) DAergic drugs have different effects on different phases of reward; ii) the relationship between DA and reward functioning appears unlikely to be linear; iii) our ability to detect the effects of DAergic drugs varies depending on whether subjective, behavioral, imaging measures are used.

Neural synchronization between the anterior cingulate and orbitofrontal cortices during effort-based decision making

Optimal decision making reflects the ability to choose the most advantageous option for various alternatives so that the anterior cingulate cortex is an important area involved in effort-based decision making. The current study aimed to investigate the functional connectivity between the ACC (anterior cingulate cortex) and the orbitofrontal cortex (OFC) during effort-based decision-making. A T-maze decision-making task with different rewards (large vs. small reward) and costs (high vs. low effort) was used, and simultaneously, local field potentials (LFP) from the ACC and OFC were also recorded in male Wistar rats. During the effort-based decision making, when the animals preferred the higher over, the lower reward, neural synchronization was observed in theta/low beta (4-20 Hz) frequency bands between both of the areas. Also, neural synchronization was not significant when the animals chose a lower reward. High gamma (80-100 Hz) synchrony between the areas was also observed; however, it was not dependent on the animal's decision. In this regard, the present findings revealed that neural synchronization and functional connectivity between the ACC and OFC in the low-frequency range (theta/low beta) is essential during the effort-based decision making.

Pupillary Dilations of Mice Performing a Vibrotactile Discrimination Task Reflect Task Engagement and Response Confidence

Pupillometry, the measure of pupil size and reactivity, has been widely used to assess cognitive processes. Changes in pupil size have been shown to correlate with various behavioral states, both externally and internally induced such as locomotion, arousal, cortical state, and decision-making processes. Besides, these pupillary responses have also been linked to the activity of neuromodulatory systems that modulate attention and perception such as the noradrenergic and cholinergic systems. Due to the extent of processes the pupil reflects, we aimed at further resolving pupillary responses in the context of behavioral state and task performance while recording pupillary transients of mice performing a vibrotactile two-alternative forced-choice task (2-AFC). We show that before the presentation of task-relevant information, pre-stimulus, pupil size differentiates between states of disengagement from task performance vs. engagement. Also, when subjects have to attend to task stimuli to attain a reward, post-stimulus, pupillary dilations exhibit a difference between correct and error responses with this difference reflecting an internal decision variable. We hypothesize that this internal decision variable relates to response confidence, the internal perception of the confidence the subject has in its choice. As opposed to this, we show that in a condition of passive performance, when the stimulus has no more task relevance due to reward being provided automatically, pupillary dilations reflect the occurrence of stimulation and reward provision but not decisional variables as under active performance. Our results provide evidence that in addition to reflecting attentiveness under task performance rather than arousal per se, pupil dilations also reflect the confidence of the subject in his ensuing response. This confidence coding is overlaid within a more pronounced pupil dilation that reflects post-decision components that are related to the response itself but not to the decision. We also provide evidence as to how different behavioral states, imposed by task demands, modulate what the pupil is reflecting, presumably showing what the underlying cognitive network is coding for.

EEG correlates of physical effort and reward processing during reinforcement learning

Effort-based decision making is often modeled using subjective value, a function of reward discounted by effort. We asked whether EEG event-related potential (ERP) correlates of reward processing are also modulated by physical effort. Human participants performed a task in which they were required to accurately produce target levels of muscle activation to receive rewards. Quadriceps muscle activation was recorded with electromyography (EMG) during isometric knee extension. On a given trial, the target muscle activation required either low or high effort. The effort was determined probabilistically according to a binary choice, such that the responses were associated with 20% and 80% probability of high effort. This contingency could only be learned through experience, and it reversed periodically. Binary reinforcement feedback depended on accurately producing the target muscle activity. Participants adaptively avoided effort by switching responses more frequently after choices that resulted in hard effort. Feedback after participants' choices that revealed the resulting effort requirement did not elicit modulation of the feedback-related negativity/reward positivity (FRN/RP). However, the neural response to reinforcement outcome after effort production was increased by preceding physical effort. Source decomposition revealed separable early and late positive deflections contributing to the ERP. The main effect of reward outcome, characteristic of the FRN/RP, loaded onto the earlier component, whereas the reward × effort interaction was observed only in the later positivity, which resembled the P300. Thus, retrospective effort modulates reward processing. This may explain paradoxical behavioral findings whereby rewards requiring more effort to obtain can become more powerful reinforcers.NEW & NOTEWORTHY Choices probabilistically determined the physical effort requirements for a subsequent task, and reward depended on task performance. Feedback revealing whether choices resulted in easy or hard effort did not elicit reinforcement learning signals. However, the neural responses to reinforcement were modulated by preceding effort. Thus, effort itself was not treated as loss or punishment, but it affected the responses to subsequent reinforcement outcomes. This may explain how effort can enhance the motivational effect of reward.

What you want may not be what you like: A test of the aberrant salience hypothesis in schizophrenia risk

Motivational abnormalities represent a key area of dysfunction in individuals with, or at risk for, schizophrenia and severely limit broad domains of functioning in these populations. The aberrant salience hypothesis posits that motivational abnormalities are the result of an over-attribution of salience to nonpleasurable stimuli but an under-attribution of salience to pleasurable ones. Consequently, people "want" what they do not "like" but do not "want" what they "like." However, it is unclear how this hypothesis manifests in schizophrenia risk beyond monetary rewards. The current research provided a multimodal investigation of the aberrant salience hypothesis in people with elevated psychotic-like experiences (PLEs) who are at risk for developing psychosis. Study 1 examined the link between liking and incentive salience using a neurobiological indicator of incentive salience (contingent negative variation/CNV) in 23 PLEs and 21 Control participants. The PLEs group showed diminished CNV reactivity to pleasant (vs. neutral) social images, which was driven by an augmented response to neutral stimuli. Study 2 examined liking, incentive salience, and conscious wanting experience using a psychological indicator of incentive salience (positive spontaneous thoughts/PSTs) in 38 PLEs and 246 Control participants. The PLEs group showed diminished correspondence between liking, PSTs, and conscious wanting across diverse reward contexts. Collectively, individuals with PLEs over-attribute salience to neutral stimuli and, to a lesser degree, under-attribute salience to rewards. Findings of the current research support abnormal salience attribution as a trait-like feature implicated in the pathophysiology and development of schizophrenia and provide valuable insights on research and treatment of this illness.

Dopamine Modulates Dynamic Decision-Making during Foraging

The mesolimbic dopaminergic system exerts a crucial influence on incentive processing. However, the contribution of dopamine in dynamic, ecological situations where reward rates vary, and decisions evolve over time, remains unclear. In such circumstances, current (foreground) reward accrual needs to be compared continuously with potential rewards that could be obtained by traveling elsewhere (background reward rate), to determine the opportunity cost of staying versus leaving. We hypothesized that dopamine specifically modulates the influence of background, but not foreground, reward information when making a dynamic comparison of these variables for optimal behavior. On a novel foraging task based on an ecological account of animal behavior (marginal value theorem), human participants of either sex decided when to leave locations in situations where foreground rewards depleted at different rates, either in rich or poor environments with high or low background reward rates. In line with theoretical accounts, people's decisions to move from current locations were independently modulated by changes in both foreground and background reward rates. Pharmacological manipulation of dopamine D2 receptor activity using the agonist cabergoline significantly affected decisions to move on, specifically modulating the effect of background reward rates. In particular, when on cabergoline, people left patches in poor environments much earlier. These results demonstrate a role of dopamine in signaling the opportunity cost of rewards, not value per se. Using this ecologically derived framework, we uncover a specific mechanism by which D2 dopamine receptor activity modulates decision-making when foreground and background reward rates are dynamically compared.SIGNIFICANCE STATEMENT Many decisions, across economic, political, and social spheres, involve choices to "leave". Such decisions depend on a continuous comparison of a current location's value, with that of other locations you could move on to. However, how the brain makes such decisions is poorly understood. Here, we developed a computerized task, based around theories of how animals make decisions to move on when foraging for food. Healthy human participants had to decide when to leave collecting financial rewards in a location, and travel to collect rewards elsewhere. Using a pharmacological manipulation, we show that the activity of dopamine in the brain modulates decisions to move on, with people valuing other locations differently depending on their dopaminergic state.

Older adults show a reduced tendency to engage in context-dependent decision biases

When we make decisions, we usually consider the context. This can sometimes lead to suboptimal choices or choice abnormalities. One such abnormality is the compromise effect, according to which deciders tend to favour options positioned as a compromise in an available set of extreme options. Theoretical accounts consider that these effects relate to available cognitive resources, which, in turn, have been found to depend on an individual's dopaminergic innervation. Referring to a correlative triad between cognition, dopamine and aging, the present study demonstrates that the compromise effect is replicable in a group of younger adults (n = 27, 20-32 years of age) yet is attenuated in older adults (n = 27, 62-80 years of age). Results from an [¹⁸F]-FDOPA-PET analysis in older adults indicate a positive association between older adults' inclination to engage in compromise effects and their striatal dopamine synthesis capacity. These results demonstrate altered context-dependent decision biases in older adults and suggest a neuromodulatory mechanism underlying this irregular choice.

Redefining the relationship between effort and reward: Choice-execution model of effort-based decisions

Neuroscientific studies reliably demonstrate that rewards play a crucial role in guiding our choices when confronted with different effortful actions we could make. At the same time, psychological and economic research shows that effort we exert is not reliably predicted by the rewards we end up receiving. Why the mismatch between the two lines of evidence? Inspired by neuroscientific literature, we argue that value-based models of decision-making expose the complexity of the relationship between effort and reward, which changes between two crucial stages of the effort-based decision making process: Choice (i.e. action selection) and Execution (i.e. action execution involving actual effort exertion). To test this assumption, in the present study we set up two experiments (E1: N = 72, E2: N = 87), using a typical neuroscientific effort-based decision-making task. The findings of these experiments reveal that when making prospective choices, rewards do guide the level of effort people are prepared to exert, consistent with typical findings from Neuroscience. At a later stage, during execution of effortful actions, performance is determined by the actual amount of effort that needs to be exerted, consistent with psychological and behavioral economic research. We use the model we tested and the findings we generated to highlight critical new insights into effort-reward relationship, bringing different literatures together in the context of questions regarding what effort its, and the role that values play.

Effort-based decision making varies by smoking status

RATIONALE

A reduced willingness to perform effort based on the magnitude and probability of potential rewards has been associated with diminished dopamine function and may be relevant to chronic drug use.

OBJECTIVES

Here, we investigated the influence of smoking status on effort-based decisions. We hypothesized that smokers would make fewer high-effort selections than ex-smokers and never-smokers.

METHODS

Current smokers (n = 25), ex-smokers (≥ 1 year quit, n = 23), and never-smokers (n = 19) completed the Effort Expenditure for Rewards Task in which participants select between low-effort and high-effort options to receive monetary rewards at varying levels of reward magnitude, probability and expected value.

RESULTS

Overall, participants selected more high-effort options as potential reward magnitude and expected value increased. Smokers did not make fewer high-effort selections overall, but smokers were less sensitive to the changes in magnitude, probability, and expected value compared to never-smokers. Smokers were also less sensitive to the changes in probability and expected value, but not magnitude, compared to ex-smokers. Among smokers and ex-smokers, less nicotine dependence was associated with an increased likelihood of high-effort selections.

CONCLUSIONS

These results demonstrate the relevance of smoking status to effort-based decisions and suggest that smokers have diminished sensitivity to nondrug reward value. Among ex-smokers, greater pre-existing sensitivity to reward value may have been conducive to smoking cessation, or sensitivity was improved by smoking cessation. Future prospective studies can investigate whether effort-related decision making is predictive of smoking initiation or cessation success.

IMPLICATIONS

Willingness to perform effort to achieve a goal and sensitivity to changes in reward value are important aspects of motivation. These results showed that smokers have decreased sensitivity to changes in effort-related reward probability and expected value compared to ex-smokers and never-smokers. Potentially, improved sensitivity to rewards among ex-smokers may be a cause or consequence of smoking cessation. These findings may help explain why some smokers are able to achieve long-term abstinence.

Signals of anticipation of reward and of mean reward rates in the human brain

Theoretical models of dopamine function stemming from reinforcement learning theory have emphasized the importance of prediction errors, which signal changes in the expectation of impending rewards. Much less is known about the effects of mean reward rates, which may be of motivational significance due to their role in computing the optimal effort put into exploiting reward opportunities. Here, we used a reinforcement learning model to design three functional neuroimaging studies and disentangle the effects of changes in reward expectations and mean reward rates, showing recruitment of specific regions in the brainstem regardless of prediction errors. While changes in reward expectations activated ventral striatal areas as in previous studies, mean reward rates preferentially modulated the substantia nigra/ventral tegmental area, deep layers of the superior colliculi, and a posterior pontomesencephalic region. These brainstem structures may work together to set motivation and attentional efforts levels according to perceived reward opportunities.

Reward presentation reduces on-task fatigue in traumatic brain injury

While cognitive fatigue is experienced by up to 80% of individuals with traumatic brain injury (TBI), little is known about its neural underpinnings. We previously hypothesized that presentation of rewarding outcomes leads to cognitive fatigue reduction and activation of the striatum, a brain region shown to be associated with cognitive fatigue in clinical populations and processing of rewarding outcomes. We have demonstrated this in individuals with multiple sclerosis. Here, we tested this hypothesis in individuals with TBI. Twenty-one individuals with TBI and 24 healthy participants underwent functional magnetic resonance imaging. Participants performed a task during which they were presented with 1) the Outcome condition where they were exposed to monetary rewards, and 2) the No Outcome condition that served as the control condition and was not associated with monetary rewards. In accordance with our hypothesis, results showed that attainment of rewarding outcomes leads to cognitive fatigue reduction in individuals with TBI, as well as activation of the striatum. Specifically, we observed a significant group by condition interaction on fatigue scores driven by the TBI group reporting lower levels of fatigue after the Outcome condition. fMRI data revealed a significant main-effect of condition in regions previously implicated in outcome processing, while a significant group by condition interaction was observed in the left ventral striatum as revealed by a priori region of interest analysis. Results suggest that a salient motivator can significantly reduce fatigue and that outcome presentation leads to increased activation of the ventral striatum in TBI. These findings can inform the development of future non-pharmacological cognitive fatigue treatment methods and contribute to the growing body of evidence showing the association between cognitive fatigue and the striatum.

An Integrative Model of Effortful Control

This article presents an integrative model of effortful control, a resource-limited top-down control mechanism involved in mental tasks and physical exercises. Based on recent findings in the fields of neuroscience, social psychology and cognitive psychology, this model posits the intrinsic costs related to a weakening of the connectivity of neural networks underpinning effortful control as the main cause of mental fatigue in long and high-demanding tasks. In this framework, effort reflects three different inter-related aspects of the same construct. First, effort is a mechanism comprising a limited number of interconnected processing units that integrate information regarding the task constraints and subject’s state. Second, effort is the main output of this mechanism, namely, the effort signal that modulates neuronal activity in brain regions involved in the current task to select pertinent information. Third, effort is a feeling that emerges in awareness during effortful tasks and reflects the costs associated with goal-directed behavior. Finally, the model opens new avenues for research investigating effortful control at the behavioral and neurophysiological levels.

Dopamine-Dependent Loss Aversion during Effort-Based Decision-Making

From psychology to economics, there has been substantial interest in how costs (e.g., delay, risk) are represented asymmetrically during decision-making when attempting to gain reward or avoid punishment. For example, in decision-making under risk, individuals show a tendency to prefer to avoid punishment rather than to acquire the equivalent reward (loss aversion). Although the cost of physical effort has recently received significant attention, it remains unclear whether loss aversion exists during effort-based decision-making. On the one hand, loss aversion may be hardwired due to asymmetric evolutionary pressure on losses and gains and therefore exists across decision-making contexts. On the other hand, distinct brain regions are involved with different decision costs, making it questionable whether similar asymmetries exist. Here, we demonstrate that young healthy human participants (females, 16; males, 6) exhibit loss aversion during effort-based decision-making by exerting more physical effort to avoid punishment than to gain a same-size reward. Next, we show that medicated Parkinson's disease (PD) patients (females, 9; males, 9) show a reduction in loss aversion compared with age-matched control subjects (females, 11; males, 9). Behavioral and computational analysis revealed that people with PD exerted similar physical effort in return for a reward but were less willing to produce effort to avoid punishment. Therefore, loss aversion is present during effort-based decision-making and can be modulated by altered dopaminergic state. This finding could have important implications for our understanding of clinical disorders that show a reduced willingness to exert effort in the pursuit of reward.SIGNIFICANCE STATEMENT Loss aversion-preferring to avoid punishment rather than to acquire equivalent reward-is an important concept in decision-making under risk. However, little is known about whether loss aversion also exists during decisions where the cost is physical effort. This is surprising given that motor cost shapes human behavior, and a reduced willingness to exert effort is a characteristic of many clinical disorders. Here, we show that healthy human individuals exert more effort to minimize punishment than to maximize reward (loss aversion). We also demonstrate that medicated Parkinson's disease patients exert similar effort to gain reward but less effort to avoid punishment when compared with healthy age-matched control subjects. This indicates that dopamine-dependent loss aversion is crucial for explaining effort-based decision-making.

Strive to Win or Not to Lose? Age-Related Differences in Framing Effects on Effort-Based Decision-Making

Objectives

This study sought to assess how framing effects modulate age-related differences in effort-based decision-making. Consistent with the selection, optimization, and compensation (SOC) model’s loss prevention account of aging, we predicted that older adults would be more willing to select high-effort options in loss contexts than gain contexts.

Method

Older and younger adults completed the effort expenditure for rewards task (EEfRT) in either a gain or loss context. The EEfRT is an effort-based decision-making paradigm in which participants choose between a low-effort, “easy” option and a high-effort, “hard” option for several trials. The probability and value of an outcome varies on a trial-by-trial basis.

Results

The results supported our prediction and the SOC model. Older adults chose more high-effort, difficult options in loss frames than gain frames. Older adults also chose more low-effort, easy options than younger adults in gain contexts, but did not differ from younger adults in loss contexts.

Discussion

These findings demonstrate that framing effects impact older adults’ effort-based decisions. Older adults appear willing to incur a greater “cost” in the form of effort to prevent a loss than to attain a reward.

Striatal responsiveness to reward under threat-of-shock and working memory load: A preliminary study

INTRODUCTION

Reward and stress are important determinants of motivated behaviors. Striatal regions play a crucial role in both motivation and hedonic processes. So far, little is known on how cognitive effort interacts with stress to modulate reward processes. This study examines how cognitive effort (load) interacts with an unpredictable acute stressor (threat-of-shock) to modulate motivational and hedonic processes in healthy adults.

MATERIALS AND METHODS

A reward task, involving stress with unpredictable mild electric shocks, was conducted in 23 healthy adults aged 20-37 (mean age: 24.7 ± 0.9; 14 females) during functional magnetic resonance imaging (fMRI). Manipulation included the use of (a) monetary reward for reinforcement, (b) threat-of-shock as the stressor, and (c) a spatial working memory task with two levels of difficulty (low and high load) for cognitive load. Reward-related activation was investigated in a priori three regions of interest, the nucleus accumbens (NAcc), caudate nucleus, and putamen.

RESULTS

During anticipation, threat-of-shock or cognitive load did not affect striatal responsiveness to reward. Anticipated reward increased activation in the ventral and dorsal striatum. During feedback delivery, both threat-of-shock and cognitive effort modulated striatal activation. Higher working memory load blunted NAcc responsiveness to reward delivery, while stress strengthened caudate nucleus reactivity regardless reinforcement or load.

CONCLUSIONS

These findings provide initial evidence that both stress and cognitive load modulate striatal responsiveness during feedback delivery but not during anticipation in healthy adults. Of clinical importance, sustained stress exposure might go along with dysregulated arousal, increasing therefore the risk for the development of maladaptive incentive-triggered motivation. This study brings new insight that might help to build a framework to understand common stress-related disorders, given that these psychiatric disorders involve disturbances of the reward system, cognitive deficits, and abnormal stress reactivity.

An amygdala-cingulate network underpins changes in effort-based decision making after a fitness program

When humans make decisions, objective rewards are mainly discounted by delay, risk and effort. Whereas recent research has demonstrated that several brain areas process costs and code subjective value in effort-based decision making, it remains obscure how neural activity patterns change when effort costs are reduced due to the acquisition of healthy habits, such as moving from sedentary to active lifestyles. Here, a sample of sedentary volunteers was behaviorally assessed and fMRI-scanned before and after completing a 3-month fitness plan. The impact of effort cost on decisions, measured as the constant defining a hyperbolic decaying function, was reduced after the plan. A logistic mixed model demonstrated that the explanatory power of effort decreased with time. At a neural level, there was a marginally significant disruption of effort-cost related functional activity in the anterior cingulate after the plan. Functional connectivity between the amygdala and anterior cingulate cortex was strengthened after habit acquisition. In turn, this interaction was stronger in those participants with lower effort discounting. Thus, we show for the first time changes in value-based decision making after moving from a sedentary to an active lifestyle, which points to the relevance of the amygdala-cingulate interplay when the impact of effort on decisions fades away.

Effort-based decision-making impairment in patients with clinically-stabilized first-episode psychosis and its relationship with amotivation and psychosocial functioning

Effort-based decision-making has recently been proposed as a potential mechanism contributing to motivational deficits (amotivation) in psychotic disorder. Previous research has identified altered effort allocation in chronic schizophrenia, but produced mixed results regarding its relationship with amotivation. No study has investigated effort allocation in first-episode psychosis (FEP). We examined effort-based decision-making in 45 clinically-stabilized FEP patients and 45 demographically-matched controls, using Effort-Expenditure for Reward Task (EEfRT) which measures allocation of physical effort for monetary reward at varying magnitude and probability levels. Our results showed that FEP patients did not demonstrate overall reduction in effort expenditure but displayed reduced willingness to expend effort for high-value/high-probability reward as compared to controls. In particular, such selective effort-related impairment was most pronounced in patients with high levels of amotivation. Furthermore, reduced allocation of greater effort for higher probability reward was related to poorer psychosocial functioning. Decreased effort exertion was generally unrelated to other symptom dimensions, self-report anhedonia, antipsychotic dose and cognitive deficits. This study thus provides the first evidence of effort-based decision-making impairment in FEP, and indicates that first-episode patients were not generally effort-averse but exhibited inefficient effort allocation by failing to make high-effort choices to maximize reward receipt. Our findings affirm the critical role of amotivation on aberrant effort allocation, and support the link between laboratory-based effort-cost measures and real-world psychosocial functioning in medicated FEP. Further longitudinal research is required to clarify trajectory of suboptimal effort allocation and its potential utility in predicting amotivation and functional outcomes in the early course of illness.

Brain mechanisms underlying apathy

The past few decades have seen growing interest in the neuropsychiatric syndrome of apathy, conceptualised as a loss of motivation manifesting as a reduction of goal-directed behaviour. Apathy occurs frequently, and with substantial impact on quality of life, in a broad range of neurological and psychiatric conditions. Apathy is also consistently associated with neuroimaging changes in specific medial frontal cortex and subcortical structures, suggesting that disruption of a common systems-level mechanism may underlie its development, irrespective of the condition that causes it. In parallel with this growing recognition of the clinical importance of apathy, significant advances have been made in understanding normal motivated behaviour in humans and animals. These developments have occurred at several different conceptual levels, from work linking neural structures and neuromodulatory systems to specific aspects of motivated behaviour, to higher order computational models that aim to unite these findings within frameworks for normal goal-directed behaviour. In this review we develop a conceptual framework for understanding pathological apathy based on this current understanding of normal motivated behaviour. We first introduce prominent theories of motivated behaviour-which often involves sequences of actions towards a goal that needs to be maintained across time. Next, we outline the behavioural effects of disrupting these processes in animal models, highlighting the specific effects of these manipulations on different components of motivated behaviour. Finally, we relate these findings to clinical apathy, demonstrating the homologies between this basic neuroscience work and emerging behavioural and physiological evidence from patient studies of this syndrome.

A Reward-Based Framework of Perceived Control

Perceived control can be broadly defined as the belief in one's ability to exert control over situations or events. It has long been known that perceived control is a major contributor toward mental and physical health as well as a strong predictor of achievements in life. However, one issue that limits a mechanistic understanding of perceived control is the heterogeneity of how the term is defined in models in psychology and neuroscience, and used in experimental settings across a wide spectrum of studies. Here, we propose a framework for studying perceived control by integrating the ideas from traditionally separate work on perceived control. Specifically, we discuss key properties of perceived control from a reward-based framework, including choice opportunity, instrumental contingency, and success/reward rate. We argue that these separate reward-related processes are integral to fostering an enhanced perception of control and influencing an individual's behavior and well-being. We draw on select studies to elucidate how these reward-related elements are implicated separately and collectively in the investigation of perceived control. We highlight the role of dopamine within corticostriatal pathways shared by reward-related processes and perceived control. Finally, through the lens of this reward-based framework of perceived control, we consider the implications of perceived control in clinical deficits and how these insights could help us better understand psychopathology and treatment options.

Cue awareness in avoiding effortful control

Based on a recent metacognitive account, cognitive effort is the result of an inferential evaluation made over explicitly available cues. Following from this account, we present here a pre-registered experiment that tested the specific hypothesis that explicit awareness of cues that are aligned with cognitive demand is a prerequisite in avoiding effortful lines of action. We attempted to modulate levels of effort avoidance behavior by introducing an incentive (between-subjects) to monitor two lines of action that, unbeknownst to individuals, varied in the probability of a task switch. Importantly, previous research has demonstrated that the difference in these probabilities is relatively opaque to individuals. We did not find strong evidence for our incentive manipulation having an effect on demand avoidance as indexed by individuals' choices in a block of the task where avoiding effort was instructed. However, we do find that being aware of the task-switching cue appears to increase the likelihood of demand avoidance. We consider these results within the context of the metacognition of cognitive effort.

Modeling cocaine relapse in rodents: Behavioral considerations and circuit mechanisms

Addiction is a chronic relapsing disorder, in that most addicted individuals who choose to quit taking drugs fail to maintain abstinence in the long-term. Relapse is especially likely when recovering addicts encounter risk factors like small "priming" doses of drug, stress, or drug-associated cues and locations. In rodents, these same factors reinstate cocaine seeking after a period of abstinence, and extensive preclinical work has used priming, stress, or cue reinstatement models to uncover brain circuits underlying cocaine reinstatement. Here, we review common rat models of cocaine relapse, and discuss how specific features of each model influence the neural circuits recruited during reinstated drug seeking. To illustrate this point, we highlight the surprisingly specific roles played by ventral pallidum subcircuits in cocaine seeking reinstated by either cocaine-associated cues, or cocaine itself. One goal of such studies is to identify, and eventually to reverse the specific circuit activity that underlies the inability of some humans to control their drug use. Based on preclinical findings, we posit that circuit activity in humans also differs based on the triggers that precipitate craving and relapse, and that associated neural responses could help predict the triggers most likely to elicit relapse in a given person. If so, examining circuit activity could facilitate diagnosis of subgroups of addicted people, allowing individualized treatment based on the most problematic risk factors.

Dysfunctional effort-based decision-making underlies apathy in genetic cerebral small vessel disease

Apathy is a syndrome of reduced motivation that commonly occurs in patients with cerebral small vessel disease, including those with the early onset form, CADASIL (cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy). The cognitive mechanisms underlying apathy are poorly understood and treatment options are limited. We hypothesized that disrupted effort-based decision-making, the cognitive process by which potential rewards and the effort cost required to obtain them is integrated to drive behaviour, might underlie the apathetic syndrome. Nineteen patients with a genetic diagnosis of CADASIL, as a model of 'pure' vascular cognitive impairment, and 19 matched controls were assessed using two different behavioural paradigms and MRI. On a decision-making task, participants decided whether to accept or reject sequential offers of monetary reward in return for exerting physical effort via handheld dynamometers. Six levels of reward and six levels of effort were manipulated independently so offers spanned the full range of possible combinations. Choice, decision time and force metrics were recorded. Each participant's effort and reward sensitivity was estimated using a computational model of choice. On a separate eye movement paradigm, physiological reward sensitivity was indexed by measuring pupillary dilatation to increasing monetary incentives. This metric was related to apathy status and compared to the behavioural metric of reward sensitivity on the decision-making task. Finally, high quality diffusion imaging and tract-based spatial statistics were used to determine whether tracts linking brain regions implicated in effort-based decision-making were disrupted in apathetic patients. Overall, apathetic patients with CADASIL rejected significantly more offers on the decision-making task, due to reduced reward sensitivity rather than effort hypersensitivity. Apathy was also associated with blunted pupillary responses to incentives. Furthermore, these independent behavioural and physiological markers of reward sensitivity were significantly correlated. Non-apathetic patients with CADASIL did not differ from controls on either task, whilst actual motor performance of apathetic patients in both tasks was also normal. Apathy was specifically associated with reduced fractional anisotropy within tracts connecting regions previously associated with effort-based decision-making. These findings demonstrate behavioural, physiological and anatomical evidence that dysfunctional effort-based decision-making underlies apathy in patients with CADASIL, a model disorder for sporadic small vessel disease. Reduced incentivization by rewards rather than hypersensitivity to effort costs drives this altered pattern of behaviour. The study provides empirical evidence of a cognitive mechanism for apathy in cerebral small vessel disease, and identifies a promising therapeutic target for interventions to improve this debilitating condition.

An Automated T-maze Based Apparatus and Protocol for Analyzing Delay- and Effort-based Decision Making in Free Moving Rodents

Many neurological and psychiatric patients demonstrate difficulties and/or deficits in decision making. Rodent models are helpful to produce a deeper understanding of the neurobiological causes underlying the decision-making problems. A cost-benefit based T-maze task is used for measuring decision making in which rodents choose between a high reward arm (HRA) and a low reward arm (LRA). There are two paradigms of the T-maze decision-making task, one in which the cost is a time delay and the other in which it is physical effort. Both paradigms require a tedious and labor-intensive management of experimental animals, multiple doors, pellet reward, and arm choice recordings. In the current work, we invented an apparatus based on traditional T-maze with full automation for pellet delivery, door management and choice recordings. This automated setup can be used for the evaluation of both delay- and effort-based decision making in rodents. With the protocol described here, our lab investigated the decision-making phenotypes of multiple genetically modified mice. In the representative data, we showed that the mice with ablated medial habenular showed aversions of both delay and effort and tended to choose the immediate and effortless reward. This protocol helps to decrease the variability caused by experimenter intervention and to enhance experiment efficiency. In addition, chronic silicon probe or microelectrode recording, fiber-optic imaging and/or manipulation of neural activity can be easily applied during the decision-making task using the setup described here.

Reward circuitry activation reflects social preferences in the face of cognitive effort

Research at the intersection of social neuroscience and cognitive effort is an interesting new area for exploration. There is great potential to broaden our understanding of how social context and cognitive effort processes, currently addressed in disparate literatures, interact with one another. In this paper, we briefly review the literature on cognitive effort, focusing on effort-linked valuation and the gap in the literature regarding cognitive effort in the social domain. Next, we present a study designed to explore valuation processes linked to cognitive effort within the social context of an inequality manipulation. More specifically, we created monetary inequality among the participant (SELF, endowed with $50) and two confederates: one also endowed with $50 (OTHER HIGH) and another with only $5 (OTHER LOW). We then scanned participants using fMRI as they attempted to earn bonus payments for themselves and others through a cognitively effortful feedback-based learning task. Positive feedback produced significantly greater activation than negative feedback in key valuation regions, the ventral striatum (VS) and ventromedial prefrontal cortex (vmPFC), both when participants were performing the task on their own behalf and when earning rewards for others. While reward-related activity in the VS was exaggerated for SELF compared to OTHER HIGH for both positive and negative feedback, activity in the vmPFC did not distinguish between recipients in the group-level results. Furthermore, participants naturally fell into two groups: those most engaged when playing for themselves and those who reported engagement for others. While Self-Engaged participants showed differences between the SELF and both OTHER conditions in the VS and vmPFC, Other-Engaged participants only showed an attenuated response to negative feedback for OTHER HIGH compared to SELF in the VS and no differences between recipient conditions in the vmPFC. Together, this work shows the importance of individual differences and the fragility of advantageous inequality aversion in the face of cognitive effort, highlighting the need to study cognitive effort in the social domain.

The roles of the anterior cingulate cortex and its dopamine receptors in self-paced cost-benefit decision making in rats

It has been shown that the anterior cingulate cortex (ACC) and its dopamine system are crucial for decision making that requires physical/emotional effort, but not for all forms of cost-benefit decision making. Previous studies had mostly employed behavioral tasks with two competing cost-reward options that were preset by the experimenters. However, few studies have been conducted using scenarios in which the subjects have full control over the energy/time expenditure required to obtain a proportional reward. Here, we assessed the roles of the ACC and its dopamine system in cost-benefit decision making by utilizing a "do more get more" (DMGM) task and a time-reward trade-off (TRTO) task, wherein the animals were able to self-determine how much effort or time to expend at a nosepoke operandum for a proportional reward. Our results showed that (1) ACC inactivation severely impaired DMGM task performance, with a reduction in the rate of correct responses and a decrease in the effort expended, but did not affect the TRTO task; and (2) blocking ACC D2 receptors had no impact on DMGM task performance in the baseline cost-benefit scenario, but it significantly reduced the attempts to invest increased effort for a large reward when the benefit-cost ratio was reduced by half. In contrast, blocking ACC D1 receptors had no effect on DMGM task performance. These findings suggest that the ACC is required for self-paced effort-based but not for time-reward trade-off decision making. Furthermore, ACC dopamine D2 but not D1 receptors are involved in DMGM decision making.

Distinct effects of apathy and dopamine on effort-based decision-making in Parkinson's disease

Effort-based decision-making is a cognitive process crucial to normal motivated behaviour. Apathy is a common and disabling complication of Parkinson's disease, but its aetiology remains unclear. Intriguingly, the neural substrates associated with apathy also subserve effort-based decision-making in animal models and humans. Furthermore, the dopaminergic system plays a core role in motivating effortful behaviour for reward, and its dysfunction has been proposed to play a crucial role in the aetiology of apathy in Parkinson's disease. We hypothesized that disrupted effort-based decision-making underlies the syndrome of apathy in Parkinson's disease, and that this disruption may be modulated by the dopaminergic system. An effort-based decision-making task was administered to 39 patients with Parkinson's disease, with and without clinical apathy, ON and OFF their normal dopaminergic medications across two separate sessions, as well as 32 healthy age- and gender-matched controls. On a trial-by-trial basis, participants decided whether to accept or reject offers of monetary reward in return for exerting different levels of physical effort via handheld, individually calibrated dynamometers. Effort and reward were manipulated independently, such that offers spanned the full range of effort/reward combinations. Apathy was assessed using the Lille apathy rating scale. Motor effects of the dopamine manipulation were assessed using the Unified Parkinson's Disease Rating Scale part three motor score. The primary outcome variable was choice (accept/decline offer) analysed using a hierarchical generalized linear mixed effects model, and the vigour of squeeze (Newtons exerted above required force). Both apathy and dopamine depletion were associated with reduced acceptance of offers. However, these effects were driven by dissociable patterns of responding. While apathy was characterized by increased rejection of predominantly low reward offers, dopamine increased responding to high effort, high reward offers, irrespective of underlying motivational state. Dopamine also exerted a main effect on motor vigour, increasing force production independently of reward offered, while apathy did not affect this measure. The findings demonstrate that disrupted effort-based decision-making underlies Parkinson's disease apathy, but in a manner distinct to that caused by dopamine depletion. Apathy is associated with reduced incentivization by the rewarding outcomes of actions. In contrast, dopamine has a general effect in motivating behaviour for high effort, high reward options without altering the response pattern that characterizes the apathetic state. Thus, the motivational deficit observed in Parkinson's disease appears not to be simply secondary to dopaminergic depletion of mesocorticolimbic pathways, suggesting non-dopaminergic therapeutic strategies for apathy may be important future targets.

Neuropsychiatric Phenotypes Produced by GABA Reduction in Mouse Cortex and Hippocampus

Whereas cortical GAD67 reduction and subsequent GABA level decrease are consistently observed in schizophrenia and depression, it remains unclear how these GABAergic abnormalities contribute to specific symptoms. We modeled cortical GAD67 reduction in mice, in which the Gad1 gene is genetically ablated from ~50% of cortical and hippocampal interneurons. Mutant mice showed a reduction of tissue GABA in the hippocampus and cortex including mPFC, and exhibited a cluster of effort-based behavior deficits including decreased home-cage wheel running and increased immobility in both tail suspension and forced swim tests. Since saccharine preference, progressive ratio responding to food, and learned helplessness task were normal, such avolition-like behavior could not be explained by anhedonia or behavioral despair. In line with the prevailing view that dopamine in anterior cingulate cortex (ACC) plays a role in evaluating effort cost for engaging in actions, we found that tail-suspension triggered dopamine release in ACC of controls, which was severely attenuated in the mutant mice. Conversely, ACC dopamine release by progressive ratio responding to reward, during which animals were allowed to effortlessly perform the nose-poking, was not affected in mutants. These results suggest that cortical GABA reduction preferentially impairs the effort-based behavior which requires much effort with little benefit, through a deficit of ACC dopamine release triggered by high-effort cost behavior, but not by reward-seeking behavior. Collectively, a subset of negative symptoms with a reduced willingness to expend costly effort, often observed in patients with schizophrenia and depression, may be attributed to cortical GABA level reduction.