Behavioral modeling of human choices reveals dissociable effects of physical effort and temporal delay on reward devaluation

Ego depletion does not negatively impact novice sprinters' start performance during a timed 20m all-out sprint

Temporary reductions in self-control due to previous cognitive effort, also known as 'ego depletion', have been shown to negatively impact performance in subsequent tasks requiring self-control. Various theoretical accounts suggest that high levels of task motivation may prevent negative behavioural effects of ego depletion. Against this background, the current study performed a conceptual replication of a previous study examining the effects of ego depletion on sprint start performance (i.e., Englert et al., 2015) and implemented additional monetary incentives to manipulate task motivation. Using a pretest-intervention-posttest design, ninety-six novice sprinters performed a sprinting task that required them to complete a series of 20-m sprints. In between pretest and posttest, participants performed a letter writing task either under high cognitive demand (depletion group) or low cognitive demand (non-depletion control group). Subsequently, and prior to starting the posttest, participants were presented either with a monetary incentive or no incentive. Manipulation checks confirmed that the letter writing task successfully manipulated perceived regulatory demand and that task motivation at the posttest was higher for participants who were offered a monetary incentive. The number of false starts decreased significantly with ego depletion, while sprint start reaction times and sprint finish times remained unaffected. Regardless of ego depletion, sprint finish times significantly improved with monetary incentives. These results suggest that brief engagement in a demanding cognitive task is unlikely to impair novice sprinters' ability to self-regulate during the start of a subsequent, maximal sprinting effort for which they are highly motivated.

Beta and theta oscillations track effort and previous reward in the human basal ganglia and prefrontal cortex during decision making

Choosing whether to exert effort to obtain rewards is fundamental to human motivated behavior. However, the neural dynamics underlying the evaluation of reward and effort in humans is poorly understood. Here, we report an exploratory investigation into this with chronic intracranial recordings from the prefrontal cortex (PFC) and basal ganglia (BG; subthalamic nuclei and globus pallidus) in people with Parkinson's disease performing a decision-making task with offers that varied in levels of reward and physical effort required. This revealed dissociable neural signatures of reward and effort, with BG beta (12 to 20 Hz) oscillations tracking effort on a single-trial basis and PFC theta (4 to 7 Hz) signaling previous trial reward, with no effects of net subjective value. Stimulation of PFC increased overall acceptance of offers and sensitivity to reward while decreasing the impact of effort on choices. This work uncovers oscillatory mechanisms that guide fundamental decisions to exert effort for reward across BG and PFC, supports a causal role of PFC for such choices, and seeds hypotheses for future studies.

The Neurobiology of Cognitive Fatigue and Its Influence on Effort-Based Choice

Feelings of cognitive fatigue emerge through repeated mental exertion and are ubiquitous in our daily lives. However, there is a limited understanding of the neurobiological mechanisms underlying the influence of cognitive fatigue on decisions to exert. We use functional magnetic resonance imaging to examine brain activity while participants make choices to exert effort for reward, before and after bouts of fatiguing cognitive exertion. We found that when participants became cognitively fatigued, they were more likely to choose to forgo higher levels of reward that required more effort. We describe a mechanism by which signals related to cognitive exertion in dlPFC influence effort value computations, instantiated by the insula, thereby influencing an individual's decisions to exert while fatigued. Our results suggest that cognitive fatigue plays a critical role in decisions to exert effort and provides a mechanistic link through which information about cognitive state shapes effort-based choice.

Dopamine and acetylcholine have distinct roles in delay- and effort-based decision-making in humans

In everyday life, we encounter situations that require tradeoffs between potential rewards and associated costs, such as time and (physical) effort. The literature indicates a prominent role for dopamine in discounting of both delay and effort, with mixed findings for delay discounting in humans. Moreover, the reciprocal antagonistic interaction between dopaminergic and cholinergic transmission in the striatum suggests a potential opponent role of acetylcholine in these processes. We found opposing effects of dopamine D2 (haloperidol) and acetylcholine M1 receptor (biperiden) antagonism on specific components of effort-based decision-making in healthy humans: haloperidol decreased, whereas biperiden increased the willingness to exert physical effort. In contrast, delay discounting was reduced under haloperidol, but not affected by biperiden. Together, our data suggest that dopamine, acting at D2 receptors, modulates both effort and delay discounting, while acetylcholine, acting at M1 receptors, appears to exert a more specific influence on effort discounting only.

Cognitive representations of intracranial self-stimulation of midbrain dopamine neurons depend on stimulation frequency

Dopamine neurons in the ventral tegmental area support intracranial self-stimulation (ICSS), yet the cognitive representations underlying this phenomenon remain unclear. Here, 20-Hz stimulation of dopamine neurons, which approximates a physiologically relevant prediction error, was not sufficient to support ICSS beyond a continuously reinforced schedule and did not endow cues with a general or specific value. However, 50-Hz stimulation of dopamine neurons was sufficient to drive robust ICSS and was represented as a specific reward to motivate behavior. The frequency dependence of this effect is due to the rate (not the number) of action potentials produced by dopamine neurons, which differently modulates dopamine release downstream.

Get active now or later? The association between physical activity and risk and time preferences

Despite the well-established link between physical activity and positive health outcomes, much of the world's population remains inactive. Many people don't invest in health behaviours, such as physical activity, in the present, despite the long-term benefits of this. The aim of this study was to assess the relationship between physical activity levels and risk and time preferences in university students. A maximum likelihood model was used to jointly estimate risk and time preferences (elicited in an incentivised choice experiment), and to examine the relationship between these preferences and self-reported physical activity. Physically inactive people discounted the future significantly more than physically active people did. Physically active people made slightly more risky choices in our risk attitude task, although this directional relationship was not statistically significant. The link between time preferences and physical activity suggests that further research on behavioural strategies such as commitment devices, nudging or temptation bundling may be helpful in increasing physical activity for individuals who discount the future in favour of more immediate benefits.

Brain stimulation over the left DLPFC enhances motivation for effortful rewards in patients with major depressive disorder

BACKGROUND

Amotivation is a typical feature in major depressive disorder (MDD), which produces reduced willingness to exert effort. The dorsolateral prefrontal cortex (DLPFC) is a crucial structure in goal-directed actions and therefore is a potential target in modulating effortful motivation. However, it remains unclear whether the intervention is effective for patients with MDD.

METHODS

We employed transcranial magnetic stimulation (TMS), computational modelling and event-related potentials (ERPs) to reveal the causal relationship between the left DLPFC and motivation for effortful rewards in MDD. Fifty patients underwent both active and sham TMS sessions, each followed by performing an Effort-Expenditure for Rewards Task, during which participants chose and implemented between low-effort/low-reward and high-effort/high-reward options.

RESULTS

The patients showed increased willingness to exert effort for rewards during the DLPFC facilitated session, compared with the sham session. They also had a trend in larger P3 amplitude for motivated attention toward chosen options, larger CNV during preparing for effort exertion, and larger SPN during anticipating a high reward. Besides, while behavior indexes for effortful choices were negatively related to depression severity in the sham session, this correlation was weakened in the active stimulation session.

CONCLUSIONS

These findings provide behavioral, computational, and neural evidence for the left DLPFC on effortful motivation for rewards. Facilitated DLPFC improves motor preparation and value anticipation after making decisions especially for highly effortful rewards in MDD. Facilitated DLPFC also has a potential function in enhancing motivated attention during cost-benefit trade-off. This neuromodulation effect provides a potential treatment for improving motivation in clinics.

Effort cost of harvest affects decisions and movement vigor of marmosets during foraging

Our decisions are guided by how we perceive the value of an option, but this evaluation also affects how we move to acquire that option. Why should economic variables such as reward and effort alter the vigor of our movements? In theory, both the option that we choose and the vigor with which we move contribute to a measure of fitness in which the objective is to maximize rewards minus efforts, divided by time. To explore this idea, we engaged marmosets in a foraging task in which on each trial they decided whether to work by making saccades to visual targets, thus accumulating food, or to harvest by licking what they had earned. We varied the effort cost of harvest by moving the food tube with respect to the mouth. Theory predicted that the subjects should respond to the increased effort costs by choosing to work longer, stockpiling food before commencing harvest, but reduce their movement vigor to conserve energy. Indeed, in response to an increased effort cost of harvest, marmosets extended their work duration, but slowed their movements. These changes in decisions and movements coincided with changes in pupil size. As the effort cost of harvest declined, work duration decreased, the pupils dilated, and the vigor of licks and saccades increased. Thus, when acquisition of reward became effortful, the pupils constricted, the decisions exhibited delayed gratification, and the movements displayed reduced vigor.

Explicit effort may not influence perceptuomotor decision-making

Many decisions that humans make are enacted by the action system. For example, humans use reach-to-grasp movements when making perceptuomotor decisions between and obtaining fruits of varying quality from a pile. Recent work suggests that the characteristics of each action alternative may influence the decision itself-there may be a bias away from making perceptuomotor alternatives associated with high effort when participants are unaware of the effort differences between responses. The present study examined if perceptuomotor decisions were influenced by explicit reaching effort differences. Neurotypical human participants were presented with random dot motion stimuli in which most dots moved in random directions and varying percentages of remaining dots moved coherently left- or rightward. Participants reported leftward motion judgements by performing leftward (or left hand) reaching movements and rightward motion judgements by performing rightward (or right hand) reaching movements. A resistance band was affixed to participants' wrists and to the table in different configurations. The configurations allowed for one movement/motion direction judgement to always require stretching of the band and, therefore, require relatively more effort. Across a set of experiments, the response context (i.e. selecting directions within a limb or selecting between limbs) and the effort difference between responses were manipulated. Overall, no experiment revealed a bias away from the perceptuomotor decision associated with high effort. Based on these results, it is concluded that, in this biomechanical context, explicit effort may not influence perceptuomotor decision-making and may point to a contextual influence of action effort on perceptuomotor decision-making.

Need for Cognition is associated with a preference for higher task load in effort discounting

When individuals set goals, they consider the subjective value (SV) of the anticipated reward and the required effort, a trade-off that is of great interest to psychological research. One approach to quantify the SVs of levels of difficulty of a cognitive task is the Cognitive Effort Discounting Paradigm by Westbrook and colleagues (2013). However, it fails to acknowledge the highly individual nature of effort, as it assumes a unidirectional, inverse relationship between task load and SVs. Therefore, it cannot map differences in effort perception that arise from traits like Need for Cognition, since individuals who enjoy effortful cognitive activities likely do not prefer the easiest level. We replicated the analysis of Westbrook and colleagues with an adapted version, the Cognitive and Affective Discounting (CAD) Paradigm. It quantifies SVs without assuming that the easiest level is preferred, thereby enabling the assessment of SVs for tasks without objective order of task load. Results show that many of the 116 participants preferred a more or the most difficult level. Variance in SVs was best explained by a declining logistic contrast of the [Formula: see text]-back levels and by the accuracy of responses, while reaction time as a predictor was highly volatile depending on the preprocessing pipeline. Participants with higher Need for Cognition scores perceived higher [Formula: see text]-back levels as less effortful and found them less aversive. Effects of Need for Cognition on SVs in lower levels did not reach significance, as group differences only emerged in higher levels. The CAD Paradigm appears to be well suited for assessing and analysing task preferences independent of the supposed objective task difficulty.Protocol registrationThe stage 1 protocol for this Registered Report was accepted in principle on August 19, 2022. The protocol, as accepted by the journal, can be found at: https://doi.org/10.17605/OSF.IO/CPXTH .

Computational mechanisms underlying the dynamics of physical and cognitive fatigue

The willingness to exert effort for reward is essential but comes at the cost of fatigue. Theories suggest fatigue increases after both physical and cognitive exertion, subsequently reducing the motivation to exert effort. Yet a mechanistic understanding of how this happens on a moment-to-moment basis, and whether mechanisms are common to both mental and physical effort, is lacking. In two studies, participants reported momentary (trial-by-trial) ratings of fatigue during an effort-based decision-making task requiring either physical (grip-force) or cognitive (mental arithmetic) effort. Using a novel computational model, we show that fatigue fluctuates from trial-to-trial as a function of exerted effort and predicts subsequent choices. This mechanism was shared across the domains. Selective to the cognitive domain, committing errors also induced momentary increases in feelings of fatigue. These findings provide insight into the computations underlying the influence of effortful exertion on fatigue and motivation, in both physical and cognitive domains.

The dorsomedial prefrontal cortex represents subjective value across effort-based and risky decision-making

Decisions that require taking effort costs into account are ubiquitous in real life. The neural common currency theory hypothesizes that a particular neural network integrates different costs (e.g., risk) and rewards into a common scale to facilitate value comparison. Although there has been a surge of interest in the computational and neural basis of effort-related value integration, it is still under debate if effort-based decision-making relies on a domain-general valuation network as implicated in the neural common currency theory. Therefore, we comprehensively compared effort-based and risky decision-making using a combination of computational modeling, univariate and multivariate fMRI analyses, and data from two independent studies. We found that effort-based decision-making can be best described by a power discounting model that accounts for both the discounting rate and effort sensitivity. At the neural level, multivariate decoding analyses indicated that the neural patterns of the dorsomedial prefrontal cortex (dmPFC) represented subjective value across different decision-making tasks including either effort or risk costs, although univariate signals were more diverse. These findings suggest that multivariate dmPFC patterns play a critical role in computing subjective value in a task-independent manner and thus extend the scope of the neural common currency theory.

Strengths of social ties modulate brain computations for third-party punishment

Costly punishment of social norm transgressors by third-parties has been considered as a decisive stage in the evolution of human cooperation. An important facet of social relationship knowledge concerns the strength of the social ties between individuals, as measured by social distance. Yet, it is unclear how the enforcement of social norms is influenced by the social distance between a third-party and a norm violator at the behavioral and the brain system levels. Here, we investigated how social distance between punishers and norm-violators influences third-party punishment. Participants as third-party punished norm violators more severely as social distance between them increased. Using model-based fMRI, we disentangled key computations contributing to third-party punishment: inequity aversion, social distance between participant and norm violator and integration of the cost to punish with these signals. Inequity aversion increased activity in the anterior cingulate cortex and bilateral insula, and processing social distance engaged a bilateral fronto-parietal cortex brain network. These two brain signals and the cost to punish were integrated in a subjective value signal of sanctions that modulated activity in the ventromedial prefrontal cortex. Together, our results reveal the neurocomputational underpinnings of third-party punishment and how social distance modulates enforcement of social norms in humans.

Does decision confidence reflect effort?

Goal directed behaviour requires transformation of sensory input to decision, and then to output action. How the sensory input is accumulated to form the decision has been extensively studied, however, the influence of output action on decision making has been largely dismissed. Although the recent emerging view postulates the reciprocal interaction between action and decision, still little is known about how the parameters of an action modulates the decision. In this study, we focused on the physical effort which necessarily entails with action. We tested if the physical effort during the deliberation period of the perceptual decision, not the effort required after deciding a particular option, can impact on the process of forming the decision. Here, we set up an experimental situation where investing effort is necessary for the initiation of the task, but importantly, is orthogonal to success in task performance. The study was pre-registered to test the hypothesis that the increased effort will decrease the metacognitive accuracy of decision, without affecting the decision accuracy. Participants judged the direction of a random-dot motion stimuli, while holding and maintaining the position of a robotic manipulandum with their right hand. In the key experimental condition, the manipulandum produced force to move away from its position, requiring the participants to resist the force while accumulating the sensory evidence for the decision. The decision was reported by a key-press using the left-hand. We found no evidence that such incidental (i.e., non-instrumental) effort may influence the subsequent decision process and most importantly decision confidence. The possible reason for this result and the future direction of the research are discussed.

Expending effort may share neural responses with reward and evokes high subjective satisfaction

Throughout our daily lives, the levels of effort we invest in various tasks are influenced by reward processing. The subjective expectation after expending effort is a primary factor affecting reward processing. However, recent studies indicate that individual differences in reward anticipation influence this subjective valuation. To better understand the relationship between effort expenditure and the subjective valuation of rewards, in this study, we perform an experiment in which we manipulate effort, control reward expectation implicitly, and measure the subjective valuation of rewards using event-related potentials (ERPs) and physical effort through behavioral measures (number of keystrokes). In the reward-task paradigm, 30 subjects performed effort and control trials, with the reward probability comparable across the effort and control conditions. We also examined the ERPs associated with the valuation of subjective rewards, including reward positivity (RewP) and set reward expectation controlled as the baseline. The results showed that the ERP amplitudes, the number of keystrokes, and explicit satisfaction ratings were all significantly greater in the effort condition than in the control condition. The participants maintained high levels of effort throughout the sessions associated with the experiment. The results of this study suggest that when reward expectations are controlled, effort expenditure evokes neural responses similar to reward feedback being given, which is linked with increased subjective satisfaction.

Ventral tegmental area integrity measured with high-resolution 7-Tesla MRI relates to motivation across depression and anxiety diagnoses

The ventral tegmental area (VTA) is one of the major sources of dopamine in the brain and has been associated with reward prediction, error-based reward learning, volitional drive and anhedonia. However, precise anatomical investigations of the VTA have been prevented by the use of standard-resolution MRI, reliance on subjective manual tracings, and lack of quantitative measures of dopamine-related signal. Here, we combine ultra-high field 400 µm³ quantitative MRI with dopamine-related signal mapping, and a mixture of machine learning and supervised computational techniques to delineate the VTA in a transdiagnostic sample of subjects with and without depression and anxiety disorders. Subjects also underwent cognitive testing to measure intrinsic and extrinsic motivational tone. Fifty-one subjects were scanned in total, including healthy control (HC) and mood/anxiety (MA) disorder subjects. MA subjects had significantly larger VTA volumes compared to HC but significantly lower signal intensity within VTA compared to HC, indicating reduced structural integrity of the dopaminergic VTA. Interestingly, while VTA integrity did not significantly correlate with self-reported depression or anxiety symptoms, it was correlated with an objective cognitive measure of extrinsic motivation, whereby lower VTA integrity was associated with lower motivation. This is the first study to demonstrate a computational pipeline for detecting and delineating the VTA in human subjects with 400 μm³ resolution. We highlight the use of objective transdiagnostic measures of cognitive function that link neural integrity to behavior across clinical and non-clinical groups.

Distinct neural representations for prosocial and self-benefiting effort

Prosocial behaviors-actions that benefit others-are central to individual and societal well-being. Although the mechanisms underlying the financial and moral costs of prosocial behaviors are increasingly understood, this work has often ignored a key influence on behavior: effort. Many prosocial acts are effortful, and people are averse to the costs of exerting them. However, how the brain encodes effort costs when actions benefit others is unknown. During fMRI, participants completed a decision-making task where they chose in each trial whether to "work" and exert force (30%-70% of maximum grip strength) or "rest" (no effort) for rewards (2-10 credits). Crucially, on separate trials, they made these decisions either to benefit another person or themselves. We used a combination of multivariate representational similarity analysis and model-based univariate analysis to reveal how the costs of prosocial and self-benefiting efforts are processed. Strikingly, we identified a unique neural signature of effort in the anterior cingulate gyrus (ACCg) for prosocial acts, both when choosing to help others and when exerting force to benefit them. This pattern was absent for self-benefiting behaviors. Moreover, stronger, specific representations of prosocial effort in the ACCg were linked to higher levels of empathy and higher subsequent exerted force to benefit others. In contrast, the ventral tegmental area and ventral insula represented value preferentially when choosing for oneself and not for prosocial acts. These findings advance our understanding of the neural mechanisms of prosocial behavior, highlighting the critical role that effort has in the brain circuits that guide helping others.

A computational account of why more valuable goals seem to require more effortful actions

To decide whether a course of action is worth pursuing, individuals typically weigh its expected costs and benefits. Optimal decision-making relies upon accurate effort cost anticipation, which is generally assumed to be performed independently from goal valuation. In two experiments (n = 46), we challenged this independence principle of standard decision theory. We presented participants with a series of treadmill routes randomly associated to monetary rewards and collected both ‘accept’ versus ‘decline’ decisions and subjective estimates of energetic cost. Behavioural results show that higher monetary prospects led participants to provide higher cost estimates, although reward was independent from effort in our design. Among candidate cognitive explanations, they support a model in which prospective cost assessment is biased by the output of an automatic computation adjusting effort expenditure to goal value. This decision bias might lead people to abandon the pursuit of valuable goals that are in fact not so costly to achieve.

Everything Comes at a Price: Considerations in Modeling Effort-Based Choice

When observing human behavior, one of the key factors determining choice is effort. It is often assumed that people prefer an easier course of action when the alternative yields the same benefits. However, recent research demonstrates that this is not always the case: effort is not always costly and can also add value. A promising avenue to study effort-based choice is to utilize formal decision models that enable quantitative modeling. In this paper, we aim to present an overview of the current approaches to modeling effort-based choice and discuss some considerations that stem from theoretical and practical issues (present and previous) in studies on the role of effort, focusing on the connections and discrepancies between formal models and the findings from the body of empirical research. Considering that effort can, in some circumstances, act as a cost and as a benefit, reconciling these discrepancies is a practical and theoretical challenge that can ultimately lead to better predictions and increased model validity. Our review identifies and discusses these discrepancies providing direction for future empirical research.

Depression reduces neural correlates of reward salience with increasing effort over the course of the progressive ratio task

BACKGROUND

Depressive disorders have been associated with altered effort-cost decision making (ECDM) in behavioral investigations, such as a decreased willingness to expend effort for reward attainment. However, little is known about neural mechanisms implicated in altered ECDM.

METHODS

The study investigates neural correlates of reward attainment during a progressive ratio task in participants with a current depressive disorder (n = 65) and never-depressed healthy individuals (n = 44). On each trial, participants completed an increasing number of button presses to attain a fixed monetary reward, indicated by an auditory reward signal. Participants could decide to quit the task anytime (breakpoint). EEG was recorded during the task and P300 amplitudes were examined in response to the auditory signal of reward attainment.

RESULTS

There was no difference in breakpoint, as both groups completed comparable numbers of button presses. In contrast, results from mixed-effects models of trial-level ERP responses indicated depression-related effects on P300 amplitudes over the course of the task. Generally, the reward-locked P300 increased with ascending effort expenditure; however, compared to healthy participants, individuals with current depression were characterized by an attenuated trajectory of the reward-locked P300, pointing towards decreased reactivity to reward attainment with increasing effort.

LIMITATIONS

Sample size and reward magnitude were possibly not large enough to detect differences in breakpoint.

CONCLUSIONS

The results of the current study demonstrate that the investigation of reward-related P300 trajectory may represent an informative novel addition to the progressive ratio task, which could help shed light on depression-related alterations in motivation and ECDM.

Motor response vigour and visual fixation patterns reflect subjective valuation during intertemporal choice

Value-based decision-making is of central interest in cognitive neuroscience and psychology, as well as in the context of neuropsychiatric disorders characterised by decision-making impairments. Studies examining (neuro-)computational mechanisms underlying choice behaviour typically focus on participants’ decisions. However, there is increasing evidence that option valuation might also be reflected in motor response vigour and eye movements, implicit measures of subjective utility. To examine motor response vigour and visual fixation correlates of option valuation in intertemporal choice, we set up a task where the participants selected an option by pressing a grip force transducer, simultaneously tracking fixation shifts between options. As outlined in our preregistration (https://osf.io/k6jct), we used hierarchical Bayesian parameter estimation to model the choices assuming hyperbolic discounting, compared variants of the softmax and drift diffusion model, and assessed the relationship between response vigour and the estimated model parameters. The behavioural data were best explained by a drift diffusion model specifying a non-linear scaling of the drift rate by the subjective value differences. Replicating previous findings, we found a magnitude effect for temporal discounting, such that higher rewards were discounted less. This magnitude effect was further reflected in motor response vigour, such that stronger forces were exerted in the high vs. the low magnitude condition. Bayesian hierarchical linear regression further revealed higher grip forces, faster response times and a lower number of fixation shifts for trials with higher subjective value differences. An exploratory analysis revealed that subjective value sums across options showed an even more pronounced association with trial-wise grip force amplitudes. Our data suggest that subjective utility or implicit valuation is reflected in motor response vigour and visual fixation patterns during intertemporal choice. Taking into account response vigour might thus provide deeper insight into decision-making, reward valuation and maladaptive changes in these processes, e.g. in the context of neuropsychiatric disorders.

Choices favoring cognitive effort in a foraging environment decrease when multiple forms of effort and delay are interleaved

Cognitive and physical effort are typically regarded as costly, but demands for effort also seemingly boost the appeal of prospects under certain conditions. One contextual factor that might influence choices for or against effort is the mix of different types of demand a decision maker encounters in a given environment. In two foraging experiments, participants encountered prospective rewards that required equally long intervals of cognitive effort, physical effort, or unfilled delay. Monetary offers varied per trial, and the two experiments differed in whether the type of effort or delay cost was the same on every trial, or varied across trials. When each participant faced only one type of cost, cognitive effort persistently produced the highest acceptance rate compared to trials with an equivalent period of either physical effort or unfilled delay. We theorized that if cognitive effort were intrinsically rewarding, we would observe the same pattern of preferences when participants foraged for varying cost types in addition to rewards. Contrary to this prediction, in the second experiment, an initially higher acceptance rate for cognitive effort trials disappeared over time amid an overall decline in acceptance rates as participants gained experience with all three conditions. Our results indicate that cognitive demands may reduce the discounting effect of delays, but not because decision makers assign intrinsic value to cognitive effort. Rather, the results suggest that a cognitive effort requirement might influence contextual factors such as subjective delay duration estimates, which can be recalibrated if multiple forms of demand are interleaved.

Disordered Decision Making: A Cognitive Framework for Apathy and Impulsivity in Huntington's Disease

A caregiver's all‐too‐familiar narrative ‐ “He doesn't think through what he does, but mostly he does nothing.” Apathy and impulsivity, debilitating and poorly understood, commonly co‐occur in Huntington's disease (HD). HD is a neurodegenerative disease with manifestations bridging clinical neurology and psychiatry. In addition to movement and cognitive symptoms, neurobehavioral disturbances, particularly apathy and impulsivity, are prevalent features of HD, occurring early in the disease course, often worsening with disease progression, and substantially reducing quality of life. Treatments remain limited, in part because of limited mechanistic understanding of these behavioral disturbances. However, emerging work within the field of decision‐making neuroscience and beyond points to common neurobiological mechanisms underpinning these seemingly disparate problems. These insights bridge the gap between underlying disease pathology and clinical phenotype, offering new treatment strategies, novel behavioral and physiological biomarkers of HD, and deeper understanding of human behavior. In this review, we apply the neurobiological framework of cost‐benefit decision making to the problems of apathy and impulsivity in HD. Through this decision‐making lens, we develop a mechanistic model that elucidates the occurrence of these behavioral disturbances and points to potential treatment strategies and crucial research priorities. © 2022 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson Movement Disorder Society.

Cognitive Effort Exertion Enhances Electrophysiological Responses to Rewarding Outcomes

Recent work has highlighted neural mechanisms underlying cognitive effort-related discounting of anticipated rewards. However, findings on whether effort exertion alters the subjective value of obtained rewards are inconsistent. Here, we provide a more nuanced account of how cognitive effort affects subsequent reward processing in a novel task designed to assess effort-induced modulations of the Reward Positivity, an event-related potential indexing reward-related neural activity. We found that neural responses to both gains and losses were significantly elevated in trials requiring more versus less cognitive effort. Moreover, time-frequency analysis revealed that these effects were mirrored in gain-related delta, but not in loss-related theta band activity, suggesting that people ascribed more value to high-effort outcomes. In addition, we also explored whether individual differences in behavioral effort discounting rates and reward sensitivity in the absence of effort may affect the relationship between effort exertion and subsequent reward processing. Together, our findings provide evidence that cognitive effort exertion can increase the subjective value of subsequent outcomes and that this effect may primarily rely on modulations of delta band activity.

Examining the reliability and validity of two versions of the Effort-Expenditure for Rewards Task (EEfRT)

The Effort-Expenditure for Rewards Task (EEfRT) has gained validity evidence from several studies. However, various modifications have been applied to the original version, which have never been compared systematically. In Study 1, we tested 120 healthy participants to directly compare two versions of the EEfRT. In Study 2, we tested a larger sample of 394 healthy participants to further examine the original EEfRT. We replicated the split-half reliability of both task versions. However, self-reported personality traits (e.g., trait BAS) correlated with only some task performance parameters in Study 1, which did not replicate for the original EEfRT in Study 2. Our results indicate complex and sometimes inconsistent relations between different personality traits, task properties, and reward attributes.

Advances in modeling learning and decision-making in neuroscience

An organism's survival depends on its ability to learn about its environment and to make adaptive decisions in the service of achieving the best possible outcomes in that environment. To study the neural circuits that support these functions, researchers have increasingly relied on models that formalize the computations required to carry them out. Here, we review the recent history of computational modeling of learning and decision-making, and how these models have been used to advance understanding of prefrontal cortex function. We discuss how such models have advanced from their origins in basic algorithms of updating and action selection to increasingly account for complexities in the cognitive processes required for learning and decision-making, and the representations over which they operate. We further discuss how a deeper understanding of the real-world complexities in these computations has shed light on the fundamental constraints on optimal behavior, and on the complex interactions between corticostriatal pathways to determine such behavior. The continuing and rapid development of these models holds great promise for understanding the mechanisms by which animals adapt to their environments, and what leads to maladaptive forms of learning and decision-making within clinical populations.

Different depression: motivational anhedonia governs antidepressant efficacy in Huntington's disease

Depression is more common in neurodegenerative diseases such as Huntington's disease than the general population. Antidepressant efficacy is well-established for depression within the general population: a recent meta-analysis showed serotonin norepinephrine reuptake inhibitors, tricyclic antidepressants and mirtazapine outperformed other antidepressants. Despite the severe morbidity, antidepressant choice in Huntington's disease is based on Class IV evidence. We used complementary approaches to determine treatment choice for depression in Huntington's disease: propensity score analyses of antidepressant treatment outcome using the ENROLL-HD data set, and a dissection of the cognitive mechanisms underlying depression in Huntington's disease using a cognitive battery based on the Research Domain Criteria for Depression. Study 1 included ENROLL-HD 5486 gene-positive adult patients started on an antidepressant medication for depression. Our outcome measures were depression (Hospital Anxiety and Depression Scale or Problem Behaviours Assessment 'Depressed Mood' item) at first follow-up (primary outcome) and all follow-ups (secondary outcome). The intervention was antidepressant class. We used Svyglm&Twang in R to perform propensity scoring, using known variables (disease progression, medical comorbidity, psychiatric morbidity, sedatives, number of antidepressants, demographics and antidepressant contraindications) to determine the probability of receiving different antidepressants (propensity score) and then included the propensity score in a model of treatment efficacy. Study 2 recruited 51 gene-positive adult patients and 26 controls from the South Wales Huntington's Disease Management Service. Participants completed a motor assessment, in addition to measures of depression and apathy, followed by tasks measuring consummatory anhedonia, motivational anhedonia, learning from reward and punishment and reaction to negative outcome. We used generalised linear models to determine the association between task performance and depression scores. Study 1 showed selective serotonin reuptake inhibitors outperformed serotonin norepinephrine reuptake inhibitors on the primary outcome (P = 0.048), whilst both selective serotonin reuptake inhibitors (P = 0.00069) and bupropion (P = 0.0045) were superior to serotonin norepinephrine reuptake inhibitors on the secondary outcome. Study 2 demonstrated an association between depression score and effort for reward that was not explained by apathy. No other mechanisms were associated with depression score. We found that selective serotonin reuptake inhibitors and bupropion outperform serotonin norepinephrine reuptake inhibitors at alleviating depression in Huntington's disease. Moreover, motivational anhedonia appears the most significant mechanism underlying depression in Huntington's disease. Bupropion is improves motivational anhedonia and has a synergistic effect with selective serotonin reuptake inhibitors. This work provides the first large-scale, objective evidence to determine treatment choice for depression in Huntington's disease, and provides a model for determining antidepressant efficacy in other neurodegenerative diseases.

Neurocomputational mechanisms underlying the subjective value of information

Humans have a striking desire to actively seek new information, even when it is devoid of any instrumental utility. However, the mechanisms that drive individuals' subjective preference for information remain unclear. Here, we used fMRI to examine the processing of subjective information value, by having participants decide how much effort they were willing to trade-off for non-instrumental information. We showed that choices were best described by a model that accounted for: (1) the variability in individuals' estimates of uncertainty, (2) their desire to reduce that uncertainty, and (3) their subjective preference for positively valenced information. Model-based analyses revealed the anterior cingulate as a key node that encodes the subjective value of information across multiple stages of decision-making - including when information was prospectively valued, and when the outcome was definitively delivered. These findings emphasise the multidimensionality of information value, and reveal the neurocomputational mechanisms underlying the variability in individuals' desire to physically pursue informative outcomes.

Dissociating the Impact of Movement Time and Energy Costs on Decision-Making and Action Initiation in Humans

Recent theories and data suggest that adapted behavior involves economic computations during which multiple trade-offs between reward value, accuracy requirement, energy expenditure, and elapsing time are solved so as to obtain rewards as soon as possible while spending the least possible amount of energy. However, the relative impact of movement energy and duration costs on perceptual decision-making and movement initiation is poorly understood. Here, we tested 31 healthy subjects on a perceptual decision-making task in which they executed reaching movements to report probabilistic choices. In distinct blocks of trials, the reaching duration (“Time” condition) and energy (“Effort” condition) costs were independently varied compared to a “Reference” block, while decision difficulty was maintained similar at the block level. Participants also performed a simple delayed-reaching (DR) task aimed at estimating movement initiation duration in each motor condition. Results in that DR task show that long duration movements extended reaction times (RTs) in most subjects, whereas energy-consuming movements led to mixed effects on RTs. In the decision task, about half of the subjects decreased their decision durations (DDs) in the Time condition, while the impact of energy on DDs were again mixed across subjects. Decision accuracy was overall similar across motor conditions. These results indicate that movement duration and, to a lesser extent, energy expenditure, idiosyncratically affect perceptual decision-making and action initiation. We propose that subjects who shortened their choices in the time-consuming condition of the decision task did so to limit a drop of reward rate.

Heightened effort discounting is a common feature of both apathy and fatigue

Apathy and fatigue have distinct aetiologies, yet can manifest in phenotypically similar ways. In particular, each can give rise to diminished goal-directed behaviour, which is often cited as a key characteristic of both traits. An important issue therefore is whether currently available approaches are capable of distinguishing between them. Here, we examined the relationship between commonly administered inventories of apathy and fatigue, and a measure of goal-directed activity that assesses the motivation to engage in effortful behaviour. 103 healthy adults completed self-report inventories on apathy (the Dimensional Apathy Scale), and fatigue (the Multidimensional Fatigue Inventory, and/or Modified Fatigue Impact Scale). In addition, all participants performed an effort discounting task, in which they made choices about their willingness to engage in physically effortful activity. Importantly, self-report ratings of apathy and fatigue were strongly correlated, suggesting that these inventories were insensitive to the fundamental differences between the two traits. Furthermore, greater effort discounting was strongly associated with higher ratings across all inventories, suggesting that a common feature of both traits is a lower motivation to engage in effortful behaviour. These results have significant implications for the assessment of both apathy and fatigue, particularly in clinical groups in which they commonly co-exist.

A Neurocomputational Model for Intrinsic Reward

Standard economic indicators provide an incomplete picture of what we value both as individuals and as a society. Furthermore, canonical macroeconomic measures, such as GDP, do not account for non-market activities (e.g., cooking, childcare) that nevertheless impact well-being. Here, we introduce a computational tool that measures the affective value of experiences (e.g., playing a musical instrument without errors). We go on to validate this tool with neural data, using fMRI to measure neural activity in male and female human subjects performing a reinforcement learning task that incorporated periodic ratings of subjective affective state. Learning performance determined level of payment (i.e., extrinsic reward). Crucially, the task also incorporated a skilled performance component (i.e., intrinsic reward) which did not influence payment. Both extrinsic and intrinsic rewards influenced affective dynamics, and their relative influence could be captured in our computational model. Individuals for whom intrinsic rewards had a greater influence on affective state than extrinsic rewards had greater ventromedial prefrontal cortex (vmPFC) activity for intrinsic than extrinsic rewards. Thus, we show that computational modeling of affective dynamics can index the subjective value of intrinsic relative to extrinsic rewards, a “computational hedonometer” that reflects both behavior and neural activity that quantifies the affective value of experience.

SIGNIFICANCE STATEMENT Traditional economic indicators are increasingly recognized to provide an incomplete picture of what we value as a society. Standard economic approaches struggle to accurately assign values to non-market activities that nevertheless may be intrinsically rewarding, prompting a need for new tools to measure what really matters to individuals. Using a combination of neuroimaging and computational modeling, we show that despite their lack of instrumental value, intrinsic rewards influence subjective affective state and ventromedial prefrontal cortex (vmPFC) activity. The relative degree to which extrinsic and intrinsic rewards influence affective state is predictive of their relative impacts on neural activity, confirming the utility of our approach for measuring the affective value of experiences and other non-market activities in individuals.

Carrots for the donkey: Influence of evaluative conditioning and training on self-paced exercise intensity and delay discounting of exercise in healthy adults

To choose exercise over alternative behaviours, subjective reward evaluation of the potential choices is a principal step in decision making. However, the selection of exercise intensity might integrate acute visceral responses (i.e. pleasant or unpleasant feelings) and motives related to goals (i.e. enjoyment, competition, health). To understand the factors determining the selection of exercise in its intensity and evaluation as a modality, we conducted a study combining exercise training and evaluative conditioning. Evaluative conditioning was performed by using a novel technique using a primary reinforcer (sweetness) as the unconditioned stimulus and physical strain i.e. heart rate elevation as the conditioned stimulus during interval training, using a randomized control design (N = 58). Pre, post-three weeks interval training w/o conditioning, and after 4 weeks follow-up, participants were tested on self-paced speed selection on treadmill measuring heart rate, subjective pleasantness, and effort levels, as well as delay-discounting of exercise and food rewards. Results revealed that the selection of exercise intensity was significantly increased by adaptation to training and evaluative conditioning, revealing the importance of visceral factors as well as learned expected rewards. Delay discounting rates of self-paced exercise were transiently reduced by training but not affected by evaluative conditioning. In conclusion, exercise decisions are suggested to separate the decision-making process into a modality-specific cognitive evaluation of exercise, and an exercise intensity selection based on acute visceral experience integrating effort, pleasantness, and learned rewards.

The neural basis of decision-making during time-based inter-role conflict

Time-based inter-role conflict is a type of conflict in which individuals are faced with simultaneous role pressures from different role domains. Some researchers have applied a decision-making perspective to investigate inter-role conflict; however, the neural basis of inter-role decision-making has rarely been discussed. In the current study, a collection of inter-role conflict scenarios with high/low levels of conflict were selected, and sixty college students were recruited to make choices between the conflicting student and family/friend demands in each scenario while their brain activities were recorded using functional magnetic resonance imaging. Blood oxygen level-dependent conjunction analysis found that making decisions in inter-role conflict activated brain areas, including the bilateral medial prefrontal cortex (mPFC), bilateral temporoparietal conjunction (TPJ), bilateral posterior cingulate cortex (PCC), and bilateral anterior temporal lobe. Direct comparisons between high versus low conflict situations showed increased activation of the left dorsal anterior cingulate. A generalized psychophysiological interaction analysis further showed enhanced connectivity among the mPFC, PCC, and bilateral TPJ in high conflict versus low conflict situations. Our study improved understanding of the relationship between brain and inter-role decision-making and provided an empirical examination on the psychological process propositions.

Computational modelling of social cognition and behaviour-a reinforcement learning primer

Social neuroscience aims to describe the neural systems that underpin social cognition and behaviour. Over the past decade, researchers have begun to combine computational models with neuroimaging to link social computations to the brain. Inspired by approaches from reinforcement learning theory, which describes how decisions are driven by the unexpectedness of outcomes, accounts of the neural basis of prosocial learning, observational learning, mentalizing and impression formation have been developed. Here we provide an introduction for researchers who wish to use these models in their studies. We consider both theoretical and practical issues related to their implementation, with a focus on specific examples from the field.

Using metabolic energy to quantify the subjective value of physical effort

Economists have known for centuries that to understand an individual's decisions, we must consider not only the objective value of the goal at stake, but its subjective value as well. However, achieving that goal ultimately requires expenditure of effort. Surprisingly, despite the ubiquitous role of effort in decision-making and movement, we currently do not understand how effort is subjectively valued in daily movements. Part of the difficulty arises from the lack of an objective measure of effort. Here, we use a physiological approach to address this knowledge gap. We quantified objective effort costs by measuring metabolic cost via expired gas analysis as participants performed a reaching task against increasing resistance. We then used neuroeconomic methods to quantify each individual's subjective valuation of effort. Rather than the diminishing sensitivity observed in reward valuation, effort was valued objectively, on average. This is significantly less than the near-quadratic sensitivity to effort observed previously in force-based motor tasks. Moreover, there was significant inter-individual variability with many participants undervaluing or overvaluing effort. These findings demonstrate that in contrast with monetary decisions in which subjective value exhibits diminishing marginal returns, effort costs are valued more objectively in low-effort reaching movements common in daily life.

Investigating the effect of losses and gains on effortful engagement during an incentivized Go/NoGo task through anticipatory cortical oscillatory changes

Losses usually have greater subjective value (SV) than gains of equal nominal value but often cause a relative deterioration in effortful performance. Since losses and gains induce differing approach/avoidance behavioral tendencies, we explored whether incentive type interacted with approach/avoidance motor-sets. Alpha- and beta-band event-related desynchronization (ERD) was hypothesized to be weakest when participants expected a loss and prepared an inhibitory motor-set, and strongest when participants expected a gain and prepared an active motor-set. It was also hypothesized that effort would modulate reward and motor-set-related cortical activation patterns. Participants completed a cued Go/NoGo task while expecting a reward (+10p), avoiding a loss (-10p), or receiving no incentive (0p); and while expecting a NoGo cue with a probability of either .75 or .25. Pre-movement alpha- and beta-band EEG power was analyzed using the ERD method, and the SV of effort was evaluated using a cognitive effort discounting task. Gains incentivized faster RTs and stronger preparatory alpha band ERD compared to loss and no incentive conditions, while inhibitory motor-sets resulted in significantly weaker alpha-band ERD. However, there was no interaction between incentive and motor-sets. Participants were more willing to expend effort in losses compared to gain trials, although the SV of effort was not associated with ERD patterns or RTs. Results suggest that incentive and approach/avoidance motor tendencies modulate cortical activations prior to a speeded RT movement independently, and are not associated with the economic value of effort. The present results favor attentional explanations of the effect of incentive modality on effort.

Body dynamics of gait affect value-based decisions

Choosing among different options typically entails weighing their anticipated costs and benefits. Previous research has predominantly focused on situations, where the costs and benefits of choices are known before an action is effectuated. Yet many decisions in daily life are made on the fly, for instance, making a snack choice while walking through the grocery store. Notably, the costs of actions change dynamically while moving. Therefore, in this study we examined whether the concurrent action dynamics of gait form part of and affect value-based decisions. In three experiments, participants had to decide which lateral (left vs. right) target (associated with different rewards) they would go to, while they were already walking. Results showed that the target choice was biased by the alternating stepping behavior, even at the expense of receiving less reward. These findings provide evidence that whole-body action dynamics affect value-based decisions.

Aging Increases Prosocial Motivation for Effort

Social cohesion relies on prosociality in increasingly aging populations. Helping other people requires effort, yet how willing people are to exert effort to benefit themselves and others, and whether such behaviors shift across the life span, is poorly understood. Using computational modeling, we tested the willingness of 95 younger adults (18-36 years old) and 92 older adults (55-84 years old) to put physical effort into self- and other-benefiting acts. Participants chose whether to work and exert force (30%-70% of maximum grip strength) for rewards (2-10 credits) accrued for themselves or, prosocially, for another. Younger adults were somewhat selfish, choosing to work more at higher effort levels for themselves, and exerted less force in prosocial work. Strikingly, compared with younger adults, older people were more willing to put in effort for others and exerted equal force for themselves and others. Increased prosociality in older people has important implications for human behavior and societal structure.

Chronic non-discriminatory social defeat stress reduces effort-related motivated behaviors in male and female mice

Reward and motivation deficits are prominent symptoms in many mood disorders, including depression. Similar reward and effort-related choice behavioral tasks can be used to study aspects of motivation in both rodents and humans. Chronic stress can precipitate mood disorders in humans and maladaptive reward and motivation behaviors in male rodents. However, while depression is more prevalent in women, there is relatively little known about whether chronic stress elicits maladaptive behaviors in female rodents in effort-related motivated tasks and whether there are any behavioral sex differences. Chronic nondiscriminatory social defeat stress (CNSDS) is a variation of chronic social defeat stress that is effective in both male and female mice. We hypothesized that CNSDS would reduce effort-related motivated and reward behaviors, including reducing sensitivity to a devalued outcome, reducing breakpoint in progressive ratio, and shifting effort-related choice behavior. Separate cohorts of adult male and female C57BL/6 J mice were divided into Control or CNSDS groups, exposed to the 10-day CNSDS paradigm, and then trained and tested in instrumental reward or effort-related behaviors. CNSDS reduced motivation to lever press in progressive ratio and shifted effort-related choice behavior from a high reward to a more easily attainable low reward in both sexes. CNSDS caused more nuanced impairments in outcome devaluation. Taken together, CNSDS induces maladaptive shifts in effort-related choice and reduces motivated lever pressing in both sexes.

The Neurobiology of Pathological Fatigue: New Models, New Questions

The last decade has seen the emergence of new theoretical frameworks to explain pathological fatigue, a much neglected, yet highly significant symptom across a wide range of diseases. While the new models of fatigue provide new hypotheses to test, they also raise a number of questions. The primary purpose of this essay is to examine the predictions of three recently proposed models of fatigue, the overlap and differences between them, and the evidence from diseases that may lend support to the models of fatigue. I also present expansions for the sensory attenuation model of fatigue. Further questions examined here are the following: What are the neural substrates of fatigue? How can sensory attenuation, which underpins agency also explain fatigue? Are fatigue and agency related?

The effort-doors task: Examining the temporal dynamics of effort-based reward processing using ERPs

Aberrant reward processing is a cardinal feature of various forms of psychopathology. However, recent research indicates that aberrant reward processing may manifest at temporally distinct substages and involve interdependent subcomponents of reward processing. To improve our understanding of both the temporal dynamics and distinct subcomponents of reward processing, we added an effort manipulation to the "doors" reward-task paradigm, to derive behavioral and event-related potential (ERP) measures of effort-based reward processing. Behavioral measures consisting of reaction time, response rate, and response rate change were used to index effort expenditure, and ERP measures were used to index attention allocated toward effort-completion cues, anticipation of reward, valuation of reward, and attention toward monetary feedback. Reduced response rate and slowing of response were evident during the high effort versus the low effort condition. ERP findings indicated increased attention to signals of high- compared to low-effort completion cues-as well as reduced anticipation of rewards, and reduced attention toward feedback information following high effort expenditure. Participants showing the most response-rate slowing evidenced the greatest reward devaluation following high versus low effort. Findings demonstrate that the addition of an effort expenditure manipulation to the doors reward paradigm produced reliable ERP and behavioral measures of effort-based reward processing, providing opportunities for future researchers to utilize the effort-doors task to parse the temporal dynamics of both anticipatory and consummatory reward processing components.

Forced choices reveal a trade-off between cognitive effort and physical pain

Cognitive effort is described as aversive, and people will generally avoid it when possible. This aversion to effort is believed to arise from a cost-benefit analysis of the actions available. The comparison of cognitive effort against other primary aversive experiences, however, remains relatively unexplored. Here, we offered participants choices between performing a cognitively demanding task or experiencing thermal pain. We found that cognitive effort can be traded off for physical pain and that people generally avoid exerting high levels of cognitive effort. We also used computational modelling to examine the aversive subjective value of effort and its effects on response behaviours. Applying this model to decision times revealed asymmetric effects of effort and pain, suggesting that cognitive effort may not share the same basic influences on avoidance behaviour as more primary aversive stimuli such as physical pain.

Roles of Ventromedial Prefrontal Cortex and Anterior Cingulate in Subjective Valuation of Prospective Effort

The perceived effort level of an action shapes everyday decisions. Despite the importance of these perceptions for decision-making, the behavioral and neural representations of the subjective cost of effort are not well understood. While a number of studies have implicated anterior cingulate cortex (ACC) in decisions about effort/reward trade-offs, none have experimentally isolated effort valuation from reward and choice difficulty, a function that is commonly ascribed to this region. We used functional magnetic resonance imaging to monitor brain activity while human participants engaged in uncertain choices for prospective physical effort. Our task was designed to examine effort-based decision-making in the absence of reward and separated from choice difficulty-allowing us to investigate the brain's role in effort valuation, independent of these other factors. Participants exhibited subjectivity in their decision-making, displaying increased sensitivity to changes in subjective effort as objective effort levels increased. Analysis of blood-oxygenation-level dependent activity revealed that the ventromedial prefrontal cortex (vmPFC) encoded the subjective valuation of prospective effort, and ACC activity was best described by choice difficulty. These results provide insight into the processes responsible for decision-making regarding effort, partly dissociating the roles of vmPFC and ACC in prospective valuation of effort and choice difficulty.

Pharmacological evidence for the implication of noradrenaline in effort

The trade-off between effort and reward is one of the main determinants of behavior, and its alteration is at the heart of major disorders such as depression or Parkinson's disease. Monoaminergic neuromodulators are thought to play a key role in this trade-off, but their relative contribution remains unclear. Rhesus monkeys (Macaca mulatta) performed a choice task requiring a trade-off between the volume of fluid reward and the amount of force to be exerted on a grip. In line with a causal role of noradrenaline in effort, decreasing noradrenaline levels with systemic clonidine injections (0.01 mg/kg) decreased exerted force and enhanced the weight of upcoming force on choices, without any effect on reward sensitivity. Using computational modeling, we showed that a single variable ("effort") could capture the amount of resources necessary for action and control both choices (as a variable for decision) and force production (as a driving force). Critically, the multiple effects of noradrenaline manipulation on behavior could be captured by a specific modulation of this single variable. Thus, our data strongly support noradrenaline's implication in effort processing.

Effort-reward balance and work motivation in rats: Effects of context and order of experience

Being motivated means exerting effort toward a goal. The 'law of least work' emphasizes a preference for exerting relatively less effort. The law crosses boundaries among species and between physical and mental work. Organisms should be highly sensitive to shifts in effort-reward balance (ERB) in order to make optimal choices. We used a free operant-foraging task to investigate changes in ERB on choice between options requiring more or less effort. Results showed a consistent preference for the option with less effort and insensitivity to shifts in ERB. A second aim explored the influence of order of experience on effort choice. Choice for the more effortful option significantly increased after experiencing an equal effort-reward relationship during the initial free operant-foraging session. This relative increase in choice for the effortful option persisted even after effort-reward imbalance. The findings highlight the importance of contextual factors such as order of experience when examining the impact of shifting effort-reward associations. Instead of ignoring or reducing order effects, the sequence of experience (e.g. for shifts in ERB) could be manipulated to enhance or reduce value of outcomes or effort itself.

The role of dopamine in dynamic effort-reward integration

When deciding to act, the neurotransmitter dopamine is implicated in a valuation of prospective effort and reward. However, its role in dynamic effort-reward integration during action, a process central to everyday behaviour, remains unclear. In a placebo-controlled, within-subject, study, we probed the impact of increasing brain dopamine levels (150 mg of levodopa) and blocking dopamine receptors (1.5 mg of haloperidol) in the context of a novel dynamic effort task in healthy human subjects. We show that modulating homoeostatic dopamine balance distinctly alters implicit and explicit effort allocation as a function of instantaneous reward. Pharmacologically boosting dopamine enhanced motor vigour, reflected in an implicit increase in effort allocation for high rewards. Conversely, pharmacological blockade of dopamine attenuated sensitivity to differences in reward context, reflected in reduced strategic effort discounting. These findings implicate dopamine in an integration of momentary physical experience and instantaneous reward, suggesting a key role of dopamine in acting to maximise reward on the fly.

Dissociating self-generated volition from externally-generated motivation

Insight into motivational processes may be gained by examining measures of willingness to exert effort for rewards, which have been linked to neuropsychiatric symptoms of anhedonia and apathy. However, while much work has focused on the development of models of motivation based on classic tasks of externally-generated levels of effort for reward, there has been less focus on the question of self-generated motivation or volition. We developed a task that aims to capture separate measures of self-generated and externally-generated motivation, with two task variants for physical and cognitive effort, and sought to test and validate this measure in two populations of healthy volunteers (N = 27 and N = 28). Similar to previous reports, a sigmoid function represented a better overall fit to the effort-reward data than a linear or Weibull model. Individual sigmoid function shapes were governed by two free parameters: bias (the amount of reward needed for effort initiation) and reward insensitivity (the amount of increase in reward needed to accelerate effort expenditure). For both physical and cognitive effort, bias was higher in the self-generated condition, indicating reduced self-generated volitional effort initiation, compared to externally-generated effort initiation, across effort domains. Bias against initial effort initiation in the self-generated condition was related to a specific dimensional measure of anticipatory anhedonia. For physical effort only, reward insensitivity was also higher in the self-generated condition compared to the externally-generated motivation condition, indicating lower self-generated effort acceleration. This work provides a novel objective measure of self-generated motivation that may provide insights into mechanisms of anhedonia and related symptoms.

Effort-based decision making varies by smoking status

RATIONALE

A reduced willingness to perform effort based on the magnitude and probability of potential rewards has been associated with diminished dopamine function and may be relevant to chronic drug use.

OBJECTIVES

Here, we investigated the influence of smoking status on effort-based decisions. We hypothesized that smokers would make fewer high-effort selections than ex-smokers and never-smokers.

METHODS

Current smokers (n = 25), ex-smokers (≥ 1 year quit, n = 23), and never-smokers (n = 19) completed the Effort Expenditure for Rewards Task in which participants select between low-effort and high-effort options to receive monetary rewards at varying levels of reward magnitude, probability and expected value.

RESULTS

Overall, participants selected more high-effort options as potential reward magnitude and expected value increased. Smokers did not make fewer high-effort selections overall, but smokers were less sensitive to the changes in magnitude, probability, and expected value compared to never-smokers. Smokers were also less sensitive to the changes in probability and expected value, but not magnitude, compared to ex-smokers. Among smokers and ex-smokers, less nicotine dependence was associated with an increased likelihood of high-effort selections.

CONCLUSIONS

These results demonstrate the relevance of smoking status to effort-based decisions and suggest that smokers have diminished sensitivity to nondrug reward value. Among ex-smokers, greater pre-existing sensitivity to reward value may have been conducive to smoking cessation, or sensitivity was improved by smoking cessation. Future prospective studies can investigate whether effort-related decision making is predictive of smoking initiation or cessation success.

IMPLICATIONS

Willingness to perform effort to achieve a goal and sensitivity to changes in reward value are important aspects of motivation. These results showed that smokers have decreased sensitivity to changes in effort-related reward probability and expected value compared to ex-smokers and never-smokers. Potentially, improved sensitivity to rewards among ex-smokers may be a cause or consequence of smoking cessation. These findings may help explain why some smokers are able to achieve long-term abstinence.

Modeling Dynamic Allocation of Effort in a Sequential Task Using Discounting Models

Most rewards in our lives require effort to obtain them. It is known that effort is seen by humans as carrying an intrinsic disutility which devalues the obtainable reward. Established models for effort discounting account for this by using participant-specific discounting parameters inferred from experiments. These parameters offer only a static glance into the bigger picture of effort exertion. The mechanism underlying the dynamic changes in a participant's willingness to exert effort is still unclear and an active topic of research. Here, we modeled dynamic effort exertion as a consequence of effort- and probability-discounting mechanisms during goal reaching, sequential behavior. To do this, we developed a novel sequential decision-making task in which participants made binary choices to reach a minimum number of points. Importantly, the time points and circumstances of effort allocation were decided by participants according to their own preferences and not imposed directly by the task. Using the computational model to analyze participants' choices, we show that the dynamics of effort exertion arise from a combination of changing task needs and forward planning. In other words, the interplay between a participant's inferred discounting parameters is sufficient to explain the dynamic allocation of effort during goal reaching. Using formal model comparison, we also inferred the forward-planning strategy used by participants. The model allowed us to characterize a participant's effort exertion in terms of only a few parameters. Moreover, the model can be adapted to a number of tasks used in establishing the neural underpinnings of forward-planning behavior and meta-control, allowing for the characterization of behavior in terms of model parameters.