Dopaminergic modulation of the trade-off between probability and time in economic decision-making

Pharmacological Modulation of Temporal Discounting: A Systematic Review

Temporal discounting is a phenomenon where a reward loses its value as a function of time (e.g., a reward is more valuable immediately than when it delays in time). This is a type of intertemporal decision-making that has an association with impulsivity and self-control. Many pathologies exhibit higher discounting rates, meaning they discount more the values of rewards, such as addictive behaviors, bipolar disorder, attention-deficit/hyperactivity disorders, social anxiety disorders, and major depressive disorder, among others; thus, many studies look for the mechanism and neuromodulators of these decisions. This systematic review aims to investigate the association between pharmacological administration and changes in temporal discounting. A search was conducted in PubMed, Scopus, Web of Science, Science Direct and Cochrane. We used the PICO strategy: healthy humans (P-Participants) that received a pharmacological administration (I-Intervention) and the absence of a pharmacological administration or placebo (C-Comparison) to analyze the relationship between the pharmacological administration and the temporal discounting (O-outcome). Nineteen studies fulfilled the inclusion criteria. The most important findings were the involvement of dopamine modulation in a U-shape for choosing the delayed outcome (metoclopradime, haloperidol, and amisulpride). Furthermore, administration of tolcapone and high doses of d-amphetamine produced a preference for the delayed option. There was a time-dependent hydrocortisone effect in the preference for the immediate reward. Thus, it can be concluded that dopamine is a crucial modulator for temporal discounting, especially the D2 receptor, and cortisol also has an important time-dependent role in this type of decision. One of the limitations of this systematic review is the heterogeneity of the drugs used to assess the effect of temporal discounting.

A process model account of the role of dopamine in intertemporal choice

Theoretical accounts disagree on the role of dopamine in intertemporal choice and assume that dopamine either promotes delay of gratification by increasing the preference for larger rewards or that dopamine reduces patience by enhancing the sensitivity to waiting costs. Here, we reconcile these conflicting accounts by providing empirical support for a novel process model according to which dopamine contributes to two dissociable components of the decision process, evidence accumulation and starting bias. We re-analyzed a previously published data set where intertemporal decisions were made either under the D2 antagonist amisulpride or under placebo by fitting a hierarchical drift diffusion model that distinguishes between dopaminergic effects on the speed of evidence accumulation and the starting point of the accumulation process. Blocking dopaminergic neurotransmission not only strengthened the sensitivity to whether a reward is perceived as worth the delay costs during evidence accumulation (drift rate) but also attenuated the impact of waiting costs on the starting point of the evidence accumulation process (bias). In contrast, re-analyzing data from a D1 agonist study provided no evidence for a causal involvement of D1R activation in intertemporal choices. Taken together, our findings support a novel, process-based account of the role of dopamine for cost-benefit decision making, highlight the potential benefits of process-informed analyses, and advance our understanding of dopaminergic contributions to decision making.

Towards a Unifying Account of Dopamine’s Role in Cost-Benefit Decision Making

Dopamine is thought to play a crucial role in cost-benefit decision making, but so far there is no consensus on the precise role of dopamine in decision making. Here, we review the literature on dopaminergic manipulations of cost-benefit decision making in humans and evaluate how well different theoretical accounts explain the existing body of evidence. Reduced D₂ stimulation tends to increase the willingness to bear delay and risk costs (i.e., wait for later rewards, take riskier options), while increased D₁ and D₂ receptor stimulation increases willingness to bear effort costs. We argue that the empirical findings can best be explained by combining the strengths of two theoretical accounts: in cost-benefit decision making, dopamine may play a dual role both in promoting the pursuit of psychologically close options (e.g., sooner and safer rewards) and in computing which costs are acceptable for a reward at stake. Moreover, we identify several limiting factors in the study designs of previous investigations that prevented a fuller understanding of dopamine's role in value-based choice. Together, the proposed theoretical framework and the methodological suggestions for future studies may bring us closer to a unifying account of dopamine in healthy and impaired cost-benefit decision making.

Temporal discounting in adolescents and adults with Tourette syndrome

Tourette syndrome is a neurodevelopmental disorder associated with hyperactivity in dopaminergic networks. Dopaminergic hyperactivity in the basal ganglia has previously been linked to increased sensitivity to positive reinforcement and increases in choice impulsivity. In this study, we examine whether this extends to changes in temporal discounting, where impulsivity is operationalized as an increased preference for smaller-but-sooner over larger-but-later rewards. We assessed intertemporal choice in two studies including nineteen adolescents (age: mean[sd] = 14.21[±2.37], 13 male subjects) and twenty-five adult patients (age: mean[sd] = 29.88 [±9.03]; 19 male subjects) with Tourette syndrome and healthy age- and education matched controls. Computational modeling using exponential and hyperbolic discounting models via hierarchical Bayesian parameter estimation revealed reduced temporal discounting in adolescent patients, and no evidence for differences in adult patients. Results are discussed with respect to neural models of temporal discounting, dopaminergic alterations in Tourette syndrome and the developmental trajectory of temporal discounting. Specifically, adolescents might show attenuated discounting due to improved inhibitory functions that also affect choice impulsivity and/or the developmental trajectory of executive control functions. Future studies would benefit from a longitudinal approach to further elucidate the developmental trajectory of these effects.

Using pharmacological manipulations to study the role of dopamine in human reward functioning: A review of studies in healthy adults

Dopamine (DA) plays a key role in reward processing and is implicated in psychological disorders such as depression, substance use, and schizophrenia. The role of DA in reward processing is an area of highly active research. One approach to this question is drug challenge studies with drugs known to alter DA function. These studies provide good experimental control and can be performed in parallel in laboratory animals and humans. This review aimed to summarize results of studies using pharmacological manipulations of DA in healthy adults. 'Reward' is a complex process, so we separated 'phases' of reward, including anticipation, evaluation of cost and benefits of upcoming reward, execution of actions to obtain reward, pleasure in response to receiving a reward, and reward learning. Results indicated that i) DAergic drugs have different effects on different phases of reward; ii) the relationship between DA and reward functioning appears unlikely to be linear; iii) our ability to detect the effects of DAergic drugs varies depending on whether subjective, behavioral, imaging measures are used.

Dopaminergic Modulation of Human Intertemporal Choice: A Diffusion Model Analysis Using the D2-Receptor Antagonist Haloperidol

The neurotransmitter dopamine is implicated in diverse functions, including reward processing, reinforcement learning, and cognitive control. The tendency to discount future rewards over time has long been discussed in the context of potential dopaminergic modulation. Here we examined the effect of a single dose of the D2 receptor antagonist haloperidol (2 mg) on temporal discounting in healthy female and male human participants. Our approach extends previous pharmacological studies in two ways. First, we applied combined temporal discounting drift diffusion models to examine choice dynamics. Second, we examined dopaminergic modulation of reward magnitude effects on temporal discounting. Hierarchical Bayesian parameter estimation revealed that the data were best accounted for by a temporal discounting drift diffusion model with nonlinear trialwise drift rate scaling. This model showed good parameter recovery, and posterior predictive checks revealed that it accurately reproduced the relationship between decision conflict and response times in individual participants. We observed reduced temporal discounting and substantially faster nondecision times under haloperidol compared with placebo. Discounting was steeper for low versus high reward magnitudes, but this effect was largely unaffected by haloperidol. Results were corroborated by model-free analyses and modeling via more standard approaches. We previously reported elevated caudate activation under haloperidol in this sample of participants, supporting the idea that haloperidol elevated dopamine neurotransmission (e.g., by blocking inhibitory feedback via presynaptic D2 auto-receptors). The present results reveal that this is associated with an augmentation of both lower-level (nondecision time) and higher-level (temporal discounting) components of the decision process.SIGNIFICANCE STATEMENT Dopamine is implicated in reward processing, reinforcement learning, and cognitive control. Here we examined the effects of a single dose of the D2 receptor antagonist haloperidol on temporal discounting and choice dynamics during the decision process. We extend previous studies by applying computational modeling using the drift diffusion model, which revealed that haloperidol reduced the nondecision time and reduced impulsive choice compared with placebo. These findings are compatible with a haloperidol-induced increase in striatal dopamine (e.g., because of a presynaptic mechanism). Our data provide novel insights into the contributions of dopamine to value-based decision-making and highlight how comprehensive model-based analyses using sequential sampling models can inform the effects of pharmacological modulation on choice processes.

Human Brain Imaging Links Dopaminergic Systems to Impulsivity

Molecular and functional imaging techniques have been used and combined with pharmacological probes to evaluate the role of dopamine in impulsivity. Overall, strong evidence links striatal dopaminergic function with impulsivity, measured by self-reports and laboratory tests of cognitive control and reward-based decision-making. The combination of molecular imaging using positron emission tomography (PET) with functional magnetic resonance imaging (fMRI) specifically implicates striatal D2-type dopamine receptors (i.e., D2 and D3) and corticostriatal connectivity in cognitive control. Low levels of striatal and midbrain D2-type receptor availability correlate with self-reported impulsivity, whereas striatal D2-type receptor availability shows positive correlation with motor response inhibition and cognitive flexibility. Impulsive choice on reward-based decision-making tasks also is related to deficits in striatal D2-type dopamine receptor availability, and there is evidence for an inverted U-shaped function in this relationship, reflecting an optimum of striatal dopaminergic activity. Findings from studies of clinical populations that present striatal dopamine D2-type receptor deficits as well as healthy control research participants identify D2-type receptors as therapeutic targets to improve cognitive control.

A neural basis for food foraging in obesity

Poverty-related food insecurity can be viewed as a form of economic and nutritional uncertainty that can lead, in some situations, to a desire for more filling and satisfying food. Given the current obesogenic food environment and the nature of the food supply, those food choices could engage a combination of sensory, neurophysiological, and genetic factors as potential determinants of obesity.

Baseline impulsivity may moderate L-DOPA effects on value-based decision-making

Research has indicated a major role of dopamine in decision-making processes, but the underlying mechanisms remain largely unknown due to inconsistency in effects of dopaminergic drugs. To clarify the impact of dopamine on impulsive choice, we administered 150 mg L-DOPA to 87 healthy adults in a randomized, placebo-controlled, double-blind, crossover study, evaluating performance in four value-based decision-making tasks. We predicted that baseline impulsivity would moderate L-DOPA effects. In support of our hypothesis, L-DOPA had no main effect on impulsive choice, but reduced risk-seeking for gains in more-impulsive subjects. Because L-DOPA effects may be influenced by body weight, we repeated our analyses on data from half of the sample (n = 44) with lower weight, anticipating a stronger effect. In addition to the effect on risk-seeking for gains, low-weight participants also exhibited baseline-dependent effects of L-DOPA on loss aversion and delay discounting. Our results are consistent with the hypothesis of an inverted U-shaped dopamine function in which both low and high extremes of dopamine signaling are associated with high-impulsive choice. Consideration of differential baseline impulsivity and body weight may resolve previous seemingly paradoxical pharmacological results and might deepen our understanding of dopaminergic mechanisms underlying impulsivity.

Neural correlates of effort-based behavioral inconsistency

According to the theory of value-based decision making, subjects tend to choose the most valuable among a set of options. However, agents may not be consistent when facing the same decision several times. In this paper, Shannon's entropy (H) is employed as a measure of behavioral inconsistency: it is a central measure of information theory that, applied to decision making, allows the estimation of behavioral preferences among a set of options. We scanned (functional magnetic resonance imaging, fMRI) 24 young (18-25 year) subjects (14 female) while performing a decision-making task, where monetary rewards were devalued by physical effort (minutes running in the treadmill) and risk. Twenty different pairs of options were presented nine times each, and H was calculated for each pair and subject. Behavioral analyses showed that subjective value (SV) significantly explained agents' preferences only in pairs with a low inconsistent response. Averaged response time positively correlated with H, confirming entropy as an indicator of choice difficulty. Group analyses on fMRI data revealed a cluster in the paracingulate cortex as the neural correlate of H. Besides, BOLD signal in the posterior cingulate correlated with the SV of the pair only in consistent decisions, confirming that SV loses its explanatory power on highly inconsistent decisions. Finally, the anterior and central cingulate were especially recruited when predicting a secured effortless reward, compared with a secured reward that involved a maximum effort. Our study shows that different regions of the cingulate cortex are involved in choice inconsistency, SV and processing effort costs.

An fMRI study of decision-making under sunk costs in gambling disorder

The sunk cost effect is the tendency to continue an investment, or take an action, even though it has higher future costs than benefits, if costs of time, money, or effort were previously incurred. This type of decision bias is pervasive in real life and has been studied in various disciplines. Previous studies and clinical observations suggest that decision-making under sunk costs is altered in gambling disorder (GD). However, the neural mechanisms of decision-making under sunk costs in GD remain largely unknown, and so is their association with the clinical characteristics of this patient group. Here, by combining functional magnetic resonance imaging and the task that demonstrated a clear example of the sunk cost effect, we investigated the neural correlates during decision-making under sunk costs in GD. We found no significant differences in the strength of the sunk cost effect between the GD and healthy control (HC) groups. However, the strength of the sunk cost effect in patients with GD showed a significant negative correlation with abstinence period and a marginally significant positive correlation with the duration of illness. We also found a reduction in the neural activation in the dorsal medial prefrontal cortex during decision-making under sunk costs for the GD group compared with the HC group. Furthermore, in patients with GD, the levels of activation in this area negatively correlated with the duration of illness. These findings have important clinical implications. This study will contribute to a better understanding of the mechanisms underlying altered decision-making abilities in GD.

Brain correlates of the intrinsic subjective cost of effort in sedentary volunteers

One key aspect of motivation is the ability of agents to overcome excessive weighting of intrinsic subjective costs. This contribution aims to analyze the subjective cost of effort and assess its neural correlates in sedentary volunteers. We recruited a sample of 57 subjects who underwent a decision-making task using a prospective, moderate, and sustained physical effort as devaluating factor. Effort discounting followed a hyperbolic function, and individual discounting constants correlated with an indicator of sedentary lifestyle (global physical activity questionnaire; R=-0.302, P=0.033). A subsample of 24 sedentary volunteers received a functional magnetic resonance imaging scan while performing a similar effort-discounting task. BOLD signal of a cluster located in the dorsomedial prefrontal cortex correlated with the subjective value of the pair of options under consideration (Z>2.3, P<0.05; cluster corrected for multiple comparisons for the whole brain). Furthermore, effort-related discounting of reward correlated with the signal of a cluster in the ventrolateral prefrontal cortex (Z>2.3, P<0.05; small volume cluster corrected for a region of interest including the ventral prefrontal cortex and striatum). This study offers empirical data about the intrinsic subjective cost of effort and its neural correlates in sedentary individuals.

Insulin, Central Dopamine D2 Receptors, and Monetary Reward Discounting in Obesity

Animal research finds that insulin regulates dopamine signaling and reward behavior, but similar research in humans is lacking. We investigated whether individual differences in body mass index, percent body fat, pancreatic β-cell function, and dopamine D2 receptor binding were related to reward discounting in obese and non-obese adult men and women. Obese (n = 27; body mass index>30) and non-obese (n = 20; body mass index<30) adults were assessed for percent body fat with dual-energy X-ray absorptiometry and for β-cell function using disposition index. Choice of larger, but delayed or less certain, monetary rewards relative to immediate, certain smaller monetary rewards was measured using delayed and probabilistic reward discounting tasks. Positron emission tomography using a non-displaceable D2-specific radioligand, [¹¹C](N-methyl)benperidol quantified striatal D2 receptor binding. Groups differed in body mass index, percent body fat, and disposition index, but not in striatal D2 receptor specific binding or reward discounting. Higher percent body fat in non-obese women related to preference for a smaller, certain reward over a larger, less likely one (greater probabilistic discounting). Lower β-cell function in the total sample and lower insulin sensitivity in obese related to stronger preference for an immediate and smaller monetary reward over delayed receipt of a larger one (greater delay discounting). In obese adults, higher striatal D2 receptor binding related to greater delay discounting. Interestingly, striatal D2 receptor binding was not significantly related to body mass index, percent body fat, or β-cell function in either group. Our findings indicate that individual differences in percent body fat, β-cell function, and striatal D2 receptor binding may each contribute to altered reward discounting behavior in non-obese and obese individuals. These results raise interesting questions about whether and how striatal D2 receptor binding and metabolic factors, including β-cell function, interact to affect reward discounting in humans.