Dopaminergic modulation of reward discounting in healthy rats: a systematic review and meta-analysis

Involvement of dopamine D3 receptor in impulsive choice decision-making in male rats

Impulsive decision-making has been linked to impulse control disorders and substance use disorders. However, the neural mechanisms underlying impulsive choice are not fully understood. While previous PET imaging and autoradiography studies have shown involvement of dopamine and D2/3 receptors in impulsive behavior, the roles of distinct D1, D2, and D3 receptors in impulsive decision-making remain unclear. In this study, we used a food reward delay-discounting task (DDT) to identify low- and high-impulsive rats, in which low-impulsive rats exhibited preference for large delayed reward over small immediate rewards, while high-impulsive rats showed the opposite preference. We then examined D1, D2, and D3 receptor gene expression using RNAscope in situ hybridization assays. We found that high-impulsive male rats exhibited lower levels of D2 and D3, and particularly D3, receptor expression in the nucleus accumbens (NAc), with no significant changes in the insular, prelimbic, and infralimbic cortices. Based on these findings, we further explored the role of the D3 receptor in impulsive decision-making. Systemic administration of a selective D3 receptor agonist (FOB02-04) significantly reduced impulsive choices in high-impulsive rats but had no effects in low-impulsive rats. Conversely, a selective D3 receptor antagonist (VK4-116) produced increased both impulsive and omission choices in both groups of rats. These findings suggest that impulsive decision-making is associated with a reduction in D3 receptor expression in the NAc. Selective D3 receptor agonists, but not antagonists, may hold therapeutic potentials for mitigating impulsivity in high-impulsive subjects.

No effects of acute stress on monetary delay discounting: A systematic literature review and meta-analysis

Many everyday decisions, including those concerning our health, finances and the environment, involve choosing between a smaller but imminent reward (e.g., €20 now) and a later but larger reward (e.g., €40 in a month). The extent to which an individual prefers smaller imminent rewards over larger delayed rewards can be measured using delay discounting tasks. Acute stress induces a cascade of biological and psychological responses with potential consequences for how individuals think about the future, process rewards, and make decisions, all of which can impact delay discounting. Several studies have shown that individuals focus more on imminent rewards under stress. These findings have been used to explain why individuals make detrimental choices under acute stress. Yet, the evidence linking acute stress to delay discounting is equivocal. To address this uncertainty, we conducted a meta-analysis of 11 studies (14 effects) to systematically quantify the effects of acute stress on monetary delay discounting. Overall, we find no effect of acute stress on delay discounting, compared to control conditions (SMD = -0.18, 95% CI [-0.57, 0.20], p = 0.32). We also find that neither the gender/sex of the participants, the type of stressor (e.g., physical vs. psychosocial) nor whether monetary decisions were hypothetical or incentivized (i.e. monetary decisions were actually paid out) moderated the impact of acute stress on monetary delay discounting. We argue that establishing the effects of acute stress on the separate processes involved in delay discounting, such as reward valuation and prospection, will help to resolve the inconsistencies in the field.

Methylphenidate, but not citalopram, decreases impulsive choice in rats performing a temporal discounting task

Introduction

Drugs targeting monoamine systems remain the most common treatment for disorders with impulse control impairments. There is a body of literature suggesting that drugs affecting serotonin reuptake and dopamine reuptake can modulate distinct aspects of impulsivity - though such tests are often performed using distinct behavioral tasks prohibiting easy comparisons.

Methods

Here, we directly compare pharmacologic agents that affect dopamine (methylphenidate) vs serotonin (citalopram) manipulations on choice impulsivity in a temporal discounting task where rats could choose between a small, immediate reward or a large reward delayed at either 2 or 10s. In control conditions, rats preferred the large reward at a small (2s) delay and discounted the large reward at a long (10s) delay.

Results

Methylphenidate, a dopamine transport inhibitor that blocks reuptake of dopamine, dose-dependently increased large reward preference in the long delay (10s) block. Citalopram, a selective serotonin reuptake inhibitor, had no effect on temporal discounting behavior. Impulsive behavior on the temporal discounting task was at least partially mediated by the nucleus accumbens shell. Bilateral lesions to the nucleus accumbens shell reduced choice impulsivity during the long delay (10s) block. Following lesions, methylphenidate did not impact impulsivity.

Discussion

Our results suggest that striatal dopaminergic systems modulate choice impulsivity via actions within the nucleus accumbens shell, whereas serotonin systems may regulate different aspects of behavioral inhibition/impulsivity.

Dopaminergic modulation of sensitivity to immediate and delayed punishment during decision-making

Effective decision-making involves careful consideration of all rewarding and aversive outcomes. Importantly, negative outcomes often occur later in time, leading to underestimation, or "discounting," of these consequences. Despite the frequent occurrence of delayed outcomes, little is known about the neurobiology underlying sensitivity to delayed punishment during decision-making. The Delayed Punishment Decision-making Task (DPDT) addresses this by assessing sensitivity to delayed versus immediate punishment in rats. Rats initially avoid punished reinforcers, then select this option more frequently when delay precedes punishment. We used DPDT to examine effects of acute systemic administration of catecholaminergic drugs on sensitivity to delayed punishment in male and female adult rats. Cocaine did not affect choice of rewards with immediate punishment but caused a dose-dependent reduction in choice of delayed punishment. Neither activation nor blockade of D1-like dopamine receptor affected decision-making, but activation of D2-like dopamine receptors reduced choice of delayed punishment. D2 blockade did not attenuate cocaine's effects on decision-making, suggesting that cocaine's effects are not dependent on D2 receptor activation. Increasing synaptic norepinephrine via atomoxetine also reduced choice of delayed (but not immediate) punishment. Notably, when DPDT was modified from ascending to descending pre-punishment delays, these drugs did not affect choice of delayed or immediate punishment, although high-dose quinpirole impaired behavioral flexibility. In summary, sensitivity to delayed punishment is regulated by both dopamine and norepinephrine transmission in task-specific fashion. Understanding the neurochemical modulation of decision-making with delayed punishment is a critical step toward treating disorders characterized by aberrant sensitivity to negative consequences.

The influence of dopamine autoreceptors on temperament and addiction risk

As a major regulator of dopamine (DA), DA autoreceptors (DAARs) exert substantial influence over DA-mediated behaviors. This paper reviews the physiological and behavioral impact of DAARs. Individual differences in DAAR functioning influences temperamental traits such as novelty responsivity and impulsivity, both of which are associated with vulnerability to addictive behavior in animal models and a broad array of externalizing behaviors in humans. DAARs additionally impact the response to psychostimulants and other drugs of abuse. Human PET studies of D2-like receptors in the midbrain provide evidence for parallels to the animal literature. These data lead to the proposal that weak DAAR regulation is a risk factor for addiction and externalizing problems. The review highlights the potential to build translational models of the functional role of DAARs in behavior. It also draws attention to key limitations in the current literature that would need to be addressed to further advance a weak DAAR regulation model of addiction and externalizing risk.

Pharmacological Modulation of Temporal Discounting: A Systematic Review

Temporal discounting is a phenomenon where a reward loses its value as a function of time (e.g., a reward is more valuable immediately than when it delays in time). This is a type of intertemporal decision-making that has an association with impulsivity and self-control. Many pathologies exhibit higher discounting rates, meaning they discount more the values of rewards, such as addictive behaviors, bipolar disorder, attention-deficit/hyperactivity disorders, social anxiety disorders, and major depressive disorder, among others; thus, many studies look for the mechanism and neuromodulators of these decisions. This systematic review aims to investigate the association between pharmacological administration and changes in temporal discounting. A search was conducted in PubMed, Scopus, Web of Science, Science Direct and Cochrane. We used the PICO strategy: healthy humans (P-Participants) that received a pharmacological administration (I-Intervention) and the absence of a pharmacological administration or placebo (C-Comparison) to analyze the relationship between the pharmacological administration and the temporal discounting (O-outcome). Nineteen studies fulfilled the inclusion criteria. The most important findings were the involvement of dopamine modulation in a U-shape for choosing the delayed outcome (metoclopradime, haloperidol, and amisulpride). Furthermore, administration of tolcapone and high doses of d-amphetamine produced a preference for the delayed option. There was a time-dependent hydrocortisone effect in the preference for the immediate reward. Thus, it can be concluded that dopamine is a crucial modulator for temporal discounting, especially the D2 receptor, and cortisol also has an important time-dependent role in this type of decision. One of the limitations of this systematic review is the heterogeneity of the drugs used to assess the effect of temporal discounting.

Pre-clinical evidence that methylphenidate increases motivation and/or reward preference to search for high value rewards

Methylphenidate is a stimulant used to treat attention deficit and hyperactivity disorder (ADHD). In the last decade, illicit use of methylphenidate has increased among healthy young adults, who consume the drug under the assumption that it will improve cognitive performance. However, the studies that aimed to assess the methylphenidate effects on memory are not consistent. Here, we tested whether the effect of methylphenidate on a spatial memory task can be explained as a motivational and/or a reward effect. We tested the effects of acute and chronic i.p. administration of 0.3, 1 or 3 mg/kg of methylphenidate on motivation, learning and memory by using the 8-arm radial maze task. Adult male Wistar rats learned that 3 of the 8 arms of the maze were consistently baited with 1, 3, or 6 sucrose pellets, and the number of entries and reentries into reinforced and non-reinforced arms of the maze were scored. Neither acute nor chronic (20 days) methylphenidate treatment affected the number of entries in the non-baited arms. However, chronic, but not acute, 1-3 mg/kg methylphenidate increased the number of reentries in the higher reward arms, which suggests a motivational/rewarding effect rather than a working memory deficit. In agreement with this hypothesis, the methylphenidate treatment also decreased the approach latency to the higher reward arms, increased the approach latency to the low reward arm, and increased the time spent in the high, but not low, reward arm. These findings suggest that methylphenidate may act more as a motivational enhancer rather than a cognitive enhancer in healthy people.

Fronto-striatal connectivity patterns account for the impact of methylphenidate on choice impulsivity among healthy adults

Choice impulsivity depicts a preference towards smaller-sooner rewards over larger-delayed rewards, and is often assessed using a delay discounting (DD) task. Previous research uncovered the prominent role of dopaminergic signaling within fronto-striatal circuits in mediating choice impulsivity. Administration of methylphenidate (MPH), an indirect dopaminergic agonist, was shown to reduce choice impulsivity in animals and pathological populations, although significant inter-individual variability in these effects was reported. Whether MPH impacts choice impulsivity among healthy individuals, and whether variability in the impact of MPH is related to fronto-striatal activation and connectivity patterns, has yet to be assessed. Here, fifty-seven healthy young adults completed the DD task twice during fMRI scans, after acute administration of either MPH (20 mg) or placebo, in a randomized double-blind placebo-controlled design. Acute MPH administration was found to reduce choice impulsivity at the group level, yet substantial variability in this behavioral response was observed. MPH was also found to increase activation in the bilateral putamen and the right caudate, and to enhance functional connectivity between the left putamen and medial prefrontal cortex (mPFC), particularly during non-impulsive choices. Notably, the more putamen-mPFC functional connectivity increased during non-impulsive choices following MPH administration, the less an individual was likely to make impulsive choices. These findings reveal, for the first time in healthy adults, that acute MPH administration is associated with reduced choice impulsivity and increased striatal activation and fronto-striatal connectivity; and furthermore, that the magnitude of MPH-induced change in fronto-striatal connectivity may account for individual differences in the impact of MPH on impulsive behavior.

Global Cerebral Ischemia in Male Long Evans Rats Impairs Dopaminergic/ΔFosB Signalling in the Mesocorticolimbic Pathway Without Altering Delay Discounting Rates

Global cerebral ischemia (GCI) in rats has been shown to promote exploration of anxiogenic zones of the Elevated-Plus Maze (EPM) and Open Field Test (OFT). This study investigated changes in impulsive choice and/or defensive responses as possible contributors of heightened anxiogenic exploration observed after ischemia. Impulsivity was assessed using delay discounting (DD) paradigms, while the Predator Odour Test (PO) served to assess changes in defensive responses towards a naturally aversive stimulus. Male Long Evans rats underwent 9 days of autoshaping training and 24 days of DD training prior to GCI or sham surgery (n = 9/group). Post-surgery, rats completed the OFT, EPM, and PO, followed by 6 days of DD sessions. Blood droplets served to evaluate corticosterone secretion associated with PO exposure. With impulsivity being regulated through mesocorticolimbic monoaminergic pathways, we also characterised post-ischemic changes in the expression of dopamine D₂ receptors (DRD₂), dopamine transporters (DAT), and 1FosB in the basolateral amygdala (BLA), nucleus accumbens core (NAcC) and shell (NAcS), and ventromedial prefrontal cortex (vmPFC) using immunohistofluorescence. Our findings revealed no impact of GCI on delay discounting rates, while PO approach behaviours were minimally affected. Nonetheless, GCI significantly reduced DRD₂ and ΔFosB-ir in the NAcS and NAcC, respectively, while DAT-ir was diminished in both NAc subregions. Collectively, our findings refine the understanding of cognitive-behavioural and biochemical responses following stroke or cardiac arrest. They support significant alterations to the dopaminergic mesocorticolimbic pathway after ischemia, which are not associated with altered impulsive choice in a DD task but may influence locomotor exploration of the OFT and EPM.

Influences of dopaminergic system dysfunction on late-life depression

Deficits in cognition, reward processing, and motor function are clinical features relevant to both aging and depression. Individuals with late-life depression often show impairment across these domains, all of which are moderated by the functioning of dopaminergic circuits. As dopaminergic function declines with normal aging and increased inflammatory burden, the role of dopamine may be particularly salient for late-life depression. We review the literature examining the role of dopamine in the pathogenesis of depression, as well as how dopamine function changes with aging and is influenced by inflammation. Applying a Research Domain Criteria (RDoC) Initiative perspective, we then review work examining how dopaminergic signaling affects these domains, specifically focusing on Cognitive, Positive Valence, and Sensorimotor Systems. We propose a unified model incorporating the effects of aging and low-grade inflammation on dopaminergic functioning, with a resulting negative effect on cognition, reward processing, and motor function. Interplay between these systems may influence development of a depressive phenotype, with an initial deficit in one domain reinforcing decline in others. This model extends RDoC concepts into late-life depression while also providing opportunities for novel and personalized interventions.

The sooner the better: clinical and neural correlates of impulsive choice in Tourette disorder

Reward sensitivity has been suggested as one of the central pathophysiological mechanisms in Tourette disorder. However, the subjective valuation of a reward by introduction of delay has received little attention in Tourette disorder, even though it has been suggested as a trans-diagnostic feature of numerous neuropsychiatric disorders. We aimed to assess delay discounting in Tourette disorder and to identify its brain functional correlates. We evaluated delayed discounting and its brain functional correlates in a large group of 54 Tourette disorder patients and 31 healthy controls using a data-driven approach. We identified a subgroup of 29 patients with steeper reward discounting, characterised by a higher burden of impulse-control disorders and a higher level of general impulsivity compared to patients with normal behavioural performance or to controls. Reward discounting was underpinned by resting-state activity of a network comprising the orbito-frontal, cingulate, pre-supplementary motor area, temporal and insular cortices, as well as ventral striatum and hippocampus. Within this network, (i) lower connectivity of pre-supplementary motor area with ventral striatum predicted a higher impulsivity and a steeper reward discounting and (ii) a greater connectivity of pre-supplementary motor area with anterior insular cortex predicted steeper reward discounting and more severe tics. Overall, our results highlight the heterogeneity of the delayed reward processing in Tourette disorder, with steeper reward discounting being a marker of burden in impulsivity and impulse control disorders, and the pre-supplementary motor area being a hub region for the delay discounting, impulsivity and tic severity.

Effects of tDCS on reward responsiveness and valuation in Parkinson's patients with impulse control disorders

Parkinson's disease (PD) patients with impulse control disorders (ICD) frequently report hypersensitivity to rewards. However, a few studies have explored the effectiveness of modulation techniques on symptoms experienced by these patients. In this study, we assessed the effect of anodal tDCS over the DLPFC on reward responsiveness and valuation in PD patients with ICD. 43 participants (15 PD patients with ICD, 13 PD without ICD, and 15 healthy matched controls) were asked to perform a reward-craving test employing both explicit (self-ratings of liking and wanting) and implicit (heart rate and skin conductance response) measures, as well as two temporal discounting tasks with food and money rewards. Each participant performed the experimental tasks during active anodal tDCS of the left dorsolateral prefrontal cortex (DLPFC), anodal tDCS of the primary motor cortex (M1), and sham tDCS. Results showed increased wanting and a steeper temporal discounting of rewards in PD with ICD compared to the other groups. Moreover, we found that PD without ICD exhibit reduced liking for rewards. tDCS results capable to modulate the altered intensity of PD patients' liking, but not wanting and temporal discounting of rewards in PD patients with ICD. These findings confirm that alterations in reward responsiveness and valuation are characteristics of impulse control disorders in patients with PD but suggest that anodal tDCS over the left DLPFC is not capable to influence these processes. At the same time, they provide new insight into affective experience of rewards in PD.

Rational inattention and tonic dopamine

Slow-timescale (tonic) changes in dopamine (DA) contribute to a wide variety of processes in reinforcement learning, interval timing, and other domains. Furthermore, changes in tonic DA exert distinct effects depending on when they occur (e.g., during learning vs. performance) and what task the subject is performing (e.g., operant vs. classical conditioning). Two influential theories of tonic DA-the average reward theory and the Bayesian theory in which DA controls precision-have each been successful at explaining a subset of empirical findings. But how the same DA signal performs two seemingly distinct functions without creating crosstalk is not well understood. Here we reconcile the two theories under the unifying framework of 'rational inattention,' which (1) conceptually links average reward and precision, (2) outlines how DA manipulations affect this relationship, and in so doing, (3) captures new empirical phenomena. In brief, rational inattention asserts that agents can increase their precision in a task (and thus improve their performance) by paying a cognitive cost. Crucially, whether this cost is worth paying depends on average reward availability, reported by DA. The monotonic relationship between average reward and precision means that the DA signal contains the information necessary to retrieve the precision. When this information is needed after the task is performed, as presumed by Bayesian inference, acute manipulations of DA will bias behavior in predictable ways. We show how this framework reconciles a remarkably large collection of experimental findings. In reinforcement learning, the rational inattention framework predicts that learning from positive and negative feedback should be enhanced in high and low DA states, respectively, and that DA should tip the exploration-exploitation balance toward exploitation. In interval timing, this framework predicts that DA should increase the speed of the internal clock and decrease the extent of interference by other temporal stimuli during temporal reproduction (the central tendency effect). Finally, rational inattention makes the new predictions that these effects should be critically dependent on the controllability of rewards, that post-reward delays in intertemporal choice tasks should be underestimated, and that average reward manipulations should affect the speed of the clock-thus capturing empirical findings that are unexplained by either theory alone. Our results suggest that a common computational repertoire may underlie the seemingly heterogeneous roles of DA.

The efficacy of real-time functional magnetic resonance imaging neurofeedback for psychiatric illness: A meta-analysis of brain and behavioral outcomes

Real-time functional magnetic resonance imaging neurofeedback (rtfMRI-NF) has gained popularity as an experimental treatment for a variety of psychiatric illnesses. However, there has yet to be a quantitative review regarding its efficacy. Here, we present the first meta-analysis of rtfMRI-NF for psychiatric disorders, evaluating its impact on brain and behavioral outcomes. Our literature review identified 17 studies and 105 effect sizes across brain and behavioral outcomes. We find that rtfMRI-NF produces a medium-sized effect on neural activity during training (g = .59, 95 % CI [.44, .75], p < .0001), a large-sized effect after training when no neurofeedback is provided (g = .84, 95 % CI [.37, 1.31], p = .005), and small-sized effects for behavioral outcomes (symptoms g = .37, 95 % CI [.16, .58], p = .002; cognition g = .23, 95 % CI [-.33, .78], p = .288). Mixed-effects analyses revealed few moderators. Together, these data suggest a positive impact of rtfMRI-NF on brain and behavioral outcomes, although more research is needed to determine how rtfMRI-NF works, for whom, and under what circumstances.