Dopaminergic modulation of risk-based decision making

Noradrenergic modulation of stress induced catecholamine release: Opposing influence of FG7142 and yohimbine

BACKGROUND

Life stress modulates decision making, particularly in the face of risk, in some cases prompting vulnerable populations to make suboptimal, life-altering choices. In the brain, stress is known to alter the extracellular release of catecholamines in structures such as basolateral amygdala (BLA) and nucleus accumbens (NAc), but the relationship between catecholamines and decision-making behavior under stress has not been systemically explored.

METHODS

We developed an operant touchscreen decision-making task for rats comprising elements of loss aversion and risk seeking behavior. Rats were first injected systemically with an adrenergic alpha 2A-receptor agonist (guanfacine) and antagonist (yohimbine), as well as a partial inverse GABAA agonist, FG 7142, known to induce anxiety and stress related physiological responses in a variety of species, including humans. We then used fiber photometry to monitor NE in the basolateral amygdala (BLA), and DA activity in the nucleus accumbens (NAc) while animals engaged in decision-making and following systemic injections of FG 7142 and yohimbine.

RESULTS

Neither yohimbine nor guanfacine had any impact on decision making strategy but altered motivational state with yohimbine making the animal almost insensitive to the reward outcome. The pharmacological induction of stress with FG 7142 biased the rats decisions towards safety, but this bias shifted toward risk when co-treated with yohimbine. In the BLA and NAc, the FG 7142 altered catecholamine release, with systemic yohimbine producing opposing effects on NE and DA release.

CONCLUSIONS

Stress induced changes in catecholamine release in the BLA and NAc can directly influence loss sensitivity, decisions and motivation, which can be modulated by the alpha 2A adrenoceptor antagonist, yohimbine.

Neuropharmacology involvement of dopamine D3 receptor in impulsive choice decision-making in male rats

Impulsive decision-making has been linked to impulse control disorders and substance use disorders. However, the neural mechanisms underlying impulsive choice are not fully understood. While previous PET imaging and autoradiography studies have shown involvement of dopamine and D2/3 receptors in impulsive behavior, the roles of distinct D1, D2, and D3 receptors in impulsive decision-making remain unclear. In this study, we used a food reward delay-discounting task (DDT) to identify low- and high-impulsive rats, in which low-impulsive rats exhibited preference for large delayed reward over small immediate rewards, while high-impulsive rats showed the opposite preference. We then examined D1, D2, and D3 receptor gene expression using RNAscope in situ hybridization assays. We found that high-impulsive male rats exhibited lower levels of D2 and D3, and particularly D3, receptor expression in the nucleus accumbens (NAc), with no significant changes in the insular, prelimbic, and infralimbic cortices. Based on these findings, we further explored the role of the D3 receptor in impulsive decision-making. Systemic administration of a selective D3 receptor agonist (FOB02-04) significantly reduced impulsive choices in high-impulsive rats but had no effects in low-impulsive rats. Conversely, a selective D3 receptor antagonist (VK4-116) produced increased both impulsive and omission choices in both groups of rats. These findings suggest that impulsive decision-making is associated with a reduction in D3 receptor expression in the NAc. Selective D3 receptor agonists, but not antagonists, may hold therapeutic potentials for mitigating impulsivity in high-impulsive subjects.

Perturbations in risk/reward decision making and frontal cortical catecholamine regulation induced by mild traumatic brain injury

Mild traumatic brain injury (mTBI) disrupts cognitive processes that influence risk taking behavior. Little is known regarding the effects of repetitive mild injury (rmTBI) or whether these outcomes are sex specific. Risk/reward decision making is mediated by the prefrontal cortex (PFC), which is densely innervated by catecholaminergic fibers. Aberrant PFC catecholamine activity has been documented following TBI and may underlie TBI-induced risky behavior. The present study characterized the effects of rmTBI on risk/reward decision making behavior and catecholamine transmitter regulatory proteins within the PFC. Rats were exposed to sham, single (smTBI), or three closed-head controlled cortical impact (CH-CCI) injuries and assessed for injury-induced effects on risk/reward decision making using a probabilistic discounting task (PDT). In the first week post-final surgery, mTBI increased risky choice preference. By the fourth week, males exhibited increased latencies to make risky choices following rmTBI, demonstrating a delayed effect on processing speed. When levels of tyrosine hydroxylase (TH) and the norepinephrine reuptake transporter (NET) were measured within subregions of the PFC, females exhibited dramatic increases of TH levels within the orbitofrontal cortex (OFC) following smTBI. However, both males and females demonstrated reduced levels of OFC NET following rmTBI. These results indicate the OFC is susceptible to catecholamine instability after rmTBI and suggests that not all areas of the PFC contribute equally to TBI-induced imbalances. Overall, the CH-CCI model of rmTBI has revealed time-dependent and sex-specific changes in risk/reward decision making and catecholamine regulation following repetitive mild head injuries.

Methamphetamine increases risky choice in rats, but only when magnitude and probability of reinforcement are manipulated within a session

Risky choice is associated with maladaptive behaviors, particularly substance use disorders. Current animal models of risky choice are often confounded by other constructs like behavioral flexibility and suboptimal choice. The purpose of the current experiment was to determine if the psychostimulant methamphetamine, a drug whose popularity has increased in recent years, increases risky choice in an equivalent expected value (EEV) task. In the EEV task, rats are given a choice between two reinforcer alternatives that differ in magnitude and probability of delivery, but have equivalent expected value. Forty-eight Sprague Dawley rats were tested in three versions of the EEV task. In the first version of the EEV task, both reinforcer magnitude and probability were adjusted across blocks of trials for both alternatives. In the second and the third versions of the EEV task, reinforcer magnitude was held constant across each block of trials (either 1 vs. 2 pellets or 4 vs. 5 pellets). We found that male rats preferred the "riskier" option, except when reinforcer magnitudes were held constant at 4 and 5 pellets across each block of trials. Methamphetamine (0.5 mg/kg) increased preference for the risky option in both males and females, but only when both reinforcer magnitude and probability were manipulated across blocks of trials for each alternative. The current results demonstrate that both magnitude of reinforcement and probability of reinforcement interact to influence risky choice. Overall, this study provides additional support for using reinforcers with expected value to measure risky choice.

Adolescent THC impacts on mPFC dopamine-mediated cognitive processes in male and female rats

Rationale

Adolescent cannabis use is linked to later-life changes in cognition, learning, and memory. Rodent experimental studies suggest Δ9-tetrahydrocannabinol (THC) influences development of circuits underlying these processes, especially in the prefrontal cortex, which matures during adolescence.

Objective

We determined how 14 daily THC injections (5mg/kg) during adolescence persistently impacts medial prefrontal cortex (mPFC) dopamine-dependent cognition.

Methods

In adult Long Evans rats treated as adolescents with THC (AdoTHC), we quantify performance on two mPFC dopamine-dependent reward-based tasks-strategy set shifting and probabilistic discounting. We also determined how acute dopamine augmentation with amphetamine (0, 0.25, 0.5 mg/kg), or specific chemogenetic stimulation of ventral tegmental area (VTA) dopamine neurons and their projections to mPFC impacts probabilistic discounting.

Results

AdoTHC sex-dependently impacts acquisition of cue-guided instrumental reward seeking, but has minimal effects on set-shifting or probabilistic discounting in either sex. When we challenged dopamine circuits acutely with amphetamine during probabilistic discounting, we found reduced discounting of improbable reward options, with AdoTHC rats being more sensitive to these effects than controls. In contrast, neither acute chemogenetic stimulation of VTA dopamine neurons nor pathway-specific chemogenetic stimulation of their projection to mPFC impacted probabilistic discounting in control rats, although stimulation of this cortical dopamine projection slightly disrupted choices in AdoTHC rats.

Conclusions

These studies confirm a marked specificity in the cognitive processes impacted by AdoTHC exposure. They also suggest that some persistent AdoTHC effects may alter amphetamine-induced cognitive changes in a manner independent of VTA dopamine projections to mPFC, or via alterations of non-VTA dopamine neurons.

Effects of systemic oxytocin receptor activation and blockade on risky decision making in female and male rats

The neuropeptide oxytocin is traditionally known for its roles in parturition, lactation, and social behavior. Other data, however, show that oxytocin can modulate behaviors outside of these contexts, including drug self-administration and some aspects of cost-benefit decision making. Here we used a pharmacological approach to investigate the contributions of oxytocin signaling to decision making under risk of explicit punishment. Female and male Long-Evans rats were trained on a risky decision-making task in which they chose between a small, "safe" food reward and a large, "risky" food reward that was accompanied by varying probabilities of mild footshock. Once stable choice behavior emerged, rats were tested in the task following acute intraperitoneal injections of oxytocin or the oxytocin receptor antagonist L-368,899. Neither drug affected task performance in males. In females, however, both oxytocin and L-368,899 caused a dose-dependent reduction in preference for large risky reward. Control experiments showed that these effects could not be accounted for by alterations in food motivation or shock sensitivity. Together, these results reveal a sex-dependent effect of oxytocin signaling on risky decision making in rats.

Sex differences in risk-based decision-making and the modulation of risk preference by dopamine-2 like receptors in rats

Heightened risk-based decision-making is observed across several neuropsychiatric disorders including schizophrenia, bipolar disorder, and Parkinson's disease, yet no treatments exist that effectively normalize this aberrant behavior. Preclinical risk-based decision-making paradigms have identified the important modulatory roles of dopamine and sex in the performance of such tasks, though specific task parameters may alter such effects (e.g., punishment and reward values). Previous work has highlighted the role of dopamine 2-like receptors (D2R) during performance of the Risk Preference Task (RPT) in male rats, however sex was not considered as a factor in this study, nor were treatments identified that reduced risk preference. Here, we utilized the RPT to determine sex-dependent differences in baseline performance and impact of the D2R receptor agonist pramipexole (PPX), and antagonist sulpiride (SUL) on behavioral performance. Female rats exhibited heightened risk-preference during baseline testing. Consistent with human studies, PPX increased risk-preference across sex, though the effects of PPX were more pronounced in female animals. Importantly, SUL reduced risk-preference in these rats across sexes. Thus, under the task specifications of the RPT that does not include punishment, female rats were more risk-preferring and required higher PPX doses to promote risky choices compared to males. Furthermore, blockade of D2R receptors may reduce risk-preference of rats, though further studies are required.

Dopaminergic modulation of sensitivity to immediate and delayed punishment during decision-making

Effective decision-making involves careful consideration of all rewarding and aversive outcomes. Importantly, negative outcomes often occur later in time, leading to underestimation, or "discounting," of these consequences. Despite the frequent occurrence of delayed outcomes, little is known about the neurobiology underlying sensitivity to delayed punishment during decision-making. The Delayed Punishment Decision-making Task (DPDT) addresses this by assessing sensitivity to delayed versus immediate punishment in rats. Rats initially avoid punished reinforcers, then select this option more frequently when delay precedes punishment. We used DPDT to examine effects of acute systemic administration of catecholaminergic drugs on sensitivity to delayed punishment in male and female adult rats. Cocaine did not affect choice of rewards with immediate punishment but caused a dose-dependent reduction in choice of delayed punishment. Neither activation nor blockade of D1-like dopamine receptor affected decision-making, but activation of D2-like dopamine receptors reduced choice of delayed punishment. D2 blockade did not attenuate cocaine's effects on decision-making, suggesting that cocaine's effects are not dependent on D2 receptor activation. Increasing synaptic norepinephrine via atomoxetine also reduced choice of delayed (but not immediate) punishment. Notably, when DPDT was modified from ascending to descending pre-punishment delays, these drugs did not affect choice of delayed or immediate punishment, although high-dose quinpirole impaired behavioral flexibility. In summary, sensitivity to delayed punishment is regulated by both dopamine and norepinephrine transmission in task-specific fashion. Understanding the neurochemical modulation of decision-making with delayed punishment is a critical step toward treating disorders characterized by aberrant sensitivity to negative consequences.

Win-Paired Cues Modulate the Effect of Dopamine Neuron Sensitization on Decision Making and Cocaine Self-administration: Divergent Effects Across Sex

BACKGROUND

Both psychostimulant use and engagement with probabilistic schedules of reward sensitize the mesocorticolimbic dopamine (DA) system. Such behaviors may act synergistically to explain the high comorbidity between stimulant use and gambling disorder. The salient audiovisual stimuli of modern electronic gambling may exacerbate the situation.

METHODS

To probe these interactions, we sensitized ventral tegmental area DA neurons via chronic chemogenetic stimulation while rats (n = 134) learned a rat gambling task in the presence or absence of casino-like cues. The same rats then learned to self-administer cocaine. In a separate cohort (n = 25), we confirmed that our chemogenetic methods sensitized the locomotor response to cocaine and potentiated phasic excitability of ventral tegmental area DA neurons through in vivo electrophysiological recordings.

RESULTS

In the absence of cues, sensitization promoted risk taking in both sexes. When rewards were cued, sensitization expedited the development of a risk-preferring phenotype in males while attenuating cue-induced risk taking in females.

CONCLUSIONS

While these results provide further confirmation that ventral tegmental area DA neurons critically modulate risky decision making, they also reveal stark sex differences in the decisional impact that dopaminergic signals exert when winning outcomes are cued. As previously observed, risky decision making on the cued rat gambling task increased as both males and females learned to self-administer cocaine. The combination of DA sensitization and win-paired cues while gambling led to significantly greater cocaine taking, but these rats did not show any increase in risky choice as a result. Therefore, cocaine and heavily cued gambles may partially substitute for each other once the DA system has been rendered labile through sensitization, thereby compounding addiction risk across modalities.

Enhanced Risky Choice in Male Rats Elicited by the Acute Pharmacological Stressor Yohimbine Involves Prefrontal Dopamine D1 Receptor Activation

BACKGROUND

Acute stress alters risk-based decision-making; however, the underlying neural and neurochemical substrates are underexplored. Given their well-documented stress-inducing effects in humans and laboratory animals, glucocorticoids such as cortisol and corticosterone and the α2-adrenoceptor antagonist yohimbine represent potent pharmacological tools to mimic some characteristics of acute stress.

METHODS

Here, we analyzed the effects of the pharmacological stressors corticosterone and yohimbine given systemically on risk-based decision-making in male rats. Moreover, we investigated whether pharmacological stressor effects on risk-based decision-making involve dopamine D1 receptor stimulation in the dorsal prelimbic cortex (PL). We used a risk discounting task that requires choosing between a certain/small reward lever that always delivered 1 pellet and a risky/large reward lever that delivered 4 pellets with a decreasing probability across subsequent trials.

RESULTS

Systemic administration of yohimbine increased the preference for the risky/large reward lever. By contrast, systemic single administration of corticosterone did not significantly promote risky choice. Moreover, co-administration of corticosterone did not enhance the effects of yohimbine on risky choice. The data further show that the increased preference for the risky/large reward lever under systemic yohimbine was lowered by a concurrent pharmacological blockade of dopamine D1 receptors in the PL.

CONCLUSIONS

Our rodent data provide causal evidence that stimulation of PL D1 receptors may represent a neurochemical mechanism by which the acute pharmacological stressor yohimbine, and possibly nonpharmacological stressors as well, promote risky choice.

An opponent striatal circuit for distributional reinforcement learning

Machine learning research has achieved large performance gains on a wide range of tasks by expanding the learning target from mean rewards to entire probability distributions of rewards - an approach known as distributional reinforcement learning (RL)1. The mesolimbic dopamine system is thought to underlie RL in the mammalian brain by updating a representation of mean value in the striatum2,3, but little is known about whether, where, and how neurons in this circuit encode information about higher-order moments of reward distributions4. To fill this gap, we used high-density probes (Neuropixels) to acutely record striatal activity from well-trained, water-restricted mice performing a classical conditioning task in which reward mean, reward variance, and stimulus identity were independently manipulated. In contrast to traditional RL accounts, we found robust evidence for abstract encoding of variance in the striatum. Remarkably, chronic ablation of dopamine inputs disorganized these distributional representations in the striatum without interfering with mean value coding. Two-photon calcium imaging and optogenetics revealed that the two major classes of striatal medium spiny neurons - D1 and D2 MSNs - contributed to this code by preferentially encoding the right and left tails of the reward distribution, respectively. We synthesize these findings into a new model of the striatum and mesolimbic dopamine that harnesses the opponency between D1 and D2 MSNs5-15 to reap the computational benefits of distributional RL.

Effects of acute and chronic nicotine administration on probability discounting

Nicotine use is a continuing public health concern. Smokers are more likely to make risky or maladaptive decisions compared to nonsmokers, so the relation between nicotine and risky choice warrants further investigation. Risky choice can be operationally defined as the choice for a larger, uncertain reinforcer over a smaller, certain reinforcer and can be assessed through a probability-discounting procedure. Acute nicotine administration has been shown to alter risky choice, but because the everyday smoker uses nicotine repeatedly, more research on chronic administration is needed and would allow for assessment of tolerance or sensitization of any effects. The present study examined effects of acute and repeated nicotine administration on probability discounting. Sprague-Dawley rats were used as subjects and the probability-discounting task involved discrete-trial choices between a small, certain reinforcer and a larger, uncertain reinforcer. The probability of larger-reinforcer delivery decreased across blocks within each session. Acute nicotine (0.1-1.0 mg/kg) administration dose-dependently increased risky choice, increased lose-stay ratios (a measure of response perseveration), and decreased reinforcement frequency. Tolerance to nicotine's effects on larger-reinforcer choice was observed after repeated 1.0 mg/kg nicotine administration. The results of the present study add to the existing literature that acute nicotine administration increases risky choice and demonstrates that tolerance to this effect develops after chronic exposure to the drug. Possible behavioral mechanisms behind this effect are discussed, as are suggestions for future research on nicotine and risky choice.

The influence of insight on risky decision making and nucleus accumbens activation

During insightful problem solving, the solution appears unexpectedly and is accompanied by the feeling of an AHA!. Research suggests that this affective component of insight can have consequences beyond the solution itself by motivating future behavior, such as risky (high reward and high uncertainty) decision making. Here, we investigate the behavioral and neural support for the motivational role of AHA in decision making involving monetary choices. The positive affect of the AHA! experience has been linked to internal reward. Reward in turn has been linked to dopaminergic signal transmission in the Nucleus Accumbens (NAcc) and risky decision making. Therefore, we hypothesized that insight activates reward-related brain areas, modulating risky decision making. We tested this hypothesis in two studies. First, in a pre-registered online study (Study 1), we demonstrated the behavioral effect of insight-related increase in risky decision making using a visual Mooney identification paradigm. Participants were more likely to choose the riskier monetary payout when they had previously solved the Mooney image with high compared to low accompanied AHA!. Second, in an fMRI study (Study 2), we measured the effects of insight on NAcc activity using a similar Mooney identification paradigm to the one of Study 1. Greater NAcc activity was found when participants solved the Mooney image with high vs low AHA!. Taken together, our results link insight to enhanced NAcc activity and a preference for high but uncertain rewards, suggesting that insight enhances reward-related brain areas possibly via dopaminergic signal transmission, promoting risky decision making.

Dopamine D2 receptors in nucleus accumbens cholinergic interneurons increase impulsive choice

Impulsive choice, often characterized by excessive preference for small, short-term rewards over larger, long-term rewards, is a prominent feature of substance use and other neuropsychiatric disorders. The neural mechanisms underlying impulsive choice are not well understood, but growing evidence implicates nucleus accumbens (NAc) dopamine and its actions on dopamine D2 receptors (D2Rs). Because several NAc cell types and afferents express D2Rs, it has been difficult to determine the specific neural mechanisms linking NAc D2Rs to impulsive choice. Of these cell types, cholinergic interneurons (CINs) of the NAc, which express D2Rs, have emerged as key regulators of striatal output and local dopamine release. Despite these relevant functions, whether D2Rs expressed specifically in these neurons contribute to impulsive choice behavior is unknown. Here, we show that D2R upregulation in CINs of the mouse NAc increases impulsive choice as measured in a delay discounting task without affecting reward magnitude sensitivity or interval timing. Conversely, mice lacking D2Rs in CINs showed decreased delay discounting. Furthermore, CIN D2R manipulations did not affect probabilistic discounting, which measures a different form of impulsive choice. Together, these findings suggest that CIN D2Rs regulate impulsive decision-making involving delay costs, providing new insight into the mechanisms by which NAc dopamine influences impulsive behavior.

Circuit and Cell-Specific Contributions to Decision Making Involving Risk of Explicit Punishment in Male and Female Rats

Decision making is a complex cognitive process that recruits a distributed network of brain regions, including the basolateral amygdala (BLA) and nucleus accumbens shell (NAcSh). Recent work suggests that communication between these structures, as well as activity of cells expressing dopamine (DA) D2 receptors (D2R) in the NAcSh, are necessary for some forms of decision making; however, the contributions of this circuit and cell population during decision making under risk of punishment are unknown. The current experiments addressed this question using circuit-specific and cell type-specific optogenetic approaches in rats during a decision making task involving risk of punishment. In experiment 1, Long-Evans rats received intra-BLA injections of halorhodopsin or mCherry (control) and in experiment 2, D2-Cre transgenic rats received intra-NAcSh injections of Cre-dependent halorhodopsin or mCherry. Optic fibers were implanted in the NAcSh in both experiments. Following training in the decision making task, BLA→NAcSh or D2R-expressing neurons were optogenetically inhibited during different phases of the decision process. Inhibition of the BLA→NAcSh during deliberation (the time between trial initiation and choice) increased preference for the large, risky reward (increased risk taking). Similarly, inhibition during delivery of the large, punished reward increased risk taking, but only in males. Inhibition of D2R-expressing neurons in the NAcSh during deliberation increased risk taking. In contrast, inhibition of these neurons during delivery of the small, safe reward decreased risk taking. These findings extend our knowledge of the neural dynamics of risk taking, revealing sex-dependent circuit recruitment and dissociable activity of selective cell populations during decision making.SIGNIFICANCE STATEMENT Until recently, the ability to dissect the neural substrates of decision making involving risk of punishment (risk taking) in a circuit-specific and cell-specific manner has been limited by the tools available for use in rats. Here, we leveraged the temporal precision of optogenetics, together with transgenic rats, to probe contributions of a specific circuit and cell population to different phases of risk-based decision making. Our findings reveal basolateral amygdala (BLA)→nucleus accumbens shell (NAcSh) is involved in evaluation of punished rewards in a sex-dependent manner. Further, NAcSh D2 receptor (D2R)-expressing neurons make unique contributions to risk taking that vary across the decision making process. These findings advance our understanding of the neural principles of decision making and provide insight into how risk taking may become compromised in neuropsychiatric diseases.

Gambling on an empty stomach: Hunger modulates preferences for learned but not described risks

INTRODUCTION

We assess risks differently when they are explicitly described, compared to when we learn directly from experience, suggesting dissociable decision-making systems. Our needs, such as hunger, could globally affect our risk preferences, but do they affect described and learned risks equally? On one hand, decision-making from descriptions is often considered flexible and context sensitive, and might therefore be modulated by metabolic needs. On the other hand, preferences learned through reinforcement might be more strongly coupled to biological drives.

METHOD

Thirty-two healthy participants (females: 20, mean age: 25.6 ± 6.5 years) with a normal weight (Body Mass Index: 22.9 ± 3.2 kg/m2 ) were tested in a within-subjects counterbalanced, randomized crossover design for the effects of hunger on two separate risk-taking tasks. We asked participants to choose between two options with different risks to obtain monetary outcomes. In one task, the outcome probabilities were described numerically, whereas in a second task, they were learned.

RESULT

In agreement with previous studies, we found that rewarding contexts induced risk-aversion when risks were explicitly described (F1,31  = 55.01, p < .0001, ηp 2  = .64), but risk-seeking when they were learned through experience (F1,31  = 10.28, p < .003, ηp 2  = .25). Crucially, hunger attenuated these contextual biases, but only for learned risks (F1,31  = 8.38, p < .007, ηp 2  = .21).

CONCLUSION

The results suggest that our metabolic state determines risk-taking biases when we lack explicit descriptions.

Pharmacological Modulation of Temporal Discounting: A Systematic Review

Temporal discounting is a phenomenon where a reward loses its value as a function of time (e.g., a reward is more valuable immediately than when it delays in time). This is a type of intertemporal decision-making that has an association with impulsivity and self-control. Many pathologies exhibit higher discounting rates, meaning they discount more the values of rewards, such as addictive behaviors, bipolar disorder, attention-deficit/hyperactivity disorders, social anxiety disorders, and major depressive disorder, among others; thus, many studies look for the mechanism and neuromodulators of these decisions. This systematic review aims to investigate the association between pharmacological administration and changes in temporal discounting. A search was conducted in PubMed, Scopus, Web of Science, Science Direct and Cochrane. We used the PICO strategy: healthy humans (P-Participants) that received a pharmacological administration (I-Intervention) and the absence of a pharmacological administration or placebo (C-Comparison) to analyze the relationship between the pharmacological administration and the temporal discounting (O-outcome). Nineteen studies fulfilled the inclusion criteria. The most important findings were the involvement of dopamine modulation in a U-shape for choosing the delayed outcome (metoclopradime, haloperidol, and amisulpride). Furthermore, administration of tolcapone and high doses of d-amphetamine produced a preference for the delayed option. There was a time-dependent hydrocortisone effect in the preference for the immediate reward. Thus, it can be concluded that dopamine is a crucial modulator for temporal discounting, especially the D2 receptor, and cortisol also has an important time-dependent role in this type of decision. One of the limitations of this systematic review is the heterogeneity of the drugs used to assess the effect of temporal discounting.

Dopamine facilitates the translation of physical exertion into assessments of effort

Our assessments of effort are critically shaped by experiences of exertion. However, it is unclear how the nervous system transforms physical exertion into assessments of effort. Availability of the neuromodulator dopamine influences features of motor performance and effort-based decision-making. To test dopamine's role in the translation of effortful exertion into assessments of effort, we had participants with Parkinson's disease, in dopamine depleted (OFF dopaminergic medication) and elevated (ON dopaminergic medication) states, exert levels of physical exertion and retrospectively assess how much effort they exerted. In a dopamine-depleted state, participants exhibited increased exertion variability and over-reported their levels of exertion, compared to the dopamine-supplemented state. Increased exertion variability was associated with less accurate effort assessment and dopamine had a protective influence on this effect, reducing the extent to which exertion variability corrupted assessments of effort. Our findings provide an account of dopamine's role in the translation of features of motor performance into judgments of effort, and a potential therapeutic target for the increased sense of effort observed across a range of neurologic and psychiatric conditions.

Cognitive mechanisms underlying decision making involving risk of explicit punishment in male and female rats

Individuals engage in the process of risk-based decision making on a daily basis to navigate various aspects of life. There are, however, individual differences in this form of decision making, with some individuals exhibiting preference for riskier choices (risk taking) and others exhibiting preference for safer choices (risk aversion). Recent work has shown that extremes in risk taking (e.g., excessive risk taking or risk aversion) are not only cognitive features of neuropsychiatric diseases, but may in fact predispose individuals to the development of such diseases. To better understand individual differences in risk taking, and thus the mechanisms by which they confer disease vulnerability, the current study investigated the cognitive contributions to risk taking in both males and females. Rats were first behaviorally characterized in a decision-making task involving risk of footshock punishment and then tested on a battery of cognitive behavioral assays. Individual variability in risk taking was compared with performance on these tasks. Consistent with prior work, females were more risk averse than males. With the exception of the Set-shifting Task, there were no sex differences in performance on other cognitive assays. There were, however, sex-dependent associations between risk taking and specific cognitive measures. Greater risk taking was associated with better cognitive flexibility in males whereas greater risk aversion was associated with better working memory in females. Collectively, these findings reveal that distinct cognitive mechanisms are associated with risk taking in males and females, which may account for sex differences in this form of decision making.

The Effects of the Inhalant Toluene on Cognitive Function and Behavioral Flexibility: A Review of Recent Findings

Substance use disorder (SUD) is characterized, in part, by lack of control over drug seeking and taking. The prefrontal cortex (PFC) is highly involved in control of behavior and deficits in PFC structure and function have been demonstrated in clinical and preclinical studies of SUD. Of the various classes of drugs associated with the development of SUD, inhalants are among the least studied despite their widespread use among adolescents and children. In this work, we review what is currently known regarding the sites and mechanisms of action of inhalants with a focus on the volatile solvent toluene that is contained in a wide variety of legal and easily obtained products. We then describe how inhalants including toluene affect various behaviors with an emphasis on those associated with PFC function and how chronic use of inhalants alters brain structure and neuronal signaling. Findings from these studies highlight advances made in recent years that have expanded our understanding of the effects of inhalants on brain structure and reinforce the need for continued work in this field.

Dopamine D2 receptors in nucleus accumbens cholinergic interneurons increase impulsive choice

Impulsive choice, often characterized by excessive preference for small, short-term rewards over larger, long-term rewards, is a prominent feature of substance use and other neuropsychiatric disorders. The neural mechanisms underlying impulsive choice are not well understood, but growing evidence implicates nucleus accumbens (NAc) dopamine and its actions on dopamine D2 receptors (D2Rs). Because several NAc cell types and afferents express D2Rs, it has been difficult to determine the specific neural mechanisms linking NAc D2Rs to impulsive choice. Of these cell types, cholinergic interneurons (CINs) of the NAc, which express D2Rs, have emerged as key regulators of striatal output and local dopamine release. Despite these relevant functions, whether D2Rs expressed specifically in these neurons contribute to impulsive choice behavior is unknown. Here, we show that D2R upregulation in CINs of the mouse NAc increases impulsive choice as measured in a delay discounting task without affecting reward magnitude sensitivity or interval timing. Conversely, mice lacking D2Rs in CINs showed decreased delay discounting. Furthermore, CIN D2R manipulations did not affect probabilistic discounting, which measures a different form of impulsive choice. Together, these findings suggest that CIN D2Rs regulate impulsive decision-making involving delay costs, providing new insight into the mechanisms by which NAc dopamine influences impulsive behavior.

Circuit and cell-specific contributions to decision making involving risk of explicit punishment in male and female rats

Decision making is a complex cognitive process that recruits a distributed network of brain regions, including the basolateral amygdala (BLA) and nucleus accumbens shell (NAcSh). Recent work suggests that communication between these structures, as well as activity of cells expressing dopamine D2 receptors (D2R) in the NAcSh, are necessary for some forms of decision making; however, the contributions of this circuit and cell population during decision making under risk of punishment are unknown. The current experiments addressed this question using circuit- and cell type-specific optogenetic approaches in rats during a decision-making task involving risk of punishment. In Experiment 1, Long-Evans rats received intra-BLA injections of halorhodopsin or mCherry (control) and in Experiment 2, D2-Cre transgenic rats received intra-NAcSh injections of Cre-dependent halorhodopsin or mCherry. Optic fibers were implanted in the NAcSh in both experiments. Following training in the decision-making task, BLA→NAcSh or D2R-expressing neurons were optogenetically inhibited during different phases of the decision process. Inhibition of the BLA→NAcSh during deliberation (the time between trial initiation and choice) increased choice of the large, risky reward (increased risk taking). Similarly, inhibition during delivery of the large, punished reward increased risk taking, but only in males. Inhibition of D2R-expressing neurons in the NAcSh during deliberation increased risk taking. In contrast, inhibition of these neurons during delivery of the small, safe reward decreased risk taking. These findings extend our knowledge of the neural dynamics of risk taking, revealing sex-dependent circuit recruitment and dissociable activity of selective cell populations during decision making.

Concurrent measures of impulsive action and choice are partially related and differentially modulated by dopamine D1- and D2-like receptors in a rat model of impulsivity

Impulsivity is a multidimensional construct, but the relationships between its constructs and their respective underlying dopaminergic underpinnings in the general population remain unclear. A cohort of Roman high- (RHA) and low- (RLA) avoidance rats were tested for impulsive action and risky decision-making in the rat gambling task, and then for delay discounting in the delay-discounting task to concurrently measure the relationships among the three constructs of impulsivity using a within-subject design. Then, we evaluated the effects of dopaminergic drugs on the three constructs of impulsivity, considering innate differences in impulsive behaviors at baseline. Risky decision-making and delay-discounting were positively correlated, indicating that both constructs of impulsive choice are related. Impulsive action positively correlated with risky decision-making but not with delay discounting, suggesting partial overlap between impulsive action and impulsive choice. RHAs showed a more impulsive phenotype in the three constructs of impulsivity compared to RLAs, demonstrating the comorbid nature of impulsivity in a population of rats. Amphetamine increased impulsive action and had no effect on risky decision-making regardless of baseline levels of impulsivity, but it decreased delay discounting only in high impulsive RHAs. In contrast, while D₁R and D₃R agonism as well as D_2/3R partial agonism decreased impulsive action regardless of baseline levels of impulsivity, D_2/3R agonism decreased impulsive action exclusively in high impulsive RHAs. Irrespective of baseline levels of impulsivity, risky decision-making was increased by D₁R and D_2/3R agonism but not by D₃R agonism or D_2/3R partial agonism. Finally, while D₁R and D₃R agonism, D_2/3R partial agonism and D₂R blockade increased delay discounting irrespective of baseline levels of impulsivity, D_2/3R agonism decreased it in low impulsive RLAs only. These findings indicate that the acute effects of dopamine drugs were partially overlapping across dimensions of impulsivity, and that only D_2/3R agonism showed baseline-dependent effects on impulsive action and impulsive choice.

Vulnerability and fraud: evidence from the COVID-19 pandemic

This study examines consumer fraud at the onset of the COVID-19 pandemic and provides novel evidence for the opportunity model of predatory victimization. Scammers have taken advantage of the COVID-19 pandemic shock to exploit victims who are already vulnerable and suffering. The number of fraud cases has greatly increased as COVID-19 spread across the U.S., consistent with the vulnerable-to-become-victimization hypothesis based on the opportunity model of predatory victimization. A Google Trends analysis shows that the increase in fraud and scams is attributable to victims' increased vulnerability rather than to their awareness of fraud and increased motivation to report scams. An improvement in financial literacy is associated with the reduction of finance-related fraud and scams. Finally, we provide important policy implications to protect people from fraud victimization.

The neural substrates of risky rewards and losses in healthy volunteers and patient groups: a PET imaging study

BACKGROUND

Risk is an essential trait of most daily decisions. Our behaviour when faced with risks involves evaluation of many factors including the outcome probabilities, the valence (gains or losses) and past experiences. Several psychiatric disorders belonging to distinct diagnostic categories, including pathological gambling and addiction, show pathological risk-taking and implicate abnormal dopaminergic, opioidergic and serotonergic neurotransmission. In this study, we adopted a transdiagnostic approach to delineate the neurochemical substrates of decision making under risk.

METHODS

We recruited 39 participants, including 17 healthy controls, 15 patients with pathological gambling and seven binge eating disorder patients, who completed an anticipatory risk-taking task. Separately, participants underwent positron emission tomography (PET) imaging with three ligands, [¹⁸F]fluorodopa (FDOPA), [¹¹C]MADAM and [¹¹C]carfentanil to assess presynaptic dopamine synthesis capacity and serotonin transporter and mu-opioid receptor binding respectively.

RESULTS

Risk-taking behaviour when faced with gains positively correlated with dorsal cingulate [¹¹C]carfentanil binding and risk-taking to losses positively correlated with [¹¹C]MADAM binding in the caudate and putamen across all subjects.

CONCLUSIONS

We show distinct neurochemical substrates underlying risk-taking with the dorsal cingulate cortex mu-opioid receptor binding associated with rewards and dorsal striatal serotonin transporter binding associated with losses. Risk-taking and goal-directed control appear to dissociate between dorsal and ventral fronto-striatal systems. Our findings thus highlight the potential role of pharmacological agents or neuromodulation on modifying valence-specific risk-taking biases.

Effort-based decision making in response to high-dose androgens: role of dopamine receptors

INTRODUCTION

Anabolic-androgenic steroids (AAS) are performance-enhancing drugs used by both world-class and rank-and-file athletes. AAS abuse has been linked with risky decision-making, ranging from drunk driving to abusing multiple drugs. Our lab uses operant behavior in rats to test the effects of AAS (testosterone) on decision making. In our previous study, testosterone caused rats to work harder for food reward during an effort discounting (ED) task. ED is sensitive to dopamine in the nucleus accumbens, and AAS alter accumbens dopamine receptor expression. Accordingly, we determined if testosterone increases response to dopamine receptor antagonists during ED.

METHODS

Rats were treated chronically with high-dose testosterone (7.5 mg/kg; n = 9) or vehicle (n = 9). We measured baseline preference for the large reward in an ED task, where rats choose between a small easy reward (one lever press for one sugar pellet) and a large difficult reward (2, 5, 10, or 15 presses for three pellets). Preference for the large reward was measured after administration of D1-like (SCH23390, 0.01 mg/kg) or D2-like (eticlopride, 0.06 mg/kg) receptor antagonists.

RESULTS

At baseline, testosterone- and vehicle-treated rats showed similar preference for the large reward lever (FR5, testosterone: 68.6 ± 9.7% and vehicle: 85.7 ± 2.5%). SCH23390 reduced large reward preference significantly in both groups (FR5, testosterone: 41.3 ± 9.2%; vehicle: 49.1 ± 8.2%; F(1,16) = 17.7; P < 0.05). Eticlopride decreased large reward preference in both groups, but more strongly in testosterone-treated rats (FR5: testosterone: 37.0 ± 9.7%; vehicle: 56.3 ± 7.8%; F(1,16) = 35.3; P < 0.05).

CONCLUSION

Testosterone increases response to dopamine D2-like receptor blockade, and this contributes to previously observed changes in decision-making behaviors.

Etiopathogenic Models of Psychosis Spectrum Illnesses Must Resolve Four Key Features

Etiopathogenic models for psychosis spectrum illnesses are converging on a number of key processes, such as the influence of specific genes on the synthesis of proteins important in synaptic functioning, alterations in how neurons respond to synaptic inputs and engage in synaptic pruning, and microcircuit dysfunction that leads to more global cortical information processing vulnerabilities. Disruptions in prefrontal operations then accumulate and propagate over time, interacting with environmental factors, developmental processes, and homeostatic mechanisms, eventually resulting in symptoms of psychosis and disability. However, there are 4 key features of psychosis spectrum illnesses that are of primary clinical relevance but have been difficult to assimilate into a single model and have thus far received little direct attention: 1) the bidirectionality of the causal influences for the emergence of psychosis, 2) the catastrophic clinical threshold seen in first episodes of psychosis and why it is irreversible in some individuals, 3) observed biotypes that are neurophysiologically distinct but clinically both convergent and divergent, and 4) a reconciliation of the role of striatal dopaminergic dysfunction with models of prefrontal cortical state instability. In this selective review, we briefly describe these 4 hallmark features and we argue that theoretically driven computational perspectives making use of both algorithmic and neurophysiologic models are needed to reduce this complexity and variability of psychosis spectrum illnesses in a principled manner.

Sequential delay and probability discounting tasks in mice reveal anchoring effects partially attributable to decision noise

Delay discounting and probability discounting decision making tasks in rodent models have high translational potential. However, it is unclear whether the discounted value of the large reward option is the main contributor to variability in animals' choices in either task, which may limit translation to humans. Male and female mice underwent sessions of delay and probability discounting in sequence to assess how choice behavior adapts over experience with each task. To control for "anchoring" (persistent choices based on the initial delay or probability), mice experienced "Worsening" schedules where the large reward was offered under initially favorable conditions that became less favorable during testing, followed by "Improving" schedules where the large reward was offered under initially unfavorable conditions that improved over a session. During delay discounting, both male and female mice showed elimination of anchoring effects over training. In probability discounting, both sexes of mice continued to show some anchoring even after months of training. One possibility is that "noisy", exploratory choices could contribute to these persistent anchoring effects, rather than constant fluctuations in value discounting. We fit choice behavior in individual animals using models that included both a value-based discounting parameter and a decision noise parameter that captured variability in choices deviating from value maximization. Changes in anchoring behavior over time were tracked by changes in both the value and decision noise parameters in delay discounting, but by the decision noise parameter in probability discounting. Exploratory decision making was also reflected in choice response times that tracked the degree of conflict caused by both uncertainty and temporal cost, but was not linked with differences in locomotor activity reflecting chamber exploration. Thus, variable discounting behavior in mice can result from changes in exploration of the decision options rather than changes in reward valuation.

Different brain systems support learning from received and avoided pain during human pain-avoidance learning

Both unexpected pain and unexpected pain absence can drive avoidance learning, but whether they do so via shared or separate neural and neurochemical systems is largely unknown. To address this issue, we combined an instrumental pain-avoidance learning task with computational modeling, functional magnetic resonance imaging (fMRI), and pharmacological manipulations of the dopaminergic (100 mg levodopa) and opioidergic (50 mg naltrexone) systems (N = 83). Computational modeling provided evidence that untreated participants learned more from received than avoided pain. Our dopamine and opioid manipulations negated this learning asymmetry by selectively increasing learning rates for avoided pain. Furthermore, our fMRI analyses revealed that pain prediction errors were encoded in subcortical and limbic brain regions, whereas no-pain prediction errors were encoded in frontal and parietal cortical regions. However, we found no effects of our pharmacological manipulations on the neural encoding of prediction errors. Together, our results suggest that human pain-avoidance learning is supported by separate threat- and safety-learning systems, and that dopamine and endogenous opioids specifically regulate learning from successfully avoided pain.

Uncertainty-guided learning with scaled prediction errors in the basal ganglia

To accurately predict rewards associated with states or actions, the variability of observations has to be taken into account. In particular, when the observations are noisy, the individual rewards should have less influence on tracking of average reward, and the estimate of the mean reward should be updated to a smaller extent after each observation. However, it is not known how the magnitude of the observation noise might be tracked and used to control prediction updates in the brain reward system. Here, we introduce a new model that uses simple, tractable learning rules that track the mean and standard deviation of reward, and leverages prediction errors scaled by uncertainty as the central feedback signal. We show that the new model has an advantage over conventional reinforcement learning models in a value tracking task, and approaches a theoretic limit of performance provided by the Kalman filter. Further, we propose a possible biological implementation of the model in the basal ganglia circuit. In the proposed network, dopaminergic neurons encode reward prediction errors scaled by standard deviation of rewards. We show that such scaling may arise if the striatal neurons learn the standard deviation of rewards and modulate the activity of dopaminergic neurons. The model is consistent with experimental findings concerning dopamine prediction error scaling relative to reward magnitude, and with many features of striatal plasticity. Our results span across the levels of implementation, algorithm, and computation, and might have important implications for understanding the dopaminergic prediction error signal and its relation to adaptive and effective learning.

Cannabinoid receptor type 1 antagonists alter aspects of risk/reward decision making independent of toluene-mediated effects

Drugs of abuse including cannabis and inhalants impair risk/reward decision making. Cannabis use is often concurrent with inhalant intoxication; yet, preclinical studies investigating the role of endocannabinoids in inhalant misuse are limited. To address this gap in the literature, we used the well-validated probabilistic discounting task to assess risk/reward decision making in rodents following combinations of toluene vapor (a common inhalant) and manipulations of cannabinoid receptor type 1 (CB1R) signaling. As reported previously, acute exposure to toluene vapor disrupted behavioral flexibility during probabilistic discounting. Systemic administration of the CB1R inverse agonist AM281 did not prevent toluene-induced alterations in risky choices, but did independently reduce win-stay behavior, increase choice latency, and increase omissions. Toluene-induced deficits in probabilistic discounting are thought to involve impaired medial prefrontal cortex (mPFC) activity. As we previously reported that some of toluene's inhibitory effects on glutamatergic signaling in the mPFC are endocannabinoid-dependent, we tested the hypothesis that mPFC CB1R activity mediates toluene-induced deficits in discounting. However, bilateral injection of the CB1R inverse agonist AM251 prior to toluene vapor exposure had no effect on toluene-induced changes in risk behavior. In a final set of experiments, we injected the CB1R inverse agonist AM251 (5 and 50 ng), the CB1R agonist WIN55,212-2 (50 ng and 500 ng), or vehicle into the mPFC prior to testing. While mPFC CB1R stimulation did not affect any of the measures tested, the CB1R inverse agonist caused a dose-dependent reduction in win-stay behavior without altering any other measures. Together, these studies indicate that toluene-induced deficits in probabilistic discounting are largely distinct from CB1R-dependent effects that include decreased effectiveness of positive reinforcement (mPFC CB1Rs), decision making speed, and task engagement (non-mPFC CB1Rs).

Role of dopamine and clinical heterogeneity in cognitive dysfunction in Parkinson's disease

Parkinson's disease (PD) is commonly treated with dopaminergic medication, which enhances some, while impairing other cognitive functions. It can even contribute to impulse control disorder and addiction. We describe the history of research supporting the dopamine overdose hypothesis, which accounts for the large within-patient variability in dopaminergic medication effects across different tasks by referring to the spatially non-uniform pattern of dopamine depletion in dorsal versus ventral striatum. However, there is tremendous variability in dopaminergic medication effects not just within patients across distinct tasks, but also across different patients. In the second part of this chapter we review recent studies addressing the large individual variability in the negative side effects of dopaminergic medication on functions that implicate dopamine, such as value-based learning and choice. These studies begin to unravel the mechanisms of dopamine overdosing, thus revising the strict version of the overdose hypothesis. For example, the work shows that the canonical boosting of reward-versus punishment-based choice by medication is greater in patients with depression and a non-tremor phenotype, which both implicate, among other pathology, more rather than less severe dysregulation of the mesolimbic dopamine system. Future longitudinal cohort studies are needed to identify how to optimally combine different clinical, personality, cognitive, neural, genetic and molecular predictors of detrimental medication effects in order to account for as much of the relevant variability as possible. This will provide a useful tool for precision neurology, allowing individual and contextual tailoring of (the dose of) dopaminergic medication in order to maximize its cognitive benefits, yet minimize its side effects.

Social Determinants of Inter-Individual Variability and Vulnerability: The Role of Dopamine

Individuals differ in their traits and preferences, which shape their interactions, their prospects for survival and their susceptibility to diseases. These correlations are well documented, yet the neurophysiological mechanisms underlying the emergence of distinct personalities and their relation to vulnerability to diseases are poorly understood. Social ties, in particular, are thought to be major modulators of personality traits and psychiatric vulnerability, yet the majority of neuroscience studies are performed on rodents in socially impoverished conditions. Rodent micro-society paradigms are therefore key experimental paradigms to understand how social life generates diversity by shaping individual traits. Dopamine circuitry is implicated at the interface between social life experiences, the expression of essential traits, and the emergence of pathologies, thus proving a possible mechanism to link these three concepts at a neuromodulatory level. Evaluating inter-individual variability in automated social testing environments shows great promise for improving our understanding of the link between social life, personality, and precision psychiatry – as well as elucidating the underlying neurophysiological mechanisms.

Impulsivity and risk-seeking as Bayesian inference under dopaminergic control

Bayesian models successfully account for several of dopamine (DA)'s effects on contextual calibration in interval timing and reward estimation. In these models, tonic levels of DA control the precision of stimulus encoding, which is weighed against contextual information when making decisions. When DA levels are high, the animal relies more heavily on the (highly precise) stimulus encoding, whereas when DA levels are low, the context affects decisions more strongly. Here, we extend this idea to intertemporal choice and probability discounting tasks. In intertemporal choice tasks, agents must choose between a small reward delivered soon and a large reward delivered later, whereas in probability discounting tasks, agents must choose between a small reward that is always delivered and a large reward that may be omitted with some probability. Beginning with the principle that animals will seek to maximize their reward rates, we show that the Bayesian model predicts a number of curious empirical findings in both tasks. First, the model predicts that higher DA levels should normally promote selection of the larger/later option, which is often taken to imply that DA decreases 'impulsivity,' and promote selection of the large/risky option, often taken to imply that DA increases 'risk-seeking.' However, if the temporal precision is sufficiently decreased, higher DA levels should have the opposite effect-promoting selection of the smaller/sooner option (higher impulsivity) and the small/safe option (lower risk-seeking). Second, high enough levels of DA can result in preference reversals. Third, selectively decreasing the temporal precision, without manipulating DA, should promote selection of the larger/later and large/risky options. Fourth, when a different post-reward delay is associated with each option, animals will not learn the option-delay contingencies, but this learning can be salvaged when the post-reward delays are made more salient. Finally, the Bayesian model predicts correlations among behavioral phenotypes: Animals that are better timers will also appear less impulsive.

Neural correlates and determinants of approach-avoidance conflict in the prelimbic prefrontal cortex

The recollection of environmental cues associated with threat or reward allows animals to select the most appropriate behavioral responses. Neurons in the prelimbic (PL) cortex respond to both threat- and reward-associated cues. However, it remains unknown whether PL regulates threat-avoidance vs. reward-approaching responses when an animals’ decision depends on previously associated memories. Using a conflict model in which male Long–Evans rats retrieve memories of shock- and food-paired cues, we observed two distinct phenotypes during conflict: (1) rats that continued to press a lever for food (Pressers) and (2) rats that exhibited a complete suppression in food seeking (Non-pressers). Single-unit recordings revealed that increased risk-taking behavior in Pressers is associated with persistent food-cue responses in PL, and reduced spontaneous activity in PL glutamatergic (PL^GLUT) neurons during conflict. Activating PL^GLUT neurons in Pressers attenuated food-seeking responses in a neutral context, whereas inhibiting PL^GLUT neurons in Non-pressers reduced defensive responses and increased food approaching during conflict. Our results establish a causal role for PL^GLUT neurons in mediating individual variability in memory-based risky decision-making by regulating threat-avoidance vs. reward-approach behaviors.

The association between risky decision making and cocaine conditioned place preference is moderated by sex

BACKGROUND

Excessive risk taking is a characteristic trait of several psychiatric conditions, including substance use disorders. High risk-taking (HiR) rats self-administer more cocaine compared to low risk-taking (LoR) rats. However, research has not determined if risk taking is associated with enhanced cocaine conditioned place preference (CPP).

METHODS

Male and female Sprague Dawley rats (n = 48 each sex) were first tested in the risky decision task (RDT), in which a response on one lever resulted in safe delivery of one food pellet, and a response on a different lever resulted in delivery of two pellets and probabilistic delivery of foot shock. Following RDT training, rats were tested for cocaine CPP. The first session was a pretest that measured rats' preference for three compartments that provided different visual and tactile cues. Rats then learned to associate one compartment with cocaine (either 10.0 mg/kg or 20.0 mg/kg; i.p.) and one compartment with saline (1.0 ml/kg; i.p.) across eight conditioning sessions. Finally, rats explored all three compartments in a drug-free state.

RESULTS

Sex significantly moderated the association between risky decision making and cocaine CPP. While increased risk aversion was somewhat positively associated with cocaine CPP in males, increased risk taking was positively correlated with cocaine CPP in females.

CONCLUSIONS

These results highlight the moderating role of sex on the relationship between risky decision making and cocaine reward.

Noradrenergic contributions to cue-driven risk-taking and impulsivity

RATIONALE

The flashing lights and sounds of modern casinos are alluring and may contribute to the addictive nature of gambling. Such cues can have a profound impact on the noradrenaline (NA) system, which could therefore be a viable therapeutic target for gambling disorder (GD). While there is substantial evidence to support the involvement of NA in the impulsive symptoms of GD, its function in mediating the "pro-addictive" impact of cues is less understood.

OBJECTIVE

We wished to investigate the role of NA in our rodent assay of decision making and impulsivity, the cued rat gambling task (crGT). Given that sex differences are prominent in addiction disorders, and increasingly reported in the monoaminergic regulation of behaviour, we also prioritised evaluating noradrenergic drugs in both sexes.

METHODS

Female and male rats were trained to stability on the crGT and then given intraperitoneal injections of the noradrenaline reuptake inhibitor atomoxetine, the α_2A receptor agonist guanfacine, the beta receptor antagonist propranolol, and the α₂ receptor antagonist yohimbine.

RESULTS

Atomoxetine dose-dependently improved decision-making score. Guanfacine selectively enhanced decision making in risk-preferring males and optimal performing females. Propranolol and yohimbine did not influence decision making. Atomoxetine and guanfacine reduced premature responses, while yohimbine bi-phasically affected this index of motor impulsivity.

CONCLUSIONS

These results support the hypothesis that NA is an important neuromodulator of the cue-induced deficits in decision making observed in laboratory-based gambling paradigms, and suggest that NAergic drugs like atomoxetine and guanfacine may be useful in treating GD.

An association between prediction errors and risk-seeking: Theory and behavioral evidence

Reward prediction errors (RPEs) and risk preferences have two things in common: both can shape decision making behavior, and both are commonly associated with dopamine. RPEs drive value learning and are thought to be represented in the phasic release of striatal dopamine. Risk preferences bias choices towards or away from uncertainty; they can be manipulated with drugs that target the dopaminergic system. Based on the common neural substrate, we hypothesize that RPEs and risk preferences are linked on the level of behavior as well. Here, we develop this hypothesis theoretically and test it empirically. First, we apply a recent theory of learning in the basal ganglia to predict how RPEs influence risk preferences. We find that positive RPEs should cause increased risk-seeking, while negative RPEs should cause risk-aversion. We then test our behavioral predictions using a novel bandit task in which value and risk vary independently across options. Critically, conditions are included where options vary in risk but are matched for value. We find that our prediction was correct: participants become more risk-seeking if choices are preceded by positive RPEs, and more risk-averse if choices are preceded by negative RPEs. These findings cannot be explained by other known effects, such as nonlinear utility curves or dynamic learning rates.

Altered risky decision making in patients with early non-affective psychosis

Abnormal risky decision making may represent an important factor contributing to functional impairment in psychotic disorders. Previous research revealed impaired decision making under risk in patients with chronic schizophrenia. However, risky decision making is under-studied in the early course of illness. We examined risky decision making in 33 patients with early non-affective psychosis and 32 demographically matched controls, using two well-validated experimental paradigms, balloon analogue risk task (BART) and Risky-Gains task (RGT), which modeled and assessed actual risk-taking behaviors in deliberative and time-pressured decision-making situations, respectively. Our results showed that patients exhibited suboptimal decision making on the BART and were more risk averse than controls by having fewer average balloon pumps in non-burst trials, lower explosion rate and lower total points gained. On the RGT, patients also behaved more conservatively than controls, with lower overall rate in choosing the risky option. Intriguingly, patients performed comparably to controls in adjusting risk-taking pattern following punished trials, suggesting relatively preserved sensitivity to punishment in early psychosis. Risk-taking measures showed no significant correlations with any symptom dimensions, impulsivity traits, cognitive functions or antipsychotic treatment after correcting for multiple comparisons. This study is the first to investigate risk-taking propensity in early psychosis based on BART/RGT performance, and consistently indicate that patients with early psychosis displayed altered risky decision making with increased risk aversion relative to healthy participants. Further investigation is warranted to clarify the longitudinal course of aberrant risky decision making and its relationship with functional outcome in early psychosis.

Medial orbitofrontal cortex dopamine D1/D2 receptors differentially modulate distinct forms of probabilistic decision-making

Efficient decision-making involves weighing the costs and benefits associated with different actions and outcomes to maximize long-term utility. The medial orbitofrontal cortex (mOFC) has been implicated in guiding choice in situations involving reward uncertainty, as inactivation in rats alters choice involving probabilistic rewards. The mOFC receives considerable dopaminergic input, yet how dopamine (DA) modulates mOFC function has been virtually unexplored. Here, we assessed how mOFC D₁ and D₂ receptors modulate two forms of reward seeking mediated by this region, probabilistic reversal learning and probabilistic discounting. Separate groups of well-trained rats received intra-mOFC microinfusions of selective D₁ or D₂ antagonists or agonists prior to task performance. mOFC D₁ and D₂ blockade had opposing effects on performance during probabilistic reversal learning and probabilistic discounting. D₁ blockade impaired, while D₂ blockade increased the number of reversals completed, both mediated by changes in errors and negative feedback sensitivity apparent during the initial discrimination of the task, which suggests changes in probabilistic reinforcement learning rather than flexibility. Similarly, D₁ blockade reduced, while D₂ blockade increased preference for larger/risky rewards. Excess D₁ stimulation had no effect on either task, while excessive D₂ stimulation impaired probabilistic reversal performance, and reduced both profitable risky choice and overall task engagement. These findings highlight a previously uncharacterized role for mOFC DA, showing that D₁ and D₂ receptors play dissociable and opposing roles in different forms of reward-related action selection. Elucidating how DA biases behavior in these situations will expand our understanding of the mechanisms regulating optimal and aberrant decision-making.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.

Dopamine receptors regulate preference between high-effort and high-risk rewards

RATIONALE

Optimal decision-making necessitates evaluation of multiple rewards that are each offset by distinct costs, such as high effort requirement or high risk of failure. The neurotransmitter dopamine is fundamental toward these cost-benefit analyses, and D1-like and D2-like dopamine receptors differently modulate the reward-discounting effects of both effort and risk. However, measuring the role of dopamine in regulating decision-making between options associated with distinct costs exceeds the scope of traditional rodent economic decision-making paradigms.

OBJECTIVES

We developed the effort vs probability economic conflict task (EvP) to model multimodal economic decision-making in rats. This task measures choice between two rewards of uniform magnitude associated with either a high effort requirement or risk of reward omission. We then tested the modulatory effects of systemic cocaine and D1/D2 blockade or activation on the preference between high-effort and high-risk alternatives.

METHODS

In the EvP, two reinforcers of equal magnitude are associated with either (1) an effort requirement that increases throughout the session (1, 5, 10, and 20 lever presses), or (2) a low probability of reward receipt (25% of probabilistic choices). Critically, the reinforcer for each choice is comparable (one pellet), which eliminates the influence of magnitude discrimination on the decision-making process. After establishing the task, the dopamine transporter blocker cocaine and D1/D2 antagonists and agonists were administered prior to EvP performance.

RESULTS

Preference shifted away from either effortful or probabilistic choice when either option became more costly, and this preference was highly variable between subjects and stable over time. Cocaine, D1 activation, and D2 blockade produced limited, dose-dependent shifts in choice preference contingent on high or low effort conditions. In contrast, D2 activation across multiple doses evoked a robust shift from effortful to risky choice that was evident even when clearly disadvantageous.

CONCLUSIONS

The EvP clearly demonstrates that rats can evaluate distinct effortful or risky costs associated with rewards of comparable magnitude, and shift preference away from either option with increasing cost. This preference is more tightly linked to D2 than D1 receptor manipulation, suggesting D2-like receptors as a possible therapeutic target for maladaptive biases toward risk-taking over effort.

Differential effects of d- and l-enantiomers of govadine on distinct forms of cognitive flexibility and a comparison with dopaminergic drugs

RATIONALE

There is an urgent need for novel drugs for treating cognitive deficits that are defining features of schizophrenia. The individual d- and l-enantiomers of the tetrahydroprotoberberine (THPB) d,l-govadine have been proposed for the treatment of cognitive deficiencies and positive symptoms of schizophrenia, respectively.

OBJECTIVES

We examined the effects of d-, l-, or d,l-govadine on two distinct forms of cognitive flexibility perturbed in schizophrenia and compared them to those induced by a selective D1 receptor agonist and D2 receptor antagonist.

METHODS

Male rats received d-, l-, or d,l-govadine (0.3, 0.5, and 1.0 mg/kg), D1 agonist SKF81297(0.1, 0.3, and 1.0 mg/kg), or D2 antagonist haloperidol (0.1-0.2 mg/kg). Experiment 1 used a strategy set-shifting task (between-subjects). In experiment 2, well-trained rats were tested on a probabilistic reversal task (within-subjects).

RESULTS

d-Govadine improved set-shifting across all doses, whereas higher doses of l-govadine impaired set-shifting. SKF81297 reduced perseverative errors at the lowest dose. Low/high doses of haloperidol increased/decreased set-shifting errors, the latter "improvement" attributable to impaired retrieval of a previous acquired rule. Probabilistic reversal performance was less affected by these drugs, but d-govadine reduced errors during the first reversal, whereas l-govadine impaired initial discrimination learning. d,l-Govadine had no reliable cognitive effects but caused psychomotor slowing like l-govadine and haloperidol.

CONCLUSIONS

These findings further highlight differences between two enantiomers of d,l-govadine that may reflect differential modulation of D1 and D2 receptors. These preclinical findings give further impetus to formal clinical evaluation of d-govadine as a treatment for cognitive deficiencies related to schizophrenia.

Dopaminergic modulation of reward discounting in healthy rats: a systematic review and meta-analysis

RATIONALE

Although numerous studies have suggested that pharmacological alteration of the dopamine (DA) system modulates reward discounting, these studies have produced inconsistent findings.

OBJECTIVES

Here, we conducted a systematic review and pre-registered meta-analysis to evaluate DA drug-mediated effects on reward discounting of time, probability, and effort costs in studies of healthy rats. This produced a total of 1343 articles to screen for inclusion/exclusion. From the literature, we identified 117 effects from approximately 1549 individual rats.

METHODS

Using random effects with maximum-likelihood estimation, we meta-analyzed placebo-controlled drug effects for (1) DA D1-like receptor agonists and (2) antagonists, (3) D2-like agonists and (4) antagonists, and (5) DA transporter-modulating drugs.

RESULTS

Meta-analytic effects showed that DAT-modulating drugs decreased reward discounting. While D1-like and D2-like antagonists both increased discounting, agonist drugs for those receptors had no significant effect on discounting behavior. A number of these effects appear contingent on study design features like cost type, rat strain, and microinfusion location.

CONCLUSIONS

These findings suggest a nuanced relationship between DA and discounting behavior and urge caution when drawing generalizations about the effects of pharmacologically manipulating dopamine on reward-based decision-making.

Regulation of risky decision making by gonadal hormones in males and females

Psychiatric diseases characterized by dysregulated risky decision making are differentially represented in males and females. The factors that govern such sex differences, however, remain poorly understood. Using a task in which rats make discrete trial choices between a small, "safe" food reward and a large food reward accompanied by varying probabilities of footshock punishment, we recently showed that females are more risk averse than males. The objective of the current experiments was to test the extent to which these sex differences in risky decision making are mediated by gonadal hormones. Male and female rats were trained in the risky decision-making task, followed by ovariectomy (OVX), orchiectomy (ORX), or sham surgery. Rats were then retested in the task, under both baseline conditions and following administration of estradiol and/or testosterone. OVX increased choice of the large, risky reward (increased risky choice), an effect that was attenuated by estradiol administration. In contrast, ORX decreased risky choice, but testosterone administration was without effect in either ORX or sham males. Estradiol, however, decreased risky choice in both groups of males. Importantly, none of the effects of hormonal manipulation on risky choice were due to altered shock sensitivity or food motivation. These data show that gonadal hormones are required for maintaining sex-typical profiles of risk-taking behavior in both males and females, and that estradiol is sufficient to promote risk aversion in both sexes. The findings provide novel information about the mechanisms supporting sex differences in risk taking and may prove useful in understanding sex differences in the prevalence of psychiatric diseases associated with altered risk taking.

Differential effects of glutamate N-methyl-D-aspartate receptor antagonists on risky choice as assessed in the risky decision task

RATIONALE

Risky choice can be measured using the risky decision task (RDT). In the RDT, animals choose between a large, risky option that is paired with probabilistic foot shock and a small, safe option that is never paired with shock. To date, studies examining the neurochemical basis of decision-making in the RDT have focused primarily on the dopaminergic system but have not focused on the glutamatergic system, which has been implicated in risky decision-making.

OBJECTIVES

Because glutamate is known to play a critical role in decision-making, we wanted to determine the contribution of the glutamatergic system to performance in the RDT.

METHODS

In the experiment, 32 rats (16 male; 16 female) were tested in the RDT. The probability of receiving a foot shock increased across the session (ascending schedule) for half of the rats but decreased across the session (descending schedule) for half of the rats. Following training, rats received injections of the N-methyl-D-aspartate (NMDA) receptor competitive antagonist CGS 19755 (0, 1.0, 2.5, 5.0 mg/kg; s.c.) and the GluN2B-selective antagonist Ro 63-1908 (0, 0.1, 0.3, 1.0 mg/kg; s.c.).

RESULTS

CGS 19755 (2.5 and 5.0 mg/kg) increased risky choice in males and females trained on the ascending schedule. Ro 63-1908 (1.0 mg/kg) decreased risky choice, but only in male rats trained on the ascending schedule.

CONCLUSIONS

Although NMDA receptor antagonists differentially alter risky choice in the RDT, the current results show that NMDA receptors are an important mediator of decision-making involving probabilistic delivery of positive punishment.

Impact of past experiences on decision-making in autism spectrum disorder

People are often influenced by past costs in their current decision-making, thus succumbing to a well-known bias recognized as the sunk cost effect. A recent study showed that the sunk cost effect is attenuated in individuals with autism spectrum disorder (ASD). However, the study only addressed one situation of utilization decision by focusing on the choice between similar attractive alternatives with different levels of sunk costs. Thus, it remains unclear how individuals with ASD behave under sunk costs in different types of decision situations, particularly progress decisions, in which the decision-maker allocates additional resources to an initially chosen alternative. The sunk cost effect in progress decisions was estimated using an economic task designed to assess the effect of the past investments on current decision-making. Twenty-four individuals with ASD and 21 age-, sex-, smoking status-, education-, and intelligence quotient-level-matched typical development (TD) subjects were evaluated. The TD participants were more willing to make the second incremental investment if a previous investment was made, indicating that their decisions were influenced by sunk costs. However, unlike the TD group, the rates of investments were not significantly increased after prior investments in the ASD group. The results agree with the previous evidence of a reduced sensitivity to context stimuli in individuals with ASD and help us obtain a broader picture of the impact of sunk costs on their decision-making. Our findings will contribute to a better understanding of ASD and may be useful in addressing practical implications of their socioeconomic behavior.

Distributional Reinforcement Learning in the Brain

Learning about rewards and punishments is critical for survival. Classical studies have demonstrated an impressive correspondence between the firing of dopamine neurons in the mammalian midbrain and the reward prediction errors of reinforcement learning algorithms, which express the difference between actual reward and predicted mean reward. However, it may be advantageous to learn not only the mean but also the complete distribution of potential rewards. Recent advances in machine learning have revealed a biologically plausible set of algorithms for reconstructing this reward distribution from experience. Here, we review the mathematical foundations of these algorithms as well as initial evidence for their neurobiological implementation. We conclude by highlighting outstanding questions regarding the circuit computation and behavioral readout of these distributional codes.

The role of opioidergic system in modulating cost/benefit decision-making in alcohol-preferring AA rats and Wistar rats

Research has highlighted the association of a positive family history of alcoholism with a positive treatment response to opioid antagonists in those with a gambling disorder. However, the role of the opioidergic system in gambling behavior is not well understood, and preclinical studies are needed to clarify this. In this study, Alko Alcohol (AA) and Wistar rats went through operant lever pressing training where the task was to choose the more profitable of two options. Different sized sucrose rewards guided the lever choices, and the probability of gaining rewards changed slowly to a level where choosing the smaller reward was the most profitable option. After training, rats were administered subcutaneously with opioid agonist morphine or opioid antagonist naltrexone to study the impact of opioidergic mechanisms on cost/benefit decisions. No difference was found in the decision-making between AA rats or Wistar rats after the morphine administration, but control data revealed a minor decision enhancing effect in AA rats. Naltrexone had no impact on the decisions in AA rats but promoted unprofitable decisions in Wistar rats. Supporting behavioral data showed that in both rat strains morphine increased, and naltrexone decreased, sucrose consumption. Naltrexone also increased the time to accomplish the operant task. The results suggest that opioid agonists could improve decision-making in cost-benefit settings in rats that are naturally prone to high alcohol drinking. The naltrexone results are ambiguous but may partly explain why opioid antagonists lack a positive pharmacotherapeutic effect in some subgroups of gamblers.