Receptor-specific modulation of risk-based decision making by nucleus accumbens dopamine

Adolescent THC impacts on mPFC dopamine-mediated cognitive processes in male and female rats

RATIONALE

Adolescent cannabis use is linked to later-life changes in cognition, learning, and memory. Rodent experimental studies suggest Δ9-tetrahydrocannabinol (THC) influences development of circuits underlying these processes, especially in the prefrontal cortex, which matures during adolescence.

OBJECTIVE

We determined how 14 daily THC injections (5 mg/kg) during adolescence persistently impacts medial prefrontal cortex (mPFC) dopamine-dependent cognition.

METHODS

In adult Long Evans rats treated as adolescents with THC (AdoTHC), we quantify performance on two mPFC dopamine-dependent reward-based tasks-strategy set shifting and probabilistic discounting. We also determined how acute dopamine augmentation with amphetamine (0, 0.25, 0.5 mg/kg), or specific chemogenetic stimulation of ventral tegmental area (VTA) dopamine neurons and their projections to mPFC impact probabilistic discounting.

RESULTS

AdoTHC sex-dependently impacts acquisition of cue-guided instrumental reward seeking, but has minimal effects on set-shifting or probabilistic discounting in either sex. When we challenged dopamine circuits acutely with amphetamine during probabilistic discounting, we found reduced discounting of improbable reward options, with AdoTHC rats being more sensitive to these effects than controls. In contrast, neither acute chemogenetic stimulation of VTA dopamine neurons nor pathway-specific chemogenetic stimulation of their projection to mPFC impacted probabilistic discounting in control rats, although stimulation of this cortical dopamine projection slightly disrupted choices in AdoTHC rats.

CONCLUSIONS

These studies confirm a marked specificity in the cognitive processes impacted by AdoTHC exposure. They also suggest that some persistent AdoTHC effects may alter amphetamine-induced cognitive changes in a manner independent of VTA dopamine neurons or their projections to mPFC.

Adolescent THC impacts on mPFC dopamine-mediated cognitive processes in male and female rats

Rationale

Adolescent cannabis use is linked to later-life changes in cognition, learning, and memory. Rodent experimental studies suggest Δ9-tetrahydrocannabinol (THC) influences development of circuits underlying these processes, especially in the prefrontal cortex, which matures during adolescence.

Objective

We determined how 14 daily THC injections (5mg/kg) during adolescence persistently impacts medial prefrontal cortex (mPFC) dopamine-dependent cognition.

Methods

In adult Long Evans rats treated as adolescents with THC (AdoTHC), we quantify performance on two mPFC dopamine-dependent reward-based tasks-strategy set shifting and probabilistic discounting. We also determined how acute dopamine augmentation with amphetamine (0, 0.25, 0.5 mg/kg), or specific chemogenetic stimulation of ventral tegmental area (VTA) dopamine neurons and their projections to mPFC impacts probabilistic discounting.

Results

AdoTHC sex-dependently impacts acquisition of cue-guided instrumental reward seeking, but has minimal effects on set-shifting or probabilistic discounting in either sex. When we challenged dopamine circuits acutely with amphetamine during probabilistic discounting, we found reduced discounting of improbable reward options, with AdoTHC rats being more sensitive to these effects than controls. In contrast, neither acute chemogenetic stimulation of VTA dopamine neurons nor pathway-specific chemogenetic stimulation of their projection to mPFC impacted probabilistic discounting in control rats, although stimulation of this cortical dopamine projection slightly disrupted choices in AdoTHC rats.

Conclusions

These studies confirm a marked specificity in the cognitive processes impacted by AdoTHC exposure. They also suggest that some persistent AdoTHC effects may alter amphetamine-induced cognitive changes in a manner independent of VTA dopamine neurons or their projections to mPFC.

Acute stress differentially alters reward-related decision making and inhibitory control under threat of punishment

Acute stress has various effects on cognition, executive function and certain forms of cost/benefit decision making. Recent studies in rodents indicate that acute stress differentially alters reward-related decisions involving particular types of costs and slows choice latencies. Yet, how stress alters decisions where rewards are linked to punishment is less clear. We examined how 1 h restraint stress, followed by behavioral testing 10 min later altered action-selection on two tasks involving reward-seeking under threat of punishment in well-trained male and female rats. One study used a risky decision-making task involving choice between a small/safe reward and a large/risky one that could coincide with shock, delivered with a probability that increased over blocks of trials. Stress increased risk aversion and punishment sensitivity, reducing preference for the larger/risky reward, while increasing decision latencies and trial omissions in both sexes, when rats were teste. A second study used a "behavioral control" task, requiring inhibition of approach towards a readily available reward associated with punishment. Here, food pellets were delivered over discrete trials, half of which coincided with a 12 s audiovisual cue, signalling that reward retrieval prior to cue termination would deliver shock. Stress exerted sex- and timing-dependent effects on inhibitory control. Males became more impulsive and received more shocks on the stress test, whereas females were unaffected on the stress test, and were actually less impulsive when tested 24 h later. None of the effects of restraint stress were recapitulated by systemic treatment with physiological doses of corticosterone. These findings suggest acute stress induces qualitatively distinct and sometimes sex-dependent effects on punished reward-seeking that are critically dependent on whether animals must either choose between different actions or withhold them to obtain rewards and avoid punishment.

Nucleus accumbens core dopamine D2 receptors are required for performance of the odor span task in male rats

RATIONALE

The nucleus accumbens (NAc) core gates motivationally relevant behavioral action sequences through afferents from cortical and subcortical brain regions. While the role of the NAc core in reward and effort-based decision making is well established, its role in working memory (WM) processes is incompletely understood. The odor span task (OST) has been proposed as a measure of non-spatial working memory capacity (WMC) as it requires rodents to select a novel odor from an increasing number of familiar odors to obtain a food reward.

OBJECTIVE

To assess the role of the NAc core in the OST using (1) reversible chemical inactivation and (2) selective blockade of dopamine D1 and D2 receptors in the area.

METHODS

Well-trained male rats were tested on the OST following intra-NAc core infusions of muscimol/baclofen, the D1 receptor antagonist SCH-23390 (1 μg/hemisphere) and the D2 receptor antagonist eticlopride (1 μg/hemisphere). Behavioral measurements included the average odor span, maximum odor span, choice latency, searching vigor, and patterns of responding during foraging that may relate to impulsivity.

RESULTS

Chemical inactivation of the NAc core significantly decreased odor span relative to sham and vehicle conditions. Selective antagonism of D2, but not D1, receptors in the NAc core also produced deficits in odor span. We found that secondary behavioral measures of choice latency, searching vigor, and responding to the first odor stimulus encountered were largely unaffected by treatment.

CONCLUSIONS

These findings suggest that D2 receptors in the NAc core are required for OST performance.

Dopaminergic modulation of sensitivity to immediate and delayed punishment during decision-making

Effective decision-making involves careful consideration of all rewarding and aversive outcomes. Importantly, negative outcomes often occur later in time, leading to underestimation, or "discounting," of these consequences. Despite the frequent occurrence of delayed outcomes, little is known about the neurobiology underlying sensitivity to delayed punishment during decision-making. The Delayed Punishment Decision-making Task (DPDT) addresses this by assessing sensitivity to delayed versus immediate punishment in rats. Rats initially avoid punished reinforcers, then select this option more frequently when delay precedes punishment. We used DPDT to examine effects of acute systemic administration of catecholaminergic drugs on sensitivity to delayed punishment in male and female adult rats. Cocaine did not affect choice of rewards with immediate punishment but caused a dose-dependent reduction in choice of delayed punishment. Neither activation nor blockade of D1-like dopamine receptor affected decision-making, but activation of D2-like dopamine receptors reduced choice of delayed punishment. D2 blockade did not attenuate cocaine's effects on decision-making, suggesting that cocaine's effects are not dependent on D2 receptor activation. Increasing synaptic norepinephrine via atomoxetine also reduced choice of delayed (but not immediate) punishment. Notably, when DPDT was modified from ascending to descending pre-punishment delays, these drugs did not affect choice of delayed or immediate punishment, although high-dose quinpirole impaired behavioral flexibility. In summary, sensitivity to delayed punishment is regulated by both dopamine and norepinephrine transmission in task-specific fashion. Understanding the neurochemical modulation of decision-making with delayed punishment is a critical step toward treating disorders characterized by aberrant sensitivity to negative consequences.

Enhanced Risky Choice in Male Rats Elicited by the Acute Pharmacological Stressor Yohimbine Involves Prefrontal Dopamine D1 Receptor Activation

BACKGROUND

Acute stress alters risk-based decision-making; however, the underlying neural and neurochemical substrates are underexplored. Given their well-documented stress-inducing effects in humans and laboratory animals, glucocorticoids such as cortisol and corticosterone and the α2-adrenoceptor antagonist yohimbine represent potent pharmacological tools to mimic some characteristics of acute stress.

METHODS

Here, we analyzed the effects of the pharmacological stressors corticosterone and yohimbine given systemically on risk-based decision-making in male rats. Moreover, we investigated whether pharmacological stressor effects on risk-based decision-making involve dopamine D1 receptor stimulation in the dorsal prelimbic cortex (PL). We used a risk discounting task that requires choosing between a certain/small reward lever that always delivered 1 pellet and a risky/large reward lever that delivered 4 pellets with a decreasing probability across subsequent trials.

RESULTS

Systemic administration of yohimbine increased the preference for the risky/large reward lever. By contrast, systemic single administration of corticosterone did not significantly promote risky choice. Moreover, co-administration of corticosterone did not enhance the effects of yohimbine on risky choice. The data further show that the increased preference for the risky/large reward lever under systemic yohimbine was lowered by a concurrent pharmacological blockade of dopamine D1 receptors in the PL.

CONCLUSIONS

Our rodent data provide causal evidence that stimulation of PL D1 receptors may represent a neurochemical mechanism by which the acute pharmacological stressor yohimbine, and possibly nonpharmacological stressors as well, promote risky choice.

The influence of insight on risky decision making and nucleus accumbens activation

During insightful problem solving, the solution appears unexpectedly and is accompanied by the feeling of an AHA!. Research suggests that this affective component of insight can have consequences beyond the solution itself by motivating future behavior, such as risky (high reward and high uncertainty) decision making. Here, we investigate the behavioral and neural support for the motivational role of AHA in decision making involving monetary choices. The positive affect of the AHA! experience has been linked to internal reward. Reward in turn has been linked to dopaminergic signal transmission in the Nucleus Accumbens (NAcc) and risky decision making. Therefore, we hypothesized that insight activates reward-related brain areas, modulating risky decision making. We tested this hypothesis in two studies. First, in a pre-registered online study (Study 1), we demonstrated the behavioral effect of insight-related increase in risky decision making using a visual Mooney identification paradigm. Participants were more likely to choose the riskier monetary payout when they had previously solved the Mooney image with high compared to low accompanied AHA!. Second, in an fMRI study (Study 2), we measured the effects of insight on NAcc activity using a similar Mooney identification paradigm to the one of Study 1. Greater NAcc activity was found when participants solved the Mooney image with high vs low AHA!. Taken together, our results link insight to enhanced NAcc activity and a preference for high but uncertain rewards, suggesting that insight enhances reward-related brain areas possibly via dopaminergic signal transmission, promoting risky decision making.

Circuit and cell-specific contributions to decision making involving risk of explicit punishment in male and female rats

Decision making is a complex cognitive process that recruits a distributed network of brain regions, including the basolateral amygdala (BLA) and nucleus accumbens shell (NAcSh). Recent work suggests that communication between these structures, as well as activity of cells expressing dopamine D2 receptors (D2R) in the NAcSh, are necessary for some forms of decision making; however, the contributions of this circuit and cell population during decision making under risk of punishment are unknown. The current experiments addressed this question using circuit- and cell type-specific optogenetic approaches in rats during a decision-making task involving risk of punishment. In Experiment 1, Long-Evans rats received intra-BLA injections of halorhodopsin or mCherry (control) and in Experiment 2, D2-Cre transgenic rats received intra-NAcSh injections of Cre-dependent halorhodopsin or mCherry. Optic fibers were implanted in the NAcSh in both experiments. Following training in the decision-making task, BLA→NAcSh or D2R-expressing neurons were optogenetically inhibited during different phases of the decision process. Inhibition of the BLA→NAcSh during deliberation (the time between trial initiation and choice) increased choice of the large, risky reward (increased risk taking). Similarly, inhibition during delivery of the large, punished reward increased risk taking, but only in males. Inhibition of D2R-expressing neurons in the NAcSh during deliberation increased risk taking. In contrast, inhibition of these neurons during delivery of the small, safe reward decreased risk taking. These findings extend our knowledge of the neural dynamics of risk taking, revealing sex-dependent circuit recruitment and dissociable activity of selective cell populations during decision making.

Regulation of sex differences in risk-based decision making by gonadal hormones: Insights from rodent models

Men and women differ in their ability to evaluate options that vary in their rewards and the risks that are associated with these outcomes. Most studies have shown that women are more risk averse than men and that gonadal hormones significantly contribute to this sex difference. Gonadal hormones can influence risk-based decision making (i.e., risk taking) by modulating the neurobiological substrates underlying this cognitive process. Indeed, estradiol, progesterone and testosterone modulate activity in the prefrontal cortex, amygdala and nucleus accumbens associated with reward and risk-related information. The use of animal models of decision making has advanced our understanding of the intersection between the behavioral, neural and hormonal mechanisms underlying sex differences in risk taking. This review will outline the current state of this literature, identify the current gaps in knowledge and suggest the neurobiological mechanisms by which hormones regulate risky decision making. Collectively, this knowledge can be used to understand the potential consequences of significant hormonal changes, whether endogenously or exogenously induced, on risk-based decision making as well as the neuroendocrinological basis of neuropsychiatric diseases that are characterized by impaired risk taking, such as substance use disorder and schizophrenia.

Cannabinoid receptor type 1 antagonists alter aspects of risk/reward decision making independent of toluene-mediated effects

Drugs of abuse including cannabis and inhalants impair risk/reward decision making. Cannabis use is often concurrent with inhalant intoxication; yet, preclinical studies investigating the role of endocannabinoids in inhalant misuse are limited. To address this gap in the literature, we used the well-validated probabilistic discounting task to assess risk/reward decision making in rodents following combinations of toluene vapor (a common inhalant) and manipulations of cannabinoid receptor type 1 (CB1R) signaling. As reported previously, acute exposure to toluene vapor disrupted behavioral flexibility during probabilistic discounting. Systemic administration of the CB1R inverse agonist AM281 did not prevent toluene-induced alterations in risky choices, but did independently reduce win-stay behavior, increase choice latency, and increase omissions. Toluene-induced deficits in probabilistic discounting are thought to involve impaired medial prefrontal cortex (mPFC) activity. As we previously reported that some of toluene's inhibitory effects on glutamatergic signaling in the mPFC are endocannabinoid-dependent, we tested the hypothesis that mPFC CB1R activity mediates toluene-induced deficits in discounting. However, bilateral injection of the CB1R inverse agonist AM251 prior to toluene vapor exposure had no effect on toluene-induced changes in risk behavior. In a final set of experiments, we injected the CB1R inverse agonist AM251 (5 and 50 ng), the CB1R agonist WIN55,212-2 (50 ng and 500 ng), or vehicle into the mPFC prior to testing. While mPFC CB1R stimulation did not affect any of the measures tested, the CB1R inverse agonist caused a dose-dependent reduction in win-stay behavior without altering any other measures. Together, these studies indicate that toluene-induced deficits in probabilistic discounting are largely distinct from CB1R-dependent effects that include decreased effectiveness of positive reinforcement (mPFC CB1Rs), decision making speed, and task engagement (non-mPFC CB1Rs).

Seeking motivation and reward: roles of dopamine, hippocampus and supramammillo-septal pathway

Reinforcement learning and goal-seeking behavior are thought to be mediated by midbrain dopamine neurons. However, little is known about neural substrates of curiosity and exploratory behavior, which occur in the absence of clear goal or reward. This is despite behavioral scientists having long suggested that curiosity and exploratory behaviors are regulated by an innate drive. We refer to such behavior as information-seeking behavior and propose 1) key neural substrates and 2) the concept of environment prediction error as a framework to understand information-seeking processes. The cognitive aspect of information-seeking behavior, including the perception of salience and uncertainty, involves, in part, the pathways from the posterior hypothalamic supramammillary region to the hippocampal formation. The vigor of such behavior is modulated by the following: supramammillary glutamatergic neurons; their projections to medial septal glutamatergic neurons; and the projections of medial septal glutamatergic neurons to ventral tegmental dopaminergic neurons. Phasic responses of dopaminergic neurons are characterized as signaling potentially important stimuli rather than rewards. This paper describes how novel stimuli and uncertainty trigger seeking motivation and how these neural substrates modulate information-seeking behavior.

Sign tracking predicts suboptimal behavior in a rodent gambling task

RATIONALE

Reward-associated cues can promote maladaptive behavior, including risky decision-making in a gambling setting. A propensity for sign tracking over goal tracking-i.e., interaction with a reward-predictive cue rather than the site of reward-demonstrates an individual's tendency to transfer motivational value to a cue. However, the relationship of sign tracking to risky decision-making remains unclear.

OBJECTIVES

To determine whether sign tracking predicts risky choice, we used a Pavlovian conditioned approach task to evaluate the tendency of male rats to sign track to a lever cue and then trained rats on a rodent gambling task (rGT) with win-associated cues. We also tested the effects of D-amphetamine, quinpirole (a D₂/D₃ receptor agonist), and PD128907 (a D₃ receptor agonist) on gambling behavior in sign tracker and goal tracker individuals.

RESULTS

Increased sign tracking relative to goal tracking was associated with suboptimal performance on the rGT, including decreased selection of the optimal choice, increased selection of a high-risk/high-reward option, and increased impulsive premature choices. Amphetamine increased choices of a low-risk/low-reward option at the expense of optimal and high-risk choices, whereas quinpirole and PD128907 had little effect on choice allocation, but reduced impulsivity. Drug effects were similar across sign tracker and goal tracker individuals.

CONCLUSIONS

Cue reactivity, as measured by sign tracking, is predictive and may be an important driver of risky and impulsive choices in a gambling setting laden with salient audiovisual cues. Evaluating an individual's sign tracking behavior may be an avenue to predict vulnerability to pathological gambling and the efficacy of treatments.

Medial orbitofrontal cortex dopamine D1/D2 receptors differentially modulate distinct forms of probabilistic decision-making

Efficient decision-making involves weighing the costs and benefits associated with different actions and outcomes to maximize long-term utility. The medial orbitofrontal cortex (mOFC) has been implicated in guiding choice in situations involving reward uncertainty, as inactivation in rats alters choice involving probabilistic rewards. The mOFC receives considerable dopaminergic input, yet how dopamine (DA) modulates mOFC function has been virtually unexplored. Here, we assessed how mOFC D₁ and D₂ receptors modulate two forms of reward seeking mediated by this region, probabilistic reversal learning and probabilistic discounting. Separate groups of well-trained rats received intra-mOFC microinfusions of selective D₁ or D₂ antagonists or agonists prior to task performance. mOFC D₁ and D₂ blockade had opposing effects on performance during probabilistic reversal learning and probabilistic discounting. D₁ blockade impaired, while D₂ blockade increased the number of reversals completed, both mediated by changes in errors and negative feedback sensitivity apparent during the initial discrimination of the task, which suggests changes in probabilistic reinforcement learning rather than flexibility. Similarly, D₁ blockade reduced, while D₂ blockade increased preference for larger/risky rewards. Excess D₁ stimulation had no effect on either task, while excessive D₂ stimulation impaired probabilistic reversal performance, and reduced both profitable risky choice and overall task engagement. These findings highlight a previously uncharacterized role for mOFC DA, showing that D₁ and D₂ receptors play dissociable and opposing roles in different forms of reward-related action selection. Elucidating how DA biases behavior in these situations will expand our understanding of the mechanisms regulating optimal and aberrant decision-making.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.

Dopaminergic modulation of reward discounting in healthy rats: a systematic review and meta-analysis

RATIONALE

Although numerous studies have suggested that pharmacological alteration of the dopamine (DA) system modulates reward discounting, these studies have produced inconsistent findings.

OBJECTIVES

Here, we conducted a systematic review and pre-registered meta-analysis to evaluate DA drug-mediated effects on reward discounting of time, probability, and effort costs in studies of healthy rats. This produced a total of 1343 articles to screen for inclusion/exclusion. From the literature, we identified 117 effects from approximately 1549 individual rats.

METHODS

Using random effects with maximum-likelihood estimation, we meta-analyzed placebo-controlled drug effects for (1) DA D1-like receptor agonists and (2) antagonists, (3) D2-like agonists and (4) antagonists, and (5) DA transporter-modulating drugs.

RESULTS

Meta-analytic effects showed that DAT-modulating drugs decreased reward discounting. While D1-like and D2-like antagonists both increased discounting, agonist drugs for those receptors had no significant effect on discounting behavior. A number of these effects appear contingent on study design features like cost type, rat strain, and microinfusion location.

CONCLUSIONS

These findings suggest a nuanced relationship between DA and discounting behavior and urge caution when drawing generalizations about the effects of pharmacologically manipulating dopamine on reward-based decision-making.

Neural Mechanisms of Human Decision-Making

We present a theory and neural network model of the neural mechanisms underlying human decision-making. We propose a detailed model of the interaction between brain regions, under a proposer-predictor-actor-critic​ ​framework. This theory is based on detailed animal data and theories of action-selection. Those theories are adapted to serial operation to bridge levels of analysis and explain human decision-making. Task-relevant areas of cortex propose a candidate plan using fast, model-free, parallel neural computations. Other areas of cortex and medial temporal lobe can then predict​ likely outcomes of that plan in this situation. This optional prediction- (or model-) based computation can produce better accuracy and generalization, at the expense of speed. Next, linked regions of basal ganglia act​ to accept or reject the proposed plan based on its reward history in similar contexts. If that plan is rejected, the process repeats to consider a new option. The reward-prediction system acts as a critic​ to determine the value of the outcome relative to expectations and produce dopamine as a training signal for cortex and basal ganglia. By operating sequentially and hierarchically, the same mechanisms previously proposed for animal action-selection could explain the most complex human plans and decisions. We discuss explanations of model-based decisions, habitization, and risky behavior based on the computational model.

The role of opioidergic system in modulating cost/benefit decision-making in alcohol-preferring AA rats and Wistar rats

Research has highlighted the association of a positive family history of alcoholism with a positive treatment response to opioid antagonists in those with a gambling disorder. However, the role of the opioidergic system in gambling behavior is not well understood, and preclinical studies are needed to clarify this. In this study, Alko Alcohol (AA) and Wistar rats went through operant lever pressing training where the task was to choose the more profitable of two options. Different sized sucrose rewards guided the lever choices, and the probability of gaining rewards changed slowly to a level where choosing the smaller reward was the most profitable option. After training, rats were administered subcutaneously with opioid agonist morphine or opioid antagonist naltrexone to study the impact of opioidergic mechanisms on cost/benefit decisions. No difference was found in the decision-making between AA rats or Wistar rats after the morphine administration, but control data revealed a minor decision enhancing effect in AA rats. Naltrexone had no impact on the decisions in AA rats but promoted unprofitable decisions in Wistar rats. Supporting behavioral data showed that in both rat strains morphine increased, and naltrexone decreased, sucrose consumption. Naltrexone also increased the time to accomplish the operant task. The results suggest that opioid agonists could improve decision-making in cost-benefit settings in rats that are naturally prone to high alcohol drinking. The naltrexone results are ambiguous but may partly explain why opioid antagonists lack a positive pharmacotherapeutic effect in some subgroups of gamblers.

Impaired working memory, cognitive flexibility and reward processing in mice genetically lacking Gpr88: Evidence for a key role for Gpr88 in multiple cortico-striatal-thalamic circuits

The GPR88 orphan G protein-coupled receptor is expressed throughout the striatum, being preferentially localised in medium spiny neurons. It is also present in lower densities in frontal cortex and thalamus. Rare mutations in humans suggest a role in cognition and motor function, while common variants are associated with psychosis. Here we evaluate the influence of genetic deletion of GPR88 upon performance in translational tasks interrogating motivation, reward evaluation and cognitive function. In an automated radial arm maze 'N-back' working memory task, Gpr88 KO mice showed impaired correct responding, suggesting a role for GPR88 receptors in working memory circuitry. Associative learning performance was similar to wild-type controls in a touchscreen task but performance was impaired at the reversal learning stage, suggesting cognitive inflexibility. Gpr88 KO mice showed higher breakpoints, reduced latencies and lengthened session time in a progressive ratio task consistent with enhanced motivation. Simultaneously, locomotor hyperactivity was apparent in this task, supporting previous findings of actions of GPR88 in a cortico-striatal-thalamic motor loop. Evidence for a role of GPR88 in reward processing was demonstrated in a touchscreen-based equivalent of the Iowa gambling task. Although both Gpr88 KO and wild-type mice showed a preference for an optimum contingency choice, Gpr88 KO mice selected more risky choices at the expense of more advantageous lower risk options. Together these novel data suggest that striatal GPR88 receptors influence activity in a range of procedures integrated by prefrontal, orbitofrontal and anterior cingulate cortico-striatal-thalamic loops leading to altered cognitive, motivational and reward evaluation processes.

Involvement of noradrenergic and serotonergic systems in risk-based decisions between options of equivalent expected value in rats

Risk perception is an important factor that may mediate risk-based decision-making processes regulated by noradrenergic (NA) and serotonergic (5-HT) systems. Most risk-based decision-making models involve complex factors, such as risk perception or reward value, such that the final decision is the result of the interactions among these factors. However, the contribution of risk perception per se in risk decisions has remained unclear. Therefore, in the present study, we made some modifications to the classical probabilistic discounting task (PDT) to focus on the impact of risk perception and noradrenergic/serotonergic systems on decision-making behavior. Meanwhile, we conducted an elevated plus-maze (EPM) test to detect the correlation between anxiety and choice behavior. In the current study, rats had to choose between a "certain" lever that delivered a certain number of pellets and a "risky" lever that delivered eight pellets in a probabilistic manner (descending: 50%, 25%, 12.5% or ascending 12.5%, 25%, 50% of the time). The long-term rewarding values of the two levers were always identical in each block within each session. According to their baseline performances in choosing the risky lever, rats were divided into the risk-prefer group and risk-averse group. The results showed that there was a significant correlation between open arm time in EPM and risky choice for both descending order and ascending order, indicating that highly anxious rats more often preferred the safe option under risk. Pharmacological stimulation of α2-adrenergic receptors via dexmedetomidine (0.01 mg/kg) decreased the preference of probabilistic rewards in the risk-prefer group, while blocking α2 receptors by atipamezole (0.3 mg/kg) also reduced risky choices. The NA reuptake inhibitor, atomoxetine, increased the preference for risky choices in the risk-prefer group, the effect of which was attained via multiple superimposed doses. Administration of the 5-HT_2A receptor agonist, DOI (0.1 mg/kg), increased risk-taking behavior in the risk-prefer group. Taken together, these results suggest that NA may be more inclined to process negative information such as loss or uncertainty in the regulation of risk-related decision making, whereas 5-HT may function primarily to increase risk-taking behavior. Our findings may help to further elucidate how noradrenergic and serotonergic systems differentially affect individuals with different risk preferences in terms of regulating risk perception in risk-related decision making.

Risk-based decision making in rats: Modulation by sex and amphetamine

Decision-making is a complex process essential to daily adaptation in many species. Risk is an inherent aspect of decision-making and it is influenced by gonadal hormones. Testosterone and 17β-estradiol may modulate decision making and impact the mesocorticolimbic dopamine pathway. Here, we explored sex differences, the effect of gonadal hormones and the dopamine agonist amphetamine on risk-based decision making. Intact or gonadectomised (GDX) male and female rats underwent to a probabilistic discounting task. High and low doses of testosterone propionate (1.0 or 0.2 mg) and 17β-estradiol benzoate (0.3 μg) were administered to assess acute effects on risk-based decision making. After 3-days of washout period, intact and GDX rats received high or low (0.5 or 0.125 mg/kg) doses of amphetamine and re-tested in the probabilistic discounting task. Under baseline conditions, males made more risky choices during probability discounting compared to female rats, particularly in the lower probability blocks, but GDX did not influence risky choice. The high, but not the low dose, of testosterone modestly reduced risky decision making in GDX male rats. Conversely, 17β-estradiol had no significant effect on risky choice regardless of GDX status in either sex. Lastly, a higher dose of amphetamine increased risky decision making in both intact males and females, but had no effect in GDX rats. These findings demonstrated sex differences in risk-based decision making, with males showing a stronger bias toward larger, uncertain rewards. GDX status influenced the effects of amphetamine, suggesting different dopaminergic regulation in risk-based choices among males and females.

Pair housing, but not using a controlled reinforcer frequency procedure, attenuates the modulatory effect of probability presentation order on amphetamine-induced changes in risky choice

Probability discounting is often measured with independent schedules. Independent schedules have several limitations, such as confounding preference for one alternative with frequency of reward presentation and generating ceiling/floor effects at certain probabilities. To address this potential caveat, a controlled reinforcer frequency schedule can be used, in which the manipulandum that leads to reinforcement is pseudo-randomly determined before each trial. This schedule ensures subjects receive equal presentations of the small and large magnitude reinforcers across each block of trials. A total of 24 pair-housed and 11 individually housed female Sprague Dawley rats were tested in a controlled reinforcer frequency procedure. For half of the rats, the odds against (OA) receiving the large magnitude reinforcer increased across the session (ascending schedule); the OA decreased across the session for half of the rats (descending schedule). Following training, rats received treatments of amphetamine (AMPH; 0, 0.25, 0.5, 1.0 mg/kg; s.c.). For pair-housed rats, AMPH (0.5 mg/kg) increased risky choice, regardless of probability presentation order, whereas a higher dose of AMPH (1.0 mg/kg) decreased discriminability of reinforcer magnitude for rats trained on the descending schedule only. For individually housed rats, probability presentation order modulated the effects of AMPH on probability discounting, as AMPH (0.25 and 0.5 mg/kg) increased risky choice in rats trained on the ascending schedule but not on the descending schedule. These results show that pair-housing animals, but not using a controlled reinforcer frequency procedure, attenuates the modulatory effects of probability presentation order on drug effects on risky choice.

Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models

Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans.

The Neurobiology of Impulsive Decision-Making and Reinforcement Learning in Nonhuman Animals

Impulsive decisions are those that favor immediate over delayed rewards, involve the acceptance of undue risk or uncertainty, or fail to adapt to environmental changes. Pathological levels of impulsive decision-making have been observed in individuals with mental illness, but there may be substantial heterogeneity in the processes that drive impulsive choices. Understanding this behavioral heterogeneity may be critical for understanding associated diverseness in the neural mechanisms that give rise to impulsivity. The application of reinforcement learning algorithms in the deconstruction of impulsive decision-making phenotypes can help bridge the gap between biology and behavior and provide insights into the biobehavioral heterogeneity of impulsive choice. This chapter will review the literature on the neurobiological mechanisms of impulsive decision-making in nonhuman animals; specifically, the role of the amine neuromodulatory systems (dopamine, serotonin, norepinephrine, and acetylcholine) in impulsive decision-making and reinforcement learning processes is discussed. Ultimately, the integration of reinforcement learning algorithms with sophisticated behavioral and neuroscience techniques may be critical for advancing the understanding of the neurochemical basis of impulsive decision-making.

Basolateral amygdala - nucleus accumbens circuitry regulates optimal cue-guided risk/reward decision making

Maladaptive decision making is a characteristic feature of substance use disorder and pathological gambling. Studies in humans and animals have implicated neural circuits that include the basolateral amygdala (BLA) and nucleus accumbens (NAc) in facilitating risk/reward decision making. However, the preclinical literature has focussed primarily on situations where animals use internally-generated information to adapt to changes in reward likelihood, whereas many real-life situations require the use of external stimuli to facilitate context-appropriate behavior. We recently developed the "Blackjack" task, to measure cued risk/reward decision making requiring rats to chose between Small/Certain and Large/Risky rewards, with auditory cues at the start of each trial explicitly informing that the probability of obtaining a large reward was either good (50%) or poor (12.5%). Here we investigated the contribution of the BLA and its interaction with the NAc in guiding these types of decisions. In well-trained male rats, bilateral inactivation of the BLA induced suboptimal decision making, primarily by reducing risky choice on good-odds trials. In comparison, pharmacological disconnection of the BLA and NAc-shell also induced suboptimal decision making, diverting choice from more preferred option by reducing or increasing risky choice on good vs. poor odds trials respectively. Together, these results suggest that the BLA-NAc circuitry plays a crucial role in integrating information provided by discriminative stimuli. Furthermore, this circuitry may aid in guiding action selection of advantageous options in situations to maximize rewards. Finally, they suggest that perturbations in optimal decision making observed in substance abuse and gambling disorders may be driven in part by dysfunction within this circuitry.

The adverse effects of pramipexole on probability discounting are not reversed by acute D2 or D3 receptor antagonism

Pramipexole (PPX) is a D₂ and D₃ dopamine receptor agonist approved for clinical use, which is associated with a higher risk of impulse-control disorders. Using a rat model, we recently found that low doses of the monoamine-depleting agent reserpine (RES; 1 mg/kg/day, SC) dramatically increased the untoward effects of PPX (0.3 mg/kg/day, SC) on probability discounting, a key impulsivity function. To further understand the neurobehavioral mechanisms underlying these effects, we first tested whether the combination of PPX and RES may lead to a generalized enhancement in risk taking, as tested in the suspended wire-beam paradigm. The association of RES and PPX did not augment the proclivity of rats to cross the bridge in order to obtain a reward, suggesting that the effects of RES and PPX on probability discounting do not reflect a generalized increase in impulsivity. We then studied what receptors mediate the effects of PPX in RES-treated rats. The combination of RES and PPX increased membrane expression and binding of D₃, but not D₂ dopamine receptors, in the nucleus accumbens. However, the behavioral effects of PPX and RES were not reduced by acute treatments with the D₂/D₃ receptor antagonist raclopride (0.01-0.05 mg/kg, SC), the highly selective D₂ receptor antagonist L-741,626 (0.1-1 mg/kg, SC) or the D₃ receptor antagonists GR 103691 (0.1-0.3 mg/kg, SC) and SB 277011A (1-10 mg/kg, SC). These findings collectively suggest that the effects of PPX in probability discounting do not reflect generalized enhancements in impulsivity or acute dopamine D₂ or D₃ receptor activation.

The Nucleus Accumbens Core Is Necessary for Responding to Incentive But Not Instructive Stimuli

An abundant literature has highlighted the importance of the nucleus accumbens core (NAcC) in behavioral tasks dependent on external stimuli. Yet, some studies have also reported the absence of involvement of the NAcC in stimuli processing. We aimed at comparing, in male rats, the underlying neuronal determinants of incentive and instructive stimuli in the same task. We developed a variant of a GO/NOGO task that reveals important differences in these two types of stimuli. The incentive stimulus invites the rat to engage in the task sequence. Once the rat has decided to initiate a trial, it remains engaged in the task until the end of the trial. This task revealed the differential contribution of the NAcC to responding to different types of stimuli: responding to the incentive stimulus depended on NAcC AMPA/NMDA and dopamine D1 receptors, but the retrieval of the response associated with the instructive stimuli (lever pressing on GO, withholding on NOGO) did not. Our electrophysiological study showed that more NAcC neurons responded more strongly to the incentive than the instructive stimuli. Furthermore, when animals did not respond to the incentive stimulus, the induced excitation was suppressed for most projection neurons, whereas interneurons were strongly activated at a latency preceding that found in projection neurons. This work provides insight on the underlying neuronal processes explaining the preferential implication of the NAcC in deciding whether and when to engage in reward-seeking rather than to decide which action to perform.SIGNIFICANCE STATEMENT The nucleus accumbens core (NAcC) is essential to process information carried by reward-predicting stimuli. Yet, stimuli have distinct properties: incentive stimuli orient the attention toward reward-seeking, whereas instructive stimuli inform about the action to perform. Our study shows that, in male rats, NAcC perturbation with glutamate or dopamine antagonists impeded responses to the incentive but not to the instructive stimulus. NAcC neuronal recordings revealed a stronger representation of incentive than instructive stimuli. Furthermore, we found that interneurons are recruited when rats fail to respond to incentive stimuli. This work provides insight on the underlying neuronal processes explaining the preferential implication of the NAcC in deciding whether and when to engage in reward-seeking rather than to decide which action to perform.

Differential effects of d-amphetamine and atomoxetine on risk-based decision making of Lewis and Fischer 344 rats

Individuals with attention-deficit/hyperactivity disorder tend to make risker choices during probabilistic-discounting procedures. Thus, how common attention-deficit/hyperactivity disorder medications affect probabilistic discounting is of interest. In general, d-amphetamine increases risk-taking while atomoxetine has produced mixed effects in rats. Results from previous studies may result from genetic factors. Lewis and F344 rats have neurochemical differences that may be relevant to probabilistic discounting and how drugs affect such behavior. In this study, we evaluated dose-dependent effects of d-amphetamine and atomoxetine on probabilistic discounting of Lewis and F344. Male Lewis and F344 chose between one food pellet delivered 100% of the time and three food pellets delivered following decreasing probabilities of delivery (i.e. 100%, 66.7%, 33.3%, 16.5%, and 8.25%). Saline, d-amphetamine (0.1-1.8 mg/kg), and atomoxetine (0.1-7.8 mg/kg) were tested acutely. Lewis and F344 did not differ in choice at baseline. d-Amphetamine increased risky choice for both rat strains at low-to-moderate doses, although it did so at a lower dose (0.1 and 0.3 mg/kg) for F344 as compared to Lewis (0.3 mg/kg only). At high doses (1.0 and 1.8 mg/kg), d-amphetamine disrupted choice, increased frequencies of omitted trials, and reduced reinforcer sensitivity. Although atomoxetine increased frequencies of omitted trials at high doses (5.6 and 7.8 mg/kg), it had no effect on probabilistic discounting for either rat strain. Although Lewis and F344 differ in various types of impulsivity (i.e. motor, choice), with Lewis being the more impulsive of the two, the present results suggest that Lewis and F344 do not differ in risk-based decision-making. Effects of d-amphetamine on probabilistic discounting may be biology-dependent and differ from effects of atomoxetine.

Alterations in effort-related decision-making induced by stimulation of dopamine D1, D2, D3, and corticotropin-releasing factor receptors in nucleus accumbens subregions

RATIONALE

Nucleus accumbens (NAc) dopamine (DA) plays an integral role in overcoming effort costs, as blockade of D₁ and D₂ receptors reduces the choice of larger, more-costly rewards. Similarly, the stress neuropeptide corticotropin-releasing factor (CRF) modulates DA transmission and mediates stress-induced alterations in effort-related choice.

OBJECTIVES

The current study explored how excessive stimulation of different DA receptors within the NAc core and shell alters effort-related decision-making and compared these effects to those induced by CRF stimulation.

METHODS

Male Long Evans rats were well-trained on an effort-discounting task wherein they choose between a low-effort/low-reward and a high-effort/high-reward lever where the effort requirement increased over blocks (2-20 presses). Dopamine D₁ (SKF 81297, 0.2-2 μg), D_2/3 (quinpirole, 1-10 μg), or D₃ (PD 128,907, 1.5-3 μg) receptor agonists, or CRF (0.5 μg), were infused into the NAc core or shell prior to testing.

RESULTS

Stimulation of D_2/3 receptors with quinpirole in the NAc core or shell markedly reduced the choice of high-effort option and increase choice latencies, without altering preference for larger vs smaller rewards. Stimulation of D₁ or D₃ receptors did not alter choice, although SKF 81297 infusions into the shell reduced response vigor. In comparison, core infusions of CRF flattened the discounting curve, reducing effortful choice when costs were low and increasing it when costs were high.

CONCLUSIONS

Excessive stimulation of NAc D₂ receptors has detrimental effects on effort-related decision-making. Furthermore, CRF stimulation induces dissociable effects on decision-making compared with those induced the effects of stimulation of different DA receptors.

Anabolic-androgenic steroids and cognitive effort discounting in male rats

Anabolic-androgenic steroids (AAS) are drugs of abuse that impair behavior and cognition. In a rodent model of AAS abuse, testosterone-treated male rats expend more physical effort, by repeatedly pressing a lever for a large reward in an operant discounting task. However, since modern society prioritizes cognitive over physical effort, it is important to determine if AAS limit cognitive effort. Here we tested the effects of AAS on a novel cognitive-effort discounting task. Each operant chamber had 3 nose-pokes, opposite 2 levers and a pellet dispenser. Rats pressed a lever to illuminate 1 nose-poke; they responded in the illuminated nose-poke to receive sugar pellets. For the 'easy' lever, the light remained on for 1 s, and a correct response earned 1 pellet. For the 'hard' lever, the light duration decreased from 1 s to 0.1 s across 5 blocks of trials, and a correct response earned 4 pellets. As the duration of the nose-poke light decreased, all rats decreased their choice of the hard lever in a modest discounting curve. Task accuracy also decreased significantly across the 5 blocks of trials. However, there was no effect of testosterone on choice of the hard lever or task accuracy. Antagonism of dopamine D1 or D2 receptors had no effect on lever choice or task accuracy. However, serotonin depletion significantly decreased preference for the hard lever, and impaired task accuracy. Thus, physical effort discounting depends on dopamine activity, while cognitive effort discounting task is sensitive to serotonin. AAS impair physical effort discounting, but not cognitive effort discounting.

Deconstructing value-based decision making via temporally selective manipulation of neural activity: Insights from rodent models

The ability to choose among options that differ in their rewards and costs (value-based decision making) has long been a topic of interest for neuroscientists, psychologists, and economists alike. This is likely because this is a cognitive process in which all animals (including humans) engage on a daily basis, be it routine (which road to take to work) or consequential (which graduate school to attend). Studies of value-based decision making (particularly at the preclinical level) often treat it as a uniform process. The results of such studies have been invaluable for our understanding of the brain substrates and neurochemical systems that contribute to decision making involving a range of different rewards and costs. Value-based decision making is not a unitary process, however, but is instead composed of distinct cognitive operations that function in concert to guide choice behavior. Within this conceptual framework, it is therefore important to consider that the known neural substrates supporting decision making may contribute to temporally distinct and dissociable components of the decision process. This review will describe this approach for investigating decision making, drawing from published studies that have used techniques that allow temporal dissection of the decision process, with an emphasis on the literature in animal models. The review will conclude with a discussion of the implications of this work for understanding pathological conditions that are characterized by impaired decision making.

The Psychopharmacology of Effort-Related Decision Making: Dopamine, Adenosine, and Insights into the Neurochemistry of Motivation

Effort-based decision making is studied using tasks that offer choices between high-effort options leading to more highly valued reinforcers versus low-effort/low-reward options. These tasks have been used to study the involvement of neural systems, including mesolimbic dopamine and related circuits, in effort-related aspects of motivation. Moreover, such tasks are useful as animal models of some of the motivational symptoms that are seen in people with depression, schizophrenia, Parkinson's disease, and other disorders. The present review will discuss the pharmacology of effort-related decision making and will focus on the use of these tasks for the development of drug treatments for motivational dysfunction. Research has identified pharmacological conditions that can alter effort-based choice and serve as models for depression-related symptoms (e.g., the vesicular monoamine transport-2 inhibitor tetrabenazine and proinflammatory cytokines). Furthermore, tests of effort-based choice have identified compounds that are particularly useful for stimulating high-effort work output and reversing the deficits induced by tetrabenazine and cytokines. These studies indicate that drugs that act by facilitating dopamine transmission, as well as adenosine A_2A antagonists, are relatively effective at reversing effort-related impairments. Studies of effort-based choice may lead to the identification of drug targets that could be useful for treating motivational treatments that are resistant to commonly used antidepressants such as serotonin transport inhibitors.

Optogenetic Dissection of Temporal Dynamics of Amygdala-Striatal Interplay during Risk/Reward Decision Making

Decision making often requires weighing costs and benefits of different options that vary in terms of reward magnitude and uncertainty. Previous studies using pharmacological inactivations have shown that the basolateral amygdala (BLA) to nucleus accumbens (NAc) pathway promotes choice towards larger/riskier rewards. Neural activity in BLA and NAc shows distinct, phasic changes in firing prior to choice and following action outcomes, yet, how these temporally-discrete patterns of activity within BLA→NAc circuitry influence choice is unclear. We assessed how optogenetic silencing of BLA terminals in the NAc altered action selection during probabilistic decision making. Rats received intra-BLA infusions of viruses encoding the inhibitory opsin eArchT and were well trained on a probabilistic discounting task, where they chose between smaller/certain rewards and larger rewards delivered in a probabilistic manner, with the odds of obtaining the larger reward changing over a session (50–12.5%). During testing, activity of BLA→NAc inputs were suppressed with 4- to 7-s pulses of light delivered via optic fibers into the NAc during discrete task events: prior to choice or after choice outcomes. Inhibition prior to choice reduced selection of the preferred option, suggesting that during deliberation, BLA→NAc activity biases choice towards preferred rewards. Inhibition during reward omissions increased risky choice during the low-probability block, indicating that activity after non-rewarded actions serves to modify subsequent choice. In contrast, silencing during rewarded outcomes did not reliably affect choice. These data demonstrate how patterns of activity in BLA→NAc circuitry convey different types of information that guide action selection in situations involving reward uncertainty.

Previous cocaine self-administration disrupts reward expectancy encoding in ventral striatum

The nucleus accumbens core (NAc) is important for integrating and providing information to downstream areas about the timing and value of anticipated reward. Although NAc is one of the first brain regions to be affected by drugs of abuse, we still do not know how neural correlates related to reward expectancy are affected by previous cocaine self-administration. To address this issue, we recorded from single neurons in the NAc of rats that had previously self-administered cocaine or sucrose (control). Neural recordings were then taken while rats performed an odor-guided decision-making task in which we independently manipulated value of expected reward by changing the delay to or size of reward across a series of trial blocks. We found that previous cocaine self-administration made rats more impulsive, biasing choice behavior toward more immediate reward. Further, compared to controls, cocaine-exposed rats showed significantly fewer neurons in the NAc that were responsive during odor cues and reward delivery, and in the reward-responsive neurons that remained, diminished directional and value encoding was observed. Lastly, we found that after cocaine exposure, reward-related firing during longer delays was reduced compared to controls. These results demonstrate that prior cocaine self-administration alters reward-expectancy encoding in NAc, which could contribute to poor decision making observed after chronic cocaine use.

Dopaminergic genes are associated with both directed and random exploration

In order to maximize long-term rewards, agents must balance exploitation (choosing the option with the highest payoff) and exploration (gathering information about options that might have higher payoffs). Although the optimal solution to this trade-off is intractable, humans make use of two effective strategies: selectively exploring options with high uncertainty (directed exploration), and increasing the randomness of their choices when they are more uncertain (random exploration). Using a task that independently manipulates these two forms of exploration, we show that single nucleotide polymorphisms related to dopamine are associated with individual differences in exploration strategies. Variation in a gene linked to prefrontal dopamine (COMT) predicted the degree of directed exploration, as well as the overall randomness of responding. Variation in a gene linked to striatal dopamine (DARPP-32) predicted the degree of both directed and random exploration. These findings suggest that dopamine makes multiple contributions to exploration, depending on its afferent target.

Effects of Estrogens on Central Nervous System Neurotransmission: Implications for Sex Differences in Mental Disorders

Nearly one of every five US individuals aged 12 years old or older lives with certain types of mental disorders. Men are more likely to use various types of substances, while women tend to be more susceptible to mood disorders, addiction, and eating disorders, all of which are risks associated with suicidal attempts. Fundamental sex differences exist in multiple aspects of the functions and activities of neurotransmitter-mediated neural circuits in the central nervous system (CNS). Dysregulation of these neural circuits leads to various types of mental disorders. The potential mechanisms of sex differences in the CNS neural circuitry regulating mood, reward, and motivation are only beginning to be understood, although they have been largely attributed to the effects of sex hormones on CNS neurotransmission pathways. Understanding this topic is important for developing prevention and treatment of mental disorders that should be tailored differently for men and women. Studies using animal models have provided important insights into pathogenesis, mechanisms, and new therapeutic approaches of human diseases, but some concerns remain to be addressed. The purpose of this chapter is to integrate human and animal studies involving the effects of the sex hormones, estrogens, on CNS neurotransmission, reward processing, and associated mental disorders. We provide an overview of existing evidence for the physiological, behavioral, cellular, and molecular actions of estrogens in the context of controlling neurotransmission in the CNS circuits regulating mood, reward, and motivation and discuss related pathology that leads to mental disorders.

Selective Effects of the Loss of NMDA or mGluR5 Receptors in the Reward System on Adaptive Decision-Making

Selecting the most advantageous actions in a changing environment is a central feature of adaptive behavior. The midbrain dopamine (DA) neurons along with the major targets of their projections, including dopaminoceptive neurons in the frontal cortex and basal ganglia, play a key role in this process. Here, we investigate the consequences of a selective genetic disruption of NMDA receptor and metabotropic glutamate receptor 5 (mGluR5) in the DA system on adaptive choice behavior in mice. We tested the effects of the mutation on performance in the probabilistic reinforcement learning and probability-discounting tasks. In case of the probabilistic choice, both the loss of NMDA receptors in dopaminergic neurons or the loss mGluR5 receptors in D1 receptor-expressing dopaminoceptive neurons reduced the probability of selecting the more rewarded alternative and lowered the likelihood of returning to the previously rewarded alternative (win-stay). When observed behavior was fitted to reinforcement learning models, we found that these two mutations were associated with a reduced effect of the expected outcome on choice (i.e., more random choices). None of the mutations affected probability discounting, which indicates that all animals had a normal ability to assess probability. However, in both behavioral tasks animals with targeted loss of NMDA receptors in dopaminergic neurons or mGluR5 receptors in D1 neurons were significantly slower to perform choices. In conclusion, these results show that glutamate receptor-dependent signaling in the DA system is essential for the speed and accuracy of choices, but at the same time probably is not critical for correct estimation of probable outcomes.

Reinforcement learning models of risky choice and the promotion of risk-taking by losses disguised as wins in rats

Risky decisions are inherently characterized by the potential to receive gains or incur losses, and these outcomes have distinct effects on subsequent decision-making. One important factor is that individuals engage in loss-chasing, in which the reception of a loss is followed by relatively increased risk-taking. Unfortunately, the mechanisms of loss-chasing are poorly understood, despite the potential importance for understanding pathological choice behavior. The goal of the present experiment was to illuminate the mechanisms governing individual differences in loss-chasing and risky-choice behaviors. Rats chose between a low-uncertainty outcome that always delivered a variable amount of reward and a high-uncertainty outcome that probabilistically delivered reward. Loss-processing and loss-chasing were assessed in the context of losses disguised as wins (LDWs), which are loss outcomes that are presented along with gain-related stimuli. LDWs have been suggested to interfere with adaptive decision-making in humans and thus potentially increase loss-making. Here, the rats presented with LDWs were riskier, in that they made more choices for the high-uncertainty outcome. A series of nonlinear models were fit to individual rats' data to elucidate the possible psychological mechanisms that best account for individual differences in high-uncertainty choices and loss-chasing behaviors. The models suggested that the rats presented with LDWs were more prone to showing a stay bias following high-uncertainty outcomes compared to rats not presented with LDWs. These results collectively suggest that LDWs acquire conditioned reinforcement properties that encourage continued risk-taking and increase loss-chasing following previous high-risk decisions. (PsycINFO Database Record

Dopamine Receptor-Specific Contributions to the Computation of Value

Dopamine is thought to play a crucial role in value-based decision making. However, the specific contributions of different dopamine receptor subtypes to the computation of subjective value remain unknown. Here we demonstrate how the balance between D1 and D2 dopamine receptor subtypes shapes subjective value computation during risky decision making. We administered the D2 receptor antagonist amisulpride or placebo before participants made choices between risky options. Compared with placebo, D2 receptor blockade resulted in more frequent choice of higher risk and higher expected value options. Using a novel model fitting procedure, we concurrently estimated the three parameters that define individual risk attitude according to an influential theoretical account of risky decision making (prospect theory). This analysis revealed that the observed reduction in risk aversion under amisulpride was driven by increased sensitivity to reward magnitude and decreased distortion of outcome probability, resulting in more linear value coding. Our data suggest that different components that govern individual risk attitude are under dopaminergic control, such that D2 receptor blockade facilitates risk taking and expected value processing.

The role of 'jackpot' stimuli in maladaptive decision-making: dissociable effects of D1/D2 receptor agonists and antagonists

RATIONALE

Laboratory experiments often model risk through a choice between a large, uncertain (LU) reward against a small, certain (SC) reward as an index of an individual's risk tolerance. An important factor generally lacking from these procedures are reward-associated cues that may modulate risk preferences.

OBJECTIVE

We tested whether the addition of cues signaling 'jackpot' wins to LU choices would modulate risk preferences and if these cue effects were mediated by dopaminergic signaling.

METHODS

Three groups of rats chose between LU and SC rewards for which the LU probability of reward decreased across blocks. The unsignaled group received a non-informative stimulus of trial outcome. The signaled group received a jackpot signal prior to reward delivery and blackout on losses. The signaled-light group received a similar jackpot for wins, but a salient loss signal distinct from the win signal.

RESULTS

Presenting win signals decreased the discounting of LU value for both signaled groups regardless of loss signal, while the unsignaled group showed discounting similar to previous research without cues. Pharmacological challenges with D1/D2 agonists and antagonists revealed that D1 antagonism increased and decreased sensitives to the relative probability of reward for unsignaled and signaled groups, respectively, while D2 agonists decreased sensitivities to the relative magnitude of reward.

CONCLUSION

The results highlight how signals predictive of wins can promote maladaptive risk taking in individuals, while loss signals have reduced effect. Additionally, the presence of reward-predictive cues may change the underlying neurobehavioral mechanisms mediating decision-making under risk.

Amphetamine primes enhanced motivation toward uncertain choices in rats with genetic alcohol preference

RATIONALE

Comorbidity with gambling disorder (GD) and alcohol use disorder (AUD) is well documented.

OBJECTIVE

The purpose of our study was to examine the influence of genetic alcohol drinking tendency on reward-guided decision making behavior of rats and the impact of dopamine releaser D-amphetamine on this behavior.

METHODS

In this study, Alko alcohol (AA) and Wistar rats went through long periods of operant lever pressing training where the task was to choose the profitable of two options. The lever choices were guided by different-sized sucrose rewards (one or three pellets), and the probability of gaining the larger reward was slowly changed to a level where choosing the smaller reward would be the most profitable in the long run. After training, rats were injected (s.c.) with dopamine releaser D-amphetamine (0.3, 1.0 mg/kg) to study the impact of rapid dopamine release on this learned decision making behavior.

RESULTS

Administration of D-amphetamine promoted unprofitable decision making of AA rats more robustly when compared to Wistar rats. At the same time, D-amphetamine reduced lever pressing responses. Interestingly, we found that this reduction in lever pressing was significantly greater in Wistar rats than in AA rats and it was not linked to motivation to consume sucrose.

CONCLUSIONS

Our results indicate that conditioning to the lever pressing in uncertain environments is more pronounced in AA than in Wistar rats and indicate that the reinforcing effects of a gambling-like environment act as a stronger conditioning factor for rats that exhibit a genetic tendency for high alcohol drinking.

Dopaminergic Drug Effects on Probability Weighting during Risky Decision Making

Dopamine has been associated with risky decision-making, as well as with pathological gambling, a behavioral addiction characterized by excessive risk-taking behavior. However, the specific mechanisms through which dopamine might act to foster risk-taking and pathological gambling remain elusive. Here we test the hypothesis that this might be achieved, in part, via modulation of subjective probability weighting during decision making. Human healthy controls (n = 21) and pathological gamblers (n = 16) played a decision-making task involving choices between sure monetary options and risky gambles both in the gain and loss domains. Each participant played the task twice, either under placebo or the dopamine D₂/D₃ receptor antagonist sulpiride, in a double-blind counterbalanced design. A prospect theory modelling approach was used to estimate subjective probability weighting and sensitivity to monetary outcomes. Consistent with prospect theory, we found that participants presented a distortion in the subjective weighting of probabilities, i.e., they overweighted low probabilities and underweighted moderate to high probabilities, both in the gain and loss domains. Compared with placebo, sulpiride attenuated this distortion in the gain domain. Across drugs, the groups did not differ in their probability weighting, although gamblers consistently underweighted losing probabilities in the placebo condition. Overall, our results reveal that dopamine D₂/D₃ receptor antagonism modulates the subjective weighting of probabilities in the gain domain, in the direction of more objective, economically rational decision making.

Anabolic-androgenic steroids (AAS) increase sensitivity to uncertainty by inhibition of dopamine D1 and D2 receptors

BACKGROUND

Anabolic-androgenic steroid abuse is implicated in maladaptive behaviors such as impaired cognition in humans. In a rat model, our lab has shown that testosterone decreases preference for a large/uncertain reward in probability discounting. Other studies have shown that androgens decrease dopamine D1 and D2 receptors in the nucleus accumbens shell, a region important for decision-making behavior in probability discounting. Thus, we attempted to restore selection of the large/uncertain reward in testosterone-treated rats by administering the D2 receptor agonist quinpirole or the D1 receptor agonist SKF81297 and testing probability discounting.

METHODS

Adolescent male Long-Evans rats were treated chronically with high-dose testosterone (7.5 mg/kg) or vehicle (13% cyclodextrin in water), and tested for probability discounting after injections of saline, 0.1 and 0.5 mg/kg of quinpirole or SKF81297. Rats chose between a small/certain reward (1 sugar pellet, 100% probability) and a large/uncertain reward (4 pellets, decreasing probability: 100, 75, 50, 25, 0%).

RESULTS

Testosterone-treated rats selected the large/uncertain reward significantly less than vehicle-treated controls after saline injection. However, acute injection with 0.1 mg/kg quinpirole increased large/uncertain reward preference in testosterone-treated rats only, indicated by a testosterone × quinpirole interaction. At 0.5 mg/kg, quinpirole increased large/uncertain reward preference in all rats. Acute injection with SKF81297 at 0.1 or 0.5 mg/kg rescued large/uncertain reward preference in testosterone-treated rats by eliminating the difference between groups.

CONCLUSIONS

It appears that altered probability discounting behavior in testosterone-treated rats is due to both decreased D1 and D2 receptor function.

Low Striatal Dopamine D2-type Receptor Availability is Linked to Simulated Drug Choice in Methamphetamine Users

Individuals with drug use disorders seek drugs over other rewarding activities, and exhibit neurochemical deficits related to dopamine, which is involved in value-based learning and decision-making. Thus, a dopaminergic disturbance may underpin drug-biased choice in addiction. Classical drug-choice assessments, which offer drug-consumption opportunities, are inappropriate for addicted individuals seeking treatment or abstaining. Fifteen recently abstinent methamphetamine users and 15 healthy controls completed two laboratory paradigms of 'simulated' drug choice (choice for drug-related vs affectively pleasant, unpleasant, and neutral images), and underwent positron emission tomography measurements of dopamine D2-type receptor availability, indicated by binding potential (BP_ND) for [¹⁸F]fallypride. Thirteen of the methamphetamine users and 10 controls also underwent [¹¹C]NNC112 PET scans to measure dopamine D1-type receptor availability. Group analyses showed that, compared with controls, methamphetamine users chose to view more methamphetamine-related images on one task, with a similar trend on the second task. Regression analyses showed that, on both tasks, the more methamphetamine users chose to view methamphetamine images, specifically vs pleasant images (the most frequently chosen images across all participants), the lower was their D2-type BP_ND in the lateral orbitofrontal cortex, an important region in value-based choice. No associations were observed with D2-type BP_ND in striatal regions, or with D1-type BP_ND in any region. These results identify a neurochemical correlate for a laboratory drug-seeking paradigm that can be administered to treatment-seeking and abstaining drug-addicted individuals. More broadly, these results refine the central hypothesis that dopamine-system deficits contribute to drug-biased decision-making in addiction, here showing a role for the orbitofrontal cortex.

A neuronal mechanism underlying decision-making deficits during hyperdopaminergic states

Hyperdopaminergic states in mental disorders are associated with disruptive deficits in decision making. However, the precise contribution of topographically distinct mesencephalic dopamine pathways to decision-making processes remains elusive. Here we show, using a multidisciplinary approach, how hyperactivity of ascending projections from the ventral tegmental area (VTA) contributes to impaired flexible decision making in rats. Activation of the VTA-nucleus accumbens pathway leads to insensitivity to loss and punishment due to impaired processing of negative reward prediction errors. In contrast, activation of the VTA-prefrontal cortex pathway promotes risky decision making without affecting the ability to choose the economically most beneficial option. Together, these findings show how malfunction of ascending VTA projections affects value-based decision making, suggesting a potential mechanism through which increased forebrain dopamine signaling leads to aberrant behavior, as is seen in substance abuse, mania, and after dopamine replacement therapy in Parkinson's disease.

Differential Contributions of Nucleus Accumbens Subregions to Cue-Guided Risk/Reward Decision Making and Implementation of Conditional Rules

The nucleus accumbens (NAc) is a key node within corticolimbic circuitry for guiding action selection and cost/benefit decision making in situations involving reward uncertainty. Preclinical studies have typically assessed risk/reward decision making using assays where decisions are guided by internally generated representations of choice-outcome contingencies. Yet, real-life decisions are often influenced by external stimuli that inform about likelihoods of obtaining rewards. How different subregions of the NAc mediate decision making in such situations is unclear. Here, we used a novel assay colloquially termed the "Blackjack" task that models these types of situations. Male Long-Evans rats were trained to choose between one lever that always delivered a one-pellet reward and another that delivered four pellets with different probabilities [either 50% (good-odds) or 12.5% (poor-odds)], which were signaled by one of two auditory cues. Under control conditions, rats selected the large/risky option more often on good-odds versus poor-odds trials. Inactivation of the NAc core caused indiscriminate choice patterns. In contrast, NAc shell inactivation increased risky choice, more prominently on poor-odds trials. Additional experiments revealed that both subregions contribute to auditory conditional discrimination. NAc core or shell inactivation reduced Pavlovian approach elicited by an auditory CS+, yet shell inactivation also increased responding during presentation of a CS-. These data highlight distinct contributions for NAc subregions in decision making and reward seeking guided by discriminative stimuli. The core is crucial for implementation of conditional rules, whereas the shell refines reward seeking by mitigating the allure of larger, unlikely rewards and reducing expression of inappropriate or non-rewarded actions.SIGNIFICANCE STATEMENT Using external cues to guide decision making is crucial for adaptive behavior. Deficits in cue-guided behavior have been associated with neuropsychiatric disorders, such as attention deficit hyperactivity disorder and schizophrenia, which in turn has been linked to aberrant processing in the nucleus accumbens. However, many preclinical studies have often assessed risk/reward decision making in the absence of explicit cues. The current study fills that gap by using a novel task that allows for the assessment of cue-guided risk/reward decision making in rodents. Our findings identified distinct yet complementary roles for the medial versus lateral portions of this nucleus that provide a broader understanding of the differential contributions it makes to decision making and reward seeking guided by discriminative stimuli.

Androgen Regulation of the Mesocorticolimbic System and Executive Function

Multiple lines of evidence indicate that androgens, such as testosterone, modulate the mesocorticolimbic system and executive function. This review integrates neuroanatomical, molecular biological, neurochemical, and behavioral studies to highlight how endogenous and exogenous androgens alter behaviors, such as behavioral flexibility, decision making, and risk taking. First, we briefly review the neuroanatomy of the mesocorticolimbic system, which mediates executive function, with a focus on the ventral tegmental area (VTA), nucleus accumbens (NAc), medial prefrontal cortex (mPFC), and orbitofrontal cortex (OFC). Second, we present evidence that androgen receptors (AR) and other steroid receptors are expressed in the mesocorticolimbic system. Using sensitive immunohistochemistry and quantitative polymerase chain reaction (qPCR) techniques, ARs are detected in the VTA, NAc, mPFC, and OFC. Third, we describe recent evidence for local androgens ("neuroandrogens") in the mesocorticolimbic system. Steroidogenic enzymes are expressed in mesocorticolimbic regions. Furthermore, following long-term gonadectomy, testosterone is nondetectable in the blood but detectable in the mesocorticolimbic system, using liquid chromatography tandem mass spectrometry. However, the physiological relevance of neuroandrogens remains unknown. Fourth, we review how anabolic-androgenic steroids (AAS) influence the mesocorticolimbic system. Fifth, we describe how androgens modulate the neurochemistry and structure of the mesocorticolimbic system, particularly with regard to dopaminergic signaling. Finally, we discuss evidence that androgens influence executive functions, including the effects of androgen deprivation therapy and AAS. Taken together, the evidence indicates that androgens are critical modulators of executive function. Similar to dopamine signaling, there might be optimal levels of androgen signaling within the mesocorticolimbic system for executive functioning. Future studies should examine the regulation and functions of neurosteroids in the mesocorticolimbic system, as well as the potential deleterious and enduring effects of AAS use.

Sex differences in animal models of decision making

The ability to weigh the costs and benefits of various options to make an adaptive decision is critical to an organism's survival and wellbeing. Many psychiatric diseases are characterized by maladaptive decision making, indicating a need for better understanding of the mechanisms underlying this process and the ways in which it is altered under pathological conditions. Great strides have been made in uncovering these mechanisms, but the majority of what is known comes from studies conducted solely in male subjects. In recent years, decision-making research has begun to include female subjects to determine whether sex differences exist and to identify the mechanisms that contribute to such differences. This Mini-Review begins by describing studies that have examined sex differences in animal (largely rodent) models of decision making. Possible explanations, both theoretical and biological, for such differences in decision making are then considered. The Mini-Review concludes with a discussion of the implications of sex differences in decision making for understanding psychiatric conditions. © 2016 Wiley Periodicals, Inc.

Deciphering Decision Making: Variation in Animal Models of Effort- and Uncertainty-Based Choice Reveals Distinct Neural Circuitries Underlying Core Cognitive Processes

Maladaptive decision-making is increasingly recognized to play a significant role in numerous psychiatric disorders, such that therapeutics capable of ameliorating core impairments in judgment may be beneficial in a range of patient populations. The field of "decision neuroscience" is therefore in its ascendancy, with researchers from diverse fields bringing their expertise to bear on this complex and fascinating problem. In addition to the advances in neuroimaging and computational neuroscience that contribute enormously to this area, an increase in the complexity and sophistication of behavioral paradigms designed for nonhuman laboratory animals has also had a significant impact on researchers' ability to test the causal nature of hypotheses pertaining to the neural circuitry underlying the choice process. Multiple such decision-making assays have been developed to investigate the neural and neurochemical bases of different types of cost/benefit decisions. However, what may seem like relatively trivial variation in behavioral methodologies can actually result in recruitment of distinct cognitive mechanisms, and alter the neurobiological processes that regulate choice. Here we focus on two areas of particular interest, namely, decisions that involve an assessment of uncertainty or effort, and compare some of the most prominent behavioral paradigms that have been used to investigate these processes in laboratory rodents. We illustrate how an appreciation of the diversity in the nature of these tasks can lead to important insights into the circumstances under which different neural regions make critical contributions to decision making.