Deciphering Decision Making: Variation in Animal Models of Effort- and Uncertainty-Based Choice Reveals Distinct Neural Circuitries Underlying Core Cognitive Processes

Sex differences in sensitivity to dopamine receptor manipulations of risk-based decision making in rats

Risky decision making involves the ability to weigh risks and rewards associated with different options to make adaptive choices. Previous work has established a necessary role for the basolateral amygdala (BLA) in mediating effective decision making under risk of punishment, but the mechanisms by which the BLA mediates this process are less clear. Because this form of decision making is profoundly sensitive to dopaminergic (DA) manipulations, we hypothesized that DA receptors in the BLA may be involved in risk-taking behavior. To test this hypothesis, male and female Long-Evans rats were trained in a decision-making task in which rats chose between a small, safe food reward and a larger food reward that was associated with a variable risk of footshock punishment. Once behavioral stability emerged, rats received intra-BLA infusions of ligands targeting distinct dopamine receptor subtypes prior to behavioral testing. Intra-BLA infusions of the dopamine D2 receptor (D2R) agonist quinpirole decreased risk taking in females at all doses, and this reduction in risk taking was accompanied by an increase in sensitivity to punishment. In males, decreased risk taking was only observed at the highest dose of quinpirole. In contrast, intra-BLA manipulations of dopamine D1 or D3 receptors (D1R and D3R, respectively) had no effect on risk taking. Considered together, these data suggest that differential D2R sensitivity in the BLA may contribute to the well-established sex differences in risk taking. Neither D1Rs nor D3Rs, however, appear to contribute to risky decision making in either sex.

Exposure to low (10 cGy) doses of 4He ions leads to an apparent increase in risk taking propensity in female rats

The planned missions to the Moon and Mars will present more significant health challenges to astronauts compared to low earth orbit missions. During deep space missions, astronauts will be constantly exposed to Space radiation (SR). Multiple rodent studies suggest that < 25 cGy of SR impairs performance in executive functions, which play a key role in advanced cognitive processes, but also regulate response inhibition and impulse control. There is the possibility that SR exposure may exacerbate aberrant behaviors evoked by psychological stress related to exposure to isolated and confined (ICE) hostile environment or independently induce additional aberrant behaviors. This study has determined that female Wistar rats exposed to 10 cGy of 250 MeV/n He had an increased risk taking propensity (RTP)\compared to shams. The increased RTP of the He-exposed rats was associated with significantly increased reaction times during the trials, suggesting a SR-induced loss of processing speed. The response times of the He-exposed rats were even further reduced in trials that immediately followed a loss, raising the possibility that conflict and interference avoidance may be impaired after SR exposure. Whether these findings occur following other types of SR exposure, and/or in male rats remains to be determined.

A Double Hit of Social and Economic Stress in Mice Precipitates Changes in Decision-Making Strategies

BACKGROUND

Economic stress can serve as a second hit for people who have already accumulated a history of adverse life experiences. How one recovers from a setback is a core feature of resilience but is seldom captured in animal studies.

METHODS

We challenged mice in a novel 2-hit stress model by first exposing them to chronic social defeat stress and then testing adaptations to increasing reward scarcity on a neuroeconomic task. Mice were tested across months on the Restaurant Row task, during which they foraged daily for their primary source of food while on a limited time budget in a closed-economy system. An abrupt transition into a reward-scarce environment elicits an economic challenge, precipitating a drop in food intake and body weight to which mice must respond to survive.

RESULTS

We found that mice with a history of social stress mounted a robust behavioral response to this economic challenge that was achieved through a complex redistribution of time allocation among competing opportunities. Interestingly, we found that mice with a history of social defeat displayed changes in the development of decision-making policies during the recovery process that are important not only for ensuring food security necessary for survival but also prioritizing subjective value and that these changes emerged only for certain types of choices.

CONCLUSIONS

These findings indicate that an individual's capacity to recover from economic challenges depends on that person's prior history of stress and can affect multiple decision-making aspects of subjective well-being, thus highlighting a motivational balance that may be altered in stress-related disorders such as depression.

Methamphetamine increases risky choice in rats, but only when magnitude and probability of reinforcement are manipulated within a session

Risky choice is associated with maladaptive behaviors, particularly substance use disorders. Current animal models of risky choice are often confounded by other constructs like behavioral flexibility and suboptimal choice. The purpose of the current experiment was to determine if the psychostimulant methamphetamine, a drug whose popularity has increased in recent years, increases risky choice in an equivalent expected value (EEV) task. In the EEV task, rats are given a choice between two reinforcer alternatives that differ in magnitude and probability of delivery, but have equivalent expected value. Forty-eight Sprague Dawley rats were tested in three versions of the EEV task. In the first version of the EEV task, both reinforcer magnitude and probability were adjusted across blocks of trials for both alternatives. In the second and the third versions of the EEV task, reinforcer magnitude was held constant across each block of trials (either 1 vs. 2 pellets or 4 vs. 5 pellets). We found that male rats preferred the "riskier" option, except when reinforcer magnitudes were held constant at 4 and 5 pellets across each block of trials. Methamphetamine (0.5 mg/kg) increased preference for the risky option in both males and females, but only when both reinforcer magnitude and probability were manipulated across blocks of trials for each alternative. The current results demonstrate that both magnitude of reinforcement and probability of reinforcement interact to influence risky choice. Overall, this study provides additional support for using reinforcers with expected value to measure risky choice.

Acute stress differentially alters reward-related decision making and inhibitory control under threat of punishment

Acute stress has various effects on cognition, executive function and certain forms of cost/benefit decision making. Recent studies in rodents indicate that acute stress differentially alters reward-related decisions involving particular types of costs and slows choice latencies. Yet, how stress alters decisions where rewards are linked to punishment is less clear. We examined how 1 h restraint stress, followed by behavioral testing 10 min later altered action-selection on two tasks involving reward-seeking under threat of punishment in well-trained male and female rats. One study used a risky decision-making task involving choice between a small/safe reward and a large/risky one that could coincide with shock, delivered with a probability that increased over blocks of trials. Stress increased risk aversion and punishment sensitivity, reducing preference for the larger/risky reward, while increasing decision latencies and trial omissions in both sexes, when rats were teste. A second study used a "behavioral control" task, requiring inhibition of approach towards a readily available reward associated with punishment. Here, food pellets were delivered over discrete trials, half of which coincided with a 12 s audiovisual cue, signalling that reward retrieval prior to cue termination would deliver shock. Stress exerted sex- and timing-dependent effects on inhibitory control. Males became more impulsive and received more shocks on the stress test, whereas females were unaffected on the stress test, and were actually less impulsive when tested 24 h later. None of the effects of restraint stress were recapitulated by systemic treatment with physiological doses of corticosterone. These findings suggest acute stress induces qualitatively distinct and sometimes sex-dependent effects on punished reward-seeking that are critically dependent on whether animals must either choose between different actions or withhold them to obtain rewards and avoid punishment.

Sex differences in risk-based decision-making and the modulation of risk preference by dopamine-2 like receptors in rats

Heightened risk-based decision-making is observed across several neuropsychiatric disorders including schizophrenia, bipolar disorder, and Parkinson's disease, yet no treatments exist that effectively normalize this aberrant behavior. Preclinical risk-based decision-making paradigms have identified the important modulatory roles of dopamine and sex in the performance of such tasks, though specific task parameters may alter such effects (e.g., punishment and reward values). Previous work has highlighted the role of dopamine 2-like receptors (D2R) during performance of the Risk Preference Task (RPT) in male rats, however sex was not considered as a factor in this study, nor were treatments identified that reduced risk preference. Here, we utilized the RPT to determine sex-dependent differences in baseline performance and impact of the D2R receptor agonist pramipexole (PPX), and antagonist sulpiride (SUL) on behavioral performance. Female rats exhibited heightened risk-preference during baseline testing. Consistent with human studies, PPX increased risk-preference across sex, though the effects of PPX were more pronounced in female animals. Importantly, SUL reduced risk-preference in these rats across sexes. Thus, under the task specifications of the RPT that does not include punishment, female rats were more risk-preferring and required higher PPX doses to promote risky choices compared to males. Furthermore, blockade of D2R receptors may reduce risk-preference of rats, though further studies are required.

Dopamine activity in the nigrostriatal pathway alters cue-induced risky choice patterns in female rats

Deficits in cost/benefit decision making is a critical risk factor for gambling disorder. Reward-paired cues may play an important role, as these stimuli can enhance risk preference in rats. Despite extensive research implicating the dorsal striatum in the compulsive aspects of addiction, the role of nigrostriatal dopaminergic activity in cue-induced risk preference remains unclear, particularly in females. Accordingly, we examined the effects of manipulating the dopaminergic nigrostriatal pathway on cue-induced risky choice in female rats. TH:Cre rats were trained on the cued version of the rat Gambling Task. This task was designed such that maximal reward is attained by avoiding the high-risk, high-reward options and instead favouring the options associated with lower per-trial gains, as they feature less frequent and shorter time-out penalties. Adding reward-paired audiovisual cues to the task leads to greater risky choice on average. To assess the role of the nigrostriatal pathway, a viral vector carrying either Cre-dependent inhibitory or excitatory DREADD was infused into the substantia nigra. Rats then received clozapine-N-oxide either during task acquisition or after a stable performance baseline was reached. Inhibition of this pathway accelerated the development of risk preference in early sessions and increased risky choice during performance, but long-term inhibition actually improved decision making. Activation of this pathway had minimal effects. These results provide evidence for the involvement of the dopaminergic nigrostriatal pathway in cue-induced risk preference in females, therefore shedding light on its role in cost/benefit decision-making deficits and expanding our knowledge of the female dopaminergic system.

A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information

Subjects are often willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No Info) vs. a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%) and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory Designer Receptors Exclusively Activated by Designer Drugs (DREADDs) in the anterior cingulate cortex, orbitofrontal cortex, basolateral amygdala, or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of the anterior cingulate cortex destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that previous choice predicted current choice in all conditions, however previously rewarded Info trials strongly predicted preference in all conditions except in female rats following anterior cingulate cortex inhibition. The results reveal a causal, sex-dependent role for the anterior cingulate cortex in decisions involving information.

Dissociable Contributions of Basolateral Amygdala and Ventrolateral Orbitofrontal Cortex to Flexible Learning Under Uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including the highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, the unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory designer receptors exclusively activated by designer drugs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-/post-reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of the ventrolateral orbitofrontal cortex (vlOFC), but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Positive correlation between measures of habitual responding and motivated responding in mice

Habit and motivation are thought to be separate processes, with motivated behavior often considered to be goal directed, whereas habits are defined by the absence of goal-directed control over behavior. However, there has been increasing interrogation of the binary nature of habitual versus goal-directed behavior. Furthermore, although drug and alcohol exposure can promote the formation of habits, drug seeking itself can also be highly flexible, pointing toward the need for complex consideration of the parallel processes that drive behavior. The goal of the current study was to determine whether there was a relation between motivation-as measured by progressive ratio-and habit-as measured by contingency degradation-and whether this relation was affected by ethanol exposure history and sex. The results showed that these measures were positively correlated such that greater contingency insensitivity was associated with achieving higher break points on the progressive-ratio task. However, this relation depended on reinforcement schedule history, ethanol exposure history, and sex. These point to potential relations between measures of habit and motivation and stress the importance of carefully parsing behavioral findings and assays. These findings are also expected to inform future substance use research, as drug history may affect these relations.

A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information

Subjects often are willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No-info) vs a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%), and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory DREADDs in the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC), basolateral amygdala (BLA), or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of ACC destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that the previous choice strongly predicted preference in control animals, but not in female rats following ACC inhibition. The results reveal a causal, sex-dependent role for ACC in decisions involving information.

Hyperdopaminergia in rats is associated with reverse effort-cost dependent performance

BACKGROUND

Dopamine is implicated in the effort-based control of motivational processes; however, whether tonic dopamine regulates the effort-cost impact on motivation, is still debated.

AIMS

The rats lacking the dopamine transporter (DAT), which have dramatically increased levels of the synaptic dopamine, were used in the present study to elucidate the role of the synaptic dopamine in motivational processes.

METHODS

To study the reward-related processes, the progressive ratio 3 (PR3) operant schedule of food reinforcement (the ratio increases by 3 after each earned reinforcer) was performed in adult male rats (DAT knockouts (DAT-KO), heterozygotes (DAT-HT) and wild-types (DAT-WT)).

RESULTS

During the PR3 session, the response rate of DAT-KO rats was gradually increased following the augmented required number of responses. In contrast, the local response rate of DAT-WT and DAT-HT decreased. d-Amphetamine sulfate salt (3 mg/kg, i.p.) altered the local response rate dynamics in DAT-WT, which became similar to that of DAT-KO. Interestingly, the reduction in response rate at low effort demands was associated with decreased rate of entries into the magazine tray in DAT-WT rats treated with amphetamine (3 mg/kg) but not in DAT-KO rats.

CONCLUSIONS

Our results suggest that the elevated tonic synaptic dopamine can strongly affect motivation/effort-cost relation in rodents.

Dissociable contributions of basolateral amygdala and ventrolateral orbitofrontal cortex to flexible learning under uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach and particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory DREADDs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-post reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of vlOFC, but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Circuit and Cell-Specific Contributions to Decision Making Involving Risk of Explicit Punishment in Male and Female Rats

Decision making is a complex cognitive process that recruits a distributed network of brain regions, including the basolateral amygdala (BLA) and nucleus accumbens shell (NAcSh). Recent work suggests that communication between these structures, as well as activity of cells expressing dopamine (DA) D2 receptors (D2R) in the NAcSh, are necessary for some forms of decision making; however, the contributions of this circuit and cell population during decision making under risk of punishment are unknown. The current experiments addressed this question using circuit-specific and cell type-specific optogenetic approaches in rats during a decision making task involving risk of punishment. In experiment 1, Long-Evans rats received intra-BLA injections of halorhodopsin or mCherry (control) and in experiment 2, D2-Cre transgenic rats received intra-NAcSh injections of Cre-dependent halorhodopsin or mCherry. Optic fibers were implanted in the NAcSh in both experiments. Following training in the decision making task, BLA→NAcSh or D2R-expressing neurons were optogenetically inhibited during different phases of the decision process. Inhibition of the BLA→NAcSh during deliberation (the time between trial initiation and choice) increased preference for the large, risky reward (increased risk taking). Similarly, inhibition during delivery of the large, punished reward increased risk taking, but only in males. Inhibition of D2R-expressing neurons in the NAcSh during deliberation increased risk taking. In contrast, inhibition of these neurons during delivery of the small, safe reward decreased risk taking. These findings extend our knowledge of the neural dynamics of risk taking, revealing sex-dependent circuit recruitment and dissociable activity of selective cell populations during decision making.SIGNIFICANCE STATEMENT Until recently, the ability to dissect the neural substrates of decision making involving risk of punishment (risk taking) in a circuit-specific and cell-specific manner has been limited by the tools available for use in rats. Here, we leveraged the temporal precision of optogenetics, together with transgenic rats, to probe contributions of a specific circuit and cell population to different phases of risk-based decision making. Our findings reveal basolateral amygdala (BLA)→nucleus accumbens shell (NAcSh) is involved in evaluation of punished rewards in a sex-dependent manner. Further, NAcSh D2 receptor (D2R)-expressing neurons make unique contributions to risk taking that vary across the decision making process. These findings advance our understanding of the neural principles of decision making and provide insight into how risk taking may become compromised in neuropsychiatric diseases.

Mechanisms of adjustments to different types of uncertainty in the reward environment across mice and monkeys

Despite being unpredictable and uncertain, reward environments often exhibit certain regularities, and animals navigating these environments try to detect and utilize such regularities to adapt their behavior. However, successful learning requires that animals also adjust to uncertainty associated with those regularities. Here, we analyzed choice data from two comparable dynamic foraging tasks in mice and monkeys to investigate mechanisms underlying adjustments to different types of uncertainty. In these tasks, animals selected between two choice options that delivered reward probabilistically, while baseline reward probabilities changed after a variable number (block) of trials without any cues to the animals. To measure adjustments in behavior, we applied multiple metrics based on information theory that quantify consistency in behavior, and fit choice data using reinforcement learning models. We found that in both species, learning and choice were affected by uncertainty about reward outcomes (in terms of determining the better option) and by expectation about when the environment may change. However, these effects were mediated through different mechanisms. First, more uncertainty about the better option resulted in slower learning and forgetting in mice, whereas it had no significant effect in monkeys. Second, expectation of block switches accompanied slower learning, faster forgetting, and increased stochasticity in choice in mice, whereas it only reduced learning rates in monkeys. Overall, while demonstrating the usefulness of metrics based on information theory in examining adaptive behavior, our study provides evidence for multiple types of adjustments in learning and choice behavior according to uncertainty in the reward environment.

D2/3 Agonist during Learning Potentiates Cued Risky Choice

Impulse control and/or gambling disorders can be triggered by dopamine agonist therapies used to treat Parkinson's disease, but the cognitive and neurobiological mechanisms underlying these adverse effects are unknown. Recent data show that adding win-paired sound and light cues to the rat gambling task (rGT) potentiates risky decision-making and impulsivity via the dopamine system, and that changing dopaminergic tone has a greater influence on behavior while subjects are learning task contingencies. Dopamine agonist therapy may therefore be potentiating risk-taking by amplifying the behavioral impact of gambling-related cues on novel behavior. Here, we show that ropinirole treatment in male rats transiently increased motor impulsivity but robustly and progressively increased choice of the high-risk/high-reward options when administered during acquisition of the cued but not uncued rGT. Early in training, ropinirole increased win-stay behavior after large unlikely wins on the cued rGT, indicative of enhanced model-free learning, which mediated the drug's effect on later risk preference. Ex vivo cFos imaging showed that both chronic ropinirole and the addition of win-paired cues suppressed the activity of dopaminergic midbrain neurons. The ratio of midbrain:prefrontal cFos+ neurons was lower in animals with suboptimal choice patterns and tended to predict risk preference across all rats. Network analyses further suggested that ropinirole induced decoupling of the dopaminergic cells of the VTA and nucleus accumbens but only when win-paired cues were present. Frontostriatal activity uninformed by the endogenous dopaminergic teaching signal therefore appeared to perpetuate risky choice, and ropinirole exaggerated this disconnect in synergy with reward-paired cues.SIGNIFICANCE STATEMENT D2/3 receptor agonists, used to treat Parkinson's disease, can cause gambling disorder through an unknown mechanism. Ropinirole increased risky decision-making in rats, but only when wins were paired with casino-inspired sounds and lights. This was mediated by increased win-stay behavior after large unlikely wins early in learning, indicating enhanced model-free learning. cFos imaging showed that ropinirole suppressed activity of midbrain dopamine neurons, an effect that was mimicked by the addition of win-paired cues. The degree of risky choice rats exhibited was uniquely predicted by the ratio of midbrain dopamine:PFC activity. Depriving the PFC of the endogenous dopaminergic teaching signal may therefore drive risky decision-making on-task, and ropinirole acts synergistically with win-paired cues to amplify this.

Circuit and cell-specific contributions to decision making involving risk of explicit punishment in male and female rats

Decision making is a complex cognitive process that recruits a distributed network of brain regions, including the basolateral amygdala (BLA) and nucleus accumbens shell (NAcSh). Recent work suggests that communication between these structures, as well as activity of cells expressing dopamine D2 receptors (D2R) in the NAcSh, are necessary for some forms of decision making; however, the contributions of this circuit and cell population during decision making under risk of punishment are unknown. The current experiments addressed this question using circuit- and cell type-specific optogenetic approaches in rats during a decision-making task involving risk of punishment. In Experiment 1, Long-Evans rats received intra-BLA injections of halorhodopsin or mCherry (control) and in Experiment 2, D2-Cre transgenic rats received intra-NAcSh injections of Cre-dependent halorhodopsin or mCherry. Optic fibers were implanted in the NAcSh in both experiments. Following training in the decision-making task, BLA→NAcSh or D2R-expressing neurons were optogenetically inhibited during different phases of the decision process. Inhibition of the BLA→NAcSh during deliberation (the time between trial initiation and choice) increased choice of the large, risky reward (increased risk taking). Similarly, inhibition during delivery of the large, punished reward increased risk taking, but only in males. Inhibition of D2R-expressing neurons in the NAcSh during deliberation increased risk taking. In contrast, inhibition of these neurons during delivery of the small, safe reward decreased risk taking. These findings extend our knowledge of the neural dynamics of risk taking, revealing sex-dependent circuit recruitment and dissociable activity of selective cell populations during decision making.

Critical review of RDoC approaches to the study of motivation with animal models: effort valuation/willingness to work

The NIMH research domain criteria (RDoC) approach was instigated to refocus mental health research on the neural circuits that mediate psychological functions, with the idea that this would foster an understanding of the neural basis of specific psychiatric dysfunctions (i.e. 'symptoms and circuits') and ultimately facilitate treatment. As a general idea, this attempt to go beyond traditional diagnostic categories and focus on neural circuit dysfunctions related to specific symptoms spanning multiple disorders has many advantages. For example, motivational dysfunctions are present in multiple disorders, including depression, schizophrenia, Parkinson's disease, and other conditions. A critical aspect of motivation is effort valuation/willingness to work, and several clinical studies have identified alterations in effort-based decision making in various patient groups. In parallel, formal animal models focusing on the exertion of effort and effort-based decision making have been developed. This paper reviews the literature on models of effort-based motivational function in the context of a discussion of the RDoC approach, with an emphasis on the dissociable nature of distinct aspects of motivation. For example, conditions associated with depression and schizophrenia blunt the selection of high-effort activities as measured by several tasks in animal models (e.g. lever pressing, barrier climbing, wheel running). Nevertheless, these manipulations also leave fundamental aspects of hedonic reactivity, food motivation, and reinforcement intact. This pattern of effects demonstrates that the general emphasis of the RDoC on the specificity of the neural circuits mediating behavioral pathologies, and the dissociative nature of these dysfunctions, is a valid concept. Nevertheless, the specific placement of effort-related processes as simply a 'sub-construct' of 'reward processing' is empirically and conceptually problematic. Thus, while the RDoC is an excellent general framework for new ways to approach research and therapeutics, it still needs further refinement.

Exploring decision-making strategies in the Iowa gambling task and rat gambling task

Decision-making requires that individuals perceive the probabilities and risks associated with different options. Experimental human and animal laboratory testing provide complimentary insights on the psychobiological underpinnings of decision-making. The Iowa gambling task (IGT) is a widely used instrument that assesses decision-making under uncertainty and risk. In the task participants are faced with a choice conflict between cards with varying monetary reinforcer/loss contingencies. The rat gambling task (rGT) is a pre-clinical version using palatable reinforcers as wins and timeouts mimicking losses. However, interspecies studies elaborating on human and rat behavior in these tasks are lacking. This study explores decision-making strategies among young adults (N = 270) performing a computerized version of the IGT, and adult outbred male Lister Hooded rats (N = 72) performing the rGT. Both group and individual data were explored by normative scoring approaches and subgroup formations based on individual choices were investigated. Overall results showed that most humans and rats learned to favor the advantageous choices, but to a widely different extent. Human performance was characterized by both exploration and learning as the task progressed, while rats showed relatively consistent pronounced preferences for the advantageous choices throughout the task. Nevertheless, humans and rats showed similar variability in individual choice preferences during end performance. Procedural differences impacting on the performance in both tasks and their potential to study different aspects of decision-making are discussed. This is a first attempt to increase the understanding of similarities and differences regarding decision-making processes in the IGT and rGT from an explorative perspective.

Effort-based decision making in response to high-dose androgens: role of dopamine receptors

INTRODUCTION

Anabolic-androgenic steroids (AAS) are performance-enhancing drugs used by both world-class and rank-and-file athletes. AAS abuse has been linked with risky decision-making, ranging from drunk driving to abusing multiple drugs. Our lab uses operant behavior in rats to test the effects of AAS (testosterone) on decision making. In our previous study, testosterone caused rats to work harder for food reward during an effort discounting (ED) task. ED is sensitive to dopamine in the nucleus accumbens, and AAS alter accumbens dopamine receptor expression. Accordingly, we determined if testosterone increases response to dopamine receptor antagonists during ED.

METHODS

Rats were treated chronically with high-dose testosterone (7.5 mg/kg; n = 9) or vehicle (n = 9). We measured baseline preference for the large reward in an ED task, where rats choose between a small easy reward (one lever press for one sugar pellet) and a large difficult reward (2, 5, 10, or 15 presses for three pellets). Preference for the large reward was measured after administration of D1-like (SCH23390, 0.01 mg/kg) or D2-like (eticlopride, 0.06 mg/kg) receptor antagonists.

RESULTS

At baseline, testosterone- and vehicle-treated rats showed similar preference for the large reward lever (FR5, testosterone: 68.6 ± 9.7% and vehicle: 85.7 ± 2.5%). SCH23390 reduced large reward preference significantly in both groups (FR5, testosterone: 41.3 ± 9.2%; vehicle: 49.1 ± 8.2%; F(1,16) = 17.7; P < 0.05). Eticlopride decreased large reward preference in both groups, but more strongly in testosterone-treated rats (FR5: testosterone: 37.0 ± 9.7%; vehicle: 56.3 ± 7.8%; F(1,16) = 35.3; P < 0.05).

CONCLUSION

Testosterone increases response to dopamine D2-like receptor blockade, and this contributes to previously observed changes in decision-making behaviors.

Vigor, Effort-Related Aspects of Motivation and Anhedonia

In this chapter we provide an overview of the pharmacological and circuit mechanisms that determine the willingness to expend effort in pursuit of rewards. A particular focus will be on the role of the mesolimbic dopamine system, as well the contributing roles of limbic and cortical brains areas involved in the evaluation, selection, and invigoration of goal-directed actions. We begin with a review of preclinical studies, which have provided key insights into the brain systems that are necessary and sufficient for effort-based decision-making and have characterized novel compounds that enhance selection of high-effort activities. Next, we summarize translational studies identifying and expanding this circuitry in humans. Finally, we discuss the relevance of this work for understanding common motivational impairments as part of the broader anhedonia symptom domain associated with mental illness, and the identification of new treatment targets within this circuitry to improve motivation and effort-expenditure.

Regulation of sex differences in risk-based decision making by gonadal hormones: Insights from rodent models

Men and women differ in their ability to evaluate options that vary in their rewards and the risks that are associated with these outcomes. Most studies have shown that women are more risk averse than men and that gonadal hormones significantly contribute to this sex difference. Gonadal hormones can influence risk-based decision making (i.e., risk taking) by modulating the neurobiological substrates underlying this cognitive process. Indeed, estradiol, progesterone and testosterone modulate activity in the prefrontal cortex, amygdala and nucleus accumbens associated with reward and risk-related information. The use of animal models of decision making has advanced our understanding of the intersection between the behavioral, neural and hormonal mechanisms underlying sex differences in risk taking. This review will outline the current state of this literature, identify the current gaps in knowledge and suggest the neurobiological mechanisms by which hormones regulate risky decision making. Collectively, this knowledge can be used to understand the potential consequences of significant hormonal changes, whether endogenously or exogenously induced, on risk-based decision making as well as the neuroendocrinological basis of neuropsychiatric diseases that are characterized by impaired risk taking, such as substance use disorder and schizophrenia.

ErbB4 in parvalbumin-positive interneurons mediates proactive interference in olfactory associative reversal learning

Consolidated memories influence later learning and cognitive processes when new information is overlapped with previous events. To reveal which cellular and molecular factors are associated with this proactive interference, we challenged mice with odor-reward associative learning followed by a reversal-learning task. The results showed that genetical ablation of ErbB4 in parvalbumin (PV)-positive interneurons improved performance in reversal-learning phase, with no alteration in learning phase, supporting that PV interneuron ErbB4 is required for proactive interference. Mechanistically, olfactory learning promoted PV interneuron excitatory synaptic plasticity and direct binding of ErbB4 with presynaptic Neurexin1β (NRXN1β) and postsynaptic scaffold PSD-95 in the prefrontal cortex. Interrupting ErbB4-NRXN1β interaction impaired network activity-driven excitatory inputs and excitatory synaptic transmission onto PV interneurons. Neuronal activity-induced ErbB4-PSD-95 association facilitated transsynaptic binding of ErbB4-NRXN1β and excitatory synapse formation in ErbB4-positive interneurons. Furthermore, ErbB4-NRXN1β binding was responsible for the activity-regulated activation of ErbB4 and extracellular signal-regulated kinase (ERK) 1/2 in PV interneurons, as well as synaptic plasticity-related expression of brain-derived neurotrophic factor (BDNF). Correlatedly, blocking ErbB4-NRXN1β coupling in the medial prefrontal cortex of adult mice facilitated reversal learning of an olfactory associative task. These findings provide novel insight into the physiological role of PV interneuron ErbB4 signaling in cognitive processes and reveal an associative learning-related transsynaptic NRXN1β-ErbB4-PSD-95 complex that affects the ERK1/2-BDNF pathway and underlies local inhibitory circuit plasticity and proactive interference.

Cannabinoid receptor type 1 antagonists alter aspects of risk/reward decision making independent of toluene-mediated effects

Drugs of abuse including cannabis and inhalants impair risk/reward decision making. Cannabis use is often concurrent with inhalant intoxication; yet, preclinical studies investigating the role of endocannabinoids in inhalant misuse are limited. To address this gap in the literature, we used the well-validated probabilistic discounting task to assess risk/reward decision making in rodents following combinations of toluene vapor (a common inhalant) and manipulations of cannabinoid receptor type 1 (CB1R) signaling. As reported previously, acute exposure to toluene vapor disrupted behavioral flexibility during probabilistic discounting. Systemic administration of the CB1R inverse agonist AM281 did not prevent toluene-induced alterations in risky choices, but did independently reduce win-stay behavior, increase choice latency, and increase omissions. Toluene-induced deficits in probabilistic discounting are thought to involve impaired medial prefrontal cortex (mPFC) activity. As we previously reported that some of toluene's inhibitory effects on glutamatergic signaling in the mPFC are endocannabinoid-dependent, we tested the hypothesis that mPFC CB1R activity mediates toluene-induced deficits in discounting. However, bilateral injection of the CB1R inverse agonist AM251 prior to toluene vapor exposure had no effect on toluene-induced changes in risk behavior. In a final set of experiments, we injected the CB1R inverse agonist AM251 (5 and 50 ng), the CB1R agonist WIN55,212-2 (50 ng and 500 ng), or vehicle into the mPFC prior to testing. While mPFC CB1R stimulation did not affect any of the measures tested, the CB1R inverse agonist caused a dose-dependent reduction in win-stay behavior without altering any other measures. Together, these studies indicate that toluene-induced deficits in probabilistic discounting are largely distinct from CB1R-dependent effects that include decreased effectiveness of positive reinforcement (mPFC CB1Rs), decision making speed, and task engagement (non-mPFC CB1Rs).

Probabilistic Reinforcement Learning and Anhedonia

Despite the prominence of anhedonic symptoms associated with diverse neuropsychiatric conditions, there are currently no approved therapeutics designed to attenuate the loss of responsivity to previously rewarding stimuli. However, the search for improved treatment options for anhedonia has been reinvigorated by a recent reconceptualization of the very construct of anhedonia, including within the Research Domain Criteria (RDoC) initiative. This chapter will focus on the RDoC Positive Valence Systems construct of reward learning generally and sub-construct of probabilistic reinforcement learning specifically. The general framework emphasizes objective measurement of a subject's responsivity to reward via reinforcement learning under asymmetrical probabilistic contingencies as a means to quantify reward learning. Indeed, blunted reward responsiveness and reward learning are central features of anhedonia and have been repeatedly described in major depression. Moreover, these probabilistic reinforcement techniques can also reveal neurobiological mechanisms to aid development of innovative treatment approaches. In this chapter, we describe how investigating reward learning can improve our understanding of anhedonia via the four RDoC-recommended tasks that have been used to probe sensitivity to probabilistic reinforcement contingencies and how such task performance is disrupted in various neuropsychiatric conditions. We also illustrate how reverse translational approaches of probabilistic reinforcement assays in laboratory animals can inform understanding of pharmacological and physiological mechanisms. Next, we briefly summarize the neurobiology of probabilistic reinforcement learning, with a focus on the prefrontal cortex, anterior cingulate cortex, striatum, and amygdala. Finally, we discuss treatment implications and future directions in this burgeoning area.

Foraging with the frontal cortex: A cross-species evaluation of reward-guided behavior

Efficient foraging is essential to survival and depends on frontal cortex in mammals. Because of its role in psychiatric disorders, frontal cortex and its contributions to reward procurement have been studied extensively in both rodents and non-human primates. How frontal cortex of these animal models compares is a source of intense debate. Here we argue that translating findings from rodents to non-human primates requires an appreciation of both the niche in which each animal forages as well as the similarities in frontal cortex anatomy and function. Consequently, we highlight similarities and differences in behavior and anatomy, before focusing on points of convergence in how parts of frontal cortex contribute to distinct aspects of foraging in rats and macaques, more specifically. In doing so, our aim is to emphasize where translation of frontal cortex function between species is clearer, where there is divergence, and where future work should focus. We finish by highlighting aspects of foraging for which have received less attention but we believe are critical to uncovering how frontal cortex promotes survival in each species.

The effects of the dopamine D2/3 agonist quinpirole on incentive value and palatability-based choice in a rodent model of attention-deficit/hyperactivity disorder

RATIONALE

Palatability and incentive value influence animal food choice. Dopamine D2/3 receptor signaling may mediate the effects of palatability and incentive value on choice. Dopamine signaling is disrupted in attention-deficit hyperactivity disorder (ADHD). Investigating behavioral choice processes under D2/3 receptor agonists will help elucidate behavioral and pharmacological correlates of ADHD.

OBJECTIVES

To determine (1) how changes in incentive value affects choice of actions for outcomes that differ in palatability; (2) the effects of the D2/3 agonist quinpirole on choice based on palatability and incentive value; (3) how choice differs in spontaneously hypertensive rats (SHR; ADHD model) compared with control strains.

METHODS

Rats responded instrumentally for two food outcomes (chocolate and grain pellets) that differed in palatability. Following specific satiety of one outcome, rats underwent a choice test. Prior to the choice test, rats were given intra-peritoneal quinpirole (0.01-0.1 mg/kg) body weight. These manipulations were conducted in three strains of rats: SHR rats; the normotensive Wistar-Kyoto (WKY) controls; and Wistar outbred (WIS) controls.

RESULTS

All rat strains responded more vigorously for chocolate pellets compared with grain pellets. Quinpirole reduced the effects of palatability and dose-dependently increased the effects of incentive value on choice. SHR rats were the least influenced by incentive value, whereas WKY rats were the least influenced by palatability.

CONCLUSIONS

These results show that D2/3 signaling modulates choice based on palatability and incentive value. Disruption of this process in SHR rats may mirror motivational impairments observed in ADHD.

Hippocampal neurogenesis promotes preference for future rewards

Adult hippocampal neurogenesis has been implicated in a number of disorders where reward processing is disrupted but whether new neurons regulate specific aspects of reward-related decision making remains unclear. Given the role of the hippocampus in future-oriented cognition, here we tested whether adult neurogenesis regulates preference for future, advantageous rewards in a delay discounting paradigm for rats. Indeed, blocking neurogenesis caused a profound aversion for delayed rewards, and biased choice behavior toward immediately available, but smaller, rewards. Consistent with a role for the ventral hippocampus in impulsive decision making and future-thinking, neurogenesis-deficient animals displayed reduced activity in the ventral hippocampus. In intact animals, delay-based decision making restructured dendrites and spines in adult-born neurons and specifically activated adult-born neurons in the ventral dentate gyrus, relative to dorsal activation in rats that chose between immediately-available rewards. Putative developmentally-born cells, located in the superficial granule cell layer, did not display task-specific activity. These findings identify a novel and specific role for neurogenesis in decisions about future rewards, thereby implicating newborn neurons in disorders where short-sighted gains are preferred at the expense of long-term health.

A sex-dependent role for the prelimbic cortex in impulsive action both before and following early cocaine abstinence

Although impulsive action is strongly associated with addiction, the neural underpinnings of this relationship and how they are influenced by sex have not been well characterized. Here, we used a titrating reaction time task to assess differences in impulsive action in male and female Long Evans rats both before and after short (4-6 days) or long (25-27 days) abstinence from 2 weeks of cocaine or water/saline self-administration (6 h daily access). Neural activity in the prelimbic cortex (PrL) and nucleus accumbens (NAc) core was assessed at each time point. We found that a history of cocaine self-administration increased impulsivity in all rats following short, but not long, abstinence. Furthermore, male rats with an increased ratio of excited to inhibited neurons in the PrL at the start of each trial in the task exhibited higher impulsivity in the naïve state (before self-administration). Following short abstinence from cocaine, PrL activity in males became more inhibited, and this change in activity predicted the shift in impulsivity. However, PrL activity did not track impulsivity in female rats. Additionally, although the NAc core tracked several aspects of behavior in the task, it did not track impulsivity in either sex. Together, these findings demonstrate a sex-dependent role for the PrL in impulsivity both before and after a history of cocaine.

The role of the medial prefrontal cortex in cognition, ageing and dementia

Humans require a plethora of higher cognitive skills to perform executive functions, such as reasoning, planning, language and social interactions, which are regulated predominantly by the prefrontal cortex. The prefrontal cortex comprises the lateral, medial and orbitofrontal regions. In higher primates, the lateral prefrontal cortex is further separated into the respective dorsal and ventral subregions. However, all these regions have variably been implicated in several fronto-subcortical circuits. Dysfunction of these circuits has been highlighted in vascular and other neurocognitive disorders. Recent advances suggest the medial prefrontal cortex plays an important regulatory role in numerous cognitive functions, including attention, inhibitory control, habit formation and working, spatial or long-term memory. The medial prefrontal cortex appears highly interconnected with subcortical regions (thalamus, amygdala and hippocampus) and exerts top-down executive control over various cognitive domains and stimuli. Much of our knowledge comes from rodent models using precise lesions and electrophysiology readouts from specific medial prefrontal cortex locations. Although, anatomical disparities of the rodent medial prefrontal cortex compared to the primate homologue are apparent, current rodent models have effectively implicated the medial prefrontal cortex as a neural substrate of cognitive decline within ageing and dementia. Human brain connectivity-based neuroimaging has demonstrated that large-scale medial prefrontal cortex networks, such as the default mode network, are equally important for cognition. However, there is little consensus on how medial prefrontal cortex functional connectivity specifically changes during brain pathological states. In context with previous work in rodents and non-human primates, we attempt to convey a consensus on the current understanding of the role of predominantly the medial prefrontal cortex and its functional connectivity measured by resting-state functional MRI in ageing associated disorders, including prodromal dementia states, Alzheimer's disease, post-ischaemic stroke, Parkinsonism and frontotemporal dementia. Previous cross-sectional studies suggest that medial prefrontal cortex functional connectivity abnormalities are consistently found in the default mode network across both ageing and neurocognitive disorders such as Alzheimer's disease and vascular cognitive impairment. Distinct disease-specific patterns of medial prefrontal cortex functional connectivity alterations within specific large-scale networks appear to consistently feature in the default mode network, whilst detrimental connectivity alterations are associated with cognitive impairments independently from structural pathological aberrations, such as grey matter atrophy. These disease-specific patterns of medial prefrontal cortex functional connectivity also precede structural pathological changes and may be driven by ageing-related vascular mechanisms. The default mode network supports utility as a potential biomarker and therapeutic target for dementia-associated conditions. Yet, these associations still require validation in longitudinal studies using larger sample sizes.

Reward/Punishment-Based Decision Making in Rodents

Deficits in decision making are at the heart of many psychiatric diseases, such as substance abuse disorders and attention deficit hyperactivity disorder. Consequently, rodent models of decision making are germane to understanding the neural mechanisms underlying adaptive choice behavior and how such mechanisms can become compromised in pathological conditions. A critical factor that must be integrated with reward value to ensure optimal decision making is the occurrence of consequences, which can differ based on probability (risk of punishment) and temporal contiguity (delayed punishment). This article will focus on two models of decision making that involve explicit punishment, both of which recapitulate different aspects of consequences during human decision making. We will discuss each behavioral protocol, the parameters to consider when designing an experiment, and finally how such animal models can be utilized in studies of psychiatric disease. © 2020 Wiley Periodicals LLC. Basic Protocol 1: Behavioral training Support Protocol: Equipment testing Alternate Protocol: Reward discrimination Basic Protocol 2: Risky decision-making task (RDT) Basic Protocol 3: Delayed punishment decision-making task (DPDT).

Sex differences in effort-related decision-making: role of dopamine D2 receptor antagonism

RATIONALE

Depressed individuals demonstrate debilitating symptoms, including depressed mood, anhedonia, and effort-related deficits. Effort-related decision-making can be measured through providing subjects with a choice between high effort/reward and low effort/reward options, which is a dopamine (DA)-dependent behavior. While previous research has shown sex differences in depression rates, this has not been examined within operant-based effort-related decision-making tasks nor has DA been shown to underlie this behavior in female rats.

OBJECTIVES

The current study investigated sex differences in an effort-related decision-making task prior to and following administration of the DA D2 receptor antagonist haloperidol (HAL).

METHODS

Adult rats were food restricted or fed freely and trained in an effort-related progressive ratio choice task. After stable responding, HAL was administered acutely (0.05-0.2 mg/kg) prior to testing.

RESULTS

Results indicate a significant effect of sex on training variables, with males having a greater number of lever presses, higher ratios, and longer active lever times. Pretreatment with HAL significantly reduced the same measures in both sexes for the high-valued reward, while increasing chow consumption in the food restricted males. Food restricted rats showed a greater number of total lever presses and achieved higher ratios; however, the effect in male food restricted rats was greatest.

CONCLUSIONS

These data suggest that, although there are sex differences in training, HAL decreases behavior across sexes, demonstrating that the D2 mechanism is similar in both sexes. These findings provide a better understanding of motivational dysfunction in both sexes and potential treatment targets for depression.

Dopamine receptors regulate preference between high-effort and high-risk rewards

RATIONALE

Optimal decision-making necessitates evaluation of multiple rewards that are each offset by distinct costs, such as high effort requirement or high risk of failure. The neurotransmitter dopamine is fundamental toward these cost-benefit analyses, and D1-like and D2-like dopamine receptors differently modulate the reward-discounting effects of both effort and risk. However, measuring the role of dopamine in regulating decision-making between options associated with distinct costs exceeds the scope of traditional rodent economic decision-making paradigms.

OBJECTIVES

We developed the effort vs probability economic conflict task (EvP) to model multimodal economic decision-making in rats. This task measures choice between two rewards of uniform magnitude associated with either a high effort requirement or risk of reward omission. We then tested the modulatory effects of systemic cocaine and D1/D2 blockade or activation on the preference between high-effort and high-risk alternatives.

METHODS

In the EvP, two reinforcers of equal magnitude are associated with either (1) an effort requirement that increases throughout the session (1, 5, 10, and 20 lever presses), or (2) a low probability of reward receipt (25% of probabilistic choices). Critically, the reinforcer for each choice is comparable (one pellet), which eliminates the influence of magnitude discrimination on the decision-making process. After establishing the task, the dopamine transporter blocker cocaine and D1/D2 antagonists and agonists were administered prior to EvP performance.

RESULTS

Preference shifted away from either effortful or probabilistic choice when either option became more costly, and this preference was highly variable between subjects and stable over time. Cocaine, D1 activation, and D2 blockade produced limited, dose-dependent shifts in choice preference contingent on high or low effort conditions. In contrast, D2 activation across multiple doses evoked a robust shift from effortful to risky choice that was evident even when clearly disadvantageous.

CONCLUSIONS

The EvP clearly demonstrates that rats can evaluate distinct effortful or risky costs associated with rewards of comparable magnitude, and shift preference away from either option with increasing cost. This preference is more tightly linked to D2 than D1 receptor manipulation, suggesting D2-like receptors as a possible therapeutic target for maladaptive biases toward risk-taking over effort.

The ventral hippocampus is necessary for cue-elicited, but not outcome driven approach-avoidance conflict decisions: a novel operant choice decision-making task

Approach-avoidance conflict is induced when an organism encounters a stimulus that carries both positive and negative attributes. Accumulating evidence implicates the ventral hippocampus (VH) in the detection and resolution of approach-avoidance conflict, largely on the basis of maze-based tasks assaying innate and conditioned responses to situations of conflict. However, its role in discrete trial approach-avoidance decision-making has yet to be elucidated. In this study, we designed a novel cued operant conflict decision-making task in which rats were required to choose and respond for a low reward option or high reward option paired with varying shock intensities on a differential reinforcement of low rates of responding schedule. Post training, the VH was chemogenetically inhibited while animals performed the task with the usual outcomes delivered, and with the presentation of cues associated with the reward vs. conflict options only (extinction condition). We found that VH inhibition led to an avoidance of the conflict option and longer latency to choose this option when decision-making was being made on the basis of cues alone with no outcomes. Consistent with these findings, VH-inhibited animals spent more time in the central component of the elevated plus maze (EPM), indicating a potential deficit in decision-making under innate forms of approach-avoidance conflict. Taken together, these findings implicate the VH in cue-driven approach-avoidance decisions in the face of motivational conflict.

Sex differences in lever pressing and running wheel tasks of effort-based choice behavior in rats: Suppression of high effort activity by the serotonin transport inhibitor fluoxetine

Selective serotonin transport (SERT) inhibitors such as fluoxetine are the most commonly prescribed treatments for depression. Although efficacious for many symptoms of depression, motivational impairments such as psychomotor retardation, anergia, fatigue and amotivation are relatively resistant to treatment with SERT inhibitors, and these drugs have been reported to exacerbate motivational deficits in some people. In order to study motivational dysfunctions in animal models, procedures have been developed to measure effort-related decision making, which offer animals a choice between high effort actions leading to highly valued reinforcers, or low effort/low reward options. In the present studies, male and female rats were tested on two different tests of effort-based choice: a fixed ratio 5 (FR5)/chow feeding choice procedure and a running wheel (RW)/chow feeding choice task. The baseline pattern of choice differed across tasks for males and females, with males pressing the lever more than females on the operant task, and females running more than males on the RW task. Administration of the SERT inhibitor and antidepressant fluoxetine suppressed the higher effort activity on each task (lever pressing and wheel running) in both males and females. The serotonin receptor mediating the suppressive effects of fluoxetine is uncertain, because serotonin antagonists with different patterns of receptor selectivity failed to reverse the effects of fluoxetine. Nevertheless, these studies uncovered important sex differences, and demonstrated that the suppressive effects of fluoxetine on high effort activities are not limited to tasks involving food reinforced behavior or appetite suppressive effects. It is possible that this line of research will contribute to an understanding of the neurochemical factors regulating selection of voluntary physical activity vs. sedentary behaviors, which could be relevant for understanding the role of physical activity in psychiatric disorders.

Differential effects of glutamate N-methyl-D-aspartate receptor antagonists on risky choice as assessed in the risky decision task

RATIONALE

Risky choice can be measured using the risky decision task (RDT). In the RDT, animals choose between a large, risky option that is paired with probabilistic foot shock and a small, safe option that is never paired with shock. To date, studies examining the neurochemical basis of decision-making in the RDT have focused primarily on the dopaminergic system but have not focused on the glutamatergic system, which has been implicated in risky decision-making.

OBJECTIVES

Because glutamate is known to play a critical role in decision-making, we wanted to determine the contribution of the glutamatergic system to performance in the RDT.

METHODS

In the experiment, 32 rats (16 male; 16 female) were tested in the RDT. The probability of receiving a foot shock increased across the session (ascending schedule) for half of the rats but decreased across the session (descending schedule) for half of the rats. Following training, rats received injections of the N-methyl-D-aspartate (NMDA) receptor competitive antagonist CGS 19755 (0, 1.0, 2.5, 5.0 mg/kg; s.c.) and the GluN2B-selective antagonist Ro 63-1908 (0, 0.1, 0.3, 1.0 mg/kg; s.c.).

RESULTS

CGS 19755 (2.5 and 5.0 mg/kg) increased risky choice in males and females trained on the ascending schedule. Ro 63-1908 (1.0 mg/kg) decreased risky choice, but only in male rats trained on the ascending schedule.

CONCLUSIONS

Although NMDA receptor antagonists differentially alter risky choice in the RDT, the current results show that NMDA receptors are an important mediator of decision-making involving probabilistic delivery of positive punishment.

Dorsomedial striatal contributions to different forms of risk/reward decision making

Optimal decision making involving reward uncertainty is integral to adaptive goal-directed behavior. In some instances, these decisions are guided by internal representations of reward history, whereas in other situations, external cues inform a decision maker about how likely certain actions are to yield reward. Different regions of the frontal lobe form distributed networks with striatal and amygdalar regions that facilitate different types of risk/reward decision making. The dorsal medial striatum (DMS) is one key output region of the prefrontal cortex, yet there have been few preclinical studies investigating the involvement of the DMS in different forms of risk/reward decision making. The present study addressed this issue, wherein separate groups of male rats were trained on one of two tasks where they chose between a small/certain or a large/risky reward. In a probabilistic discounting task, reward probabilities changed systematically over blocks of trials (100-6.25% or 6.25-100%), requiring rats to use internal representations of reward history to guide choice. Cue-guided decision-making was assessed with a "Blackjack" task, where different auditory cues indicated the odds associated with the large/risky option (50 or 12.5%). Inactivation of the DMS with GABA agonists impaired adjustments in choice biases during probabilistic discounting, resulting in either increases or decreases in risky choice as the probabilities associated with the large/risky reward decreased or increased over a session. In comparison, DMS inactivation increased risky choices on poor-odds trials on the Blackjack task, which was associated with a reduced impact that non-rewarded choices had on subsequent choices. DMS inactivation also impaired performance of an auditory conditional discrimination. These findings highlight a previously uncharacterized role for the DMS in facilitating flexible action selection during multiple forms of risk/reward decision making.

The role of opioidergic system in modulating cost/benefit decision-making in alcohol-preferring AA rats and Wistar rats

Research has highlighted the association of a positive family history of alcoholism with a positive treatment response to opioid antagonists in those with a gambling disorder. However, the role of the opioidergic system in gambling behavior is not well understood, and preclinical studies are needed to clarify this. In this study, Alko Alcohol (AA) and Wistar rats went through operant lever pressing training where the task was to choose the more profitable of two options. Different sized sucrose rewards guided the lever choices, and the probability of gaining rewards changed slowly to a level where choosing the smaller reward was the most profitable option. After training, rats were administered subcutaneously with opioid agonist morphine or opioid antagonist naltrexone to study the impact of opioidergic mechanisms on cost/benefit decisions. No difference was found in the decision-making between AA rats or Wistar rats after the morphine administration, but control data revealed a minor decision enhancing effect in AA rats. Naltrexone had no impact on the decisions in AA rats but promoted unprofitable decisions in Wistar rats. Supporting behavioral data showed that in both rat strains morphine increased, and naltrexone decreased, sucrose consumption. Naltrexone also increased the time to accomplish the operant task. The results suggest that opioid agonists could improve decision-making in cost-benefit settings in rats that are naturally prone to high alcohol drinking. The naltrexone results are ambiguous but may partly explain why opioid antagonists lack a positive pharmacotherapeutic effect in some subgroups of gamblers.

Using pharmacological manipulations to study the role of dopamine in human reward functioning: A review of studies in healthy adults

Dopamine (DA) plays a key role in reward processing and is implicated in psychological disorders such as depression, substance use, and schizophrenia. The role of DA in reward processing is an area of highly active research. One approach to this question is drug challenge studies with drugs known to alter DA function. These studies provide good experimental control and can be performed in parallel in laboratory animals and humans. This review aimed to summarize results of studies using pharmacological manipulations of DA in healthy adults. 'Reward' is a complex process, so we separated 'phases' of reward, including anticipation, evaluation of cost and benefits of upcoming reward, execution of actions to obtain reward, pleasure in response to receiving a reward, and reward learning. Results indicated that i) DAergic drugs have different effects on different phases of reward; ii) the relationship between DA and reward functioning appears unlikely to be linear; iii) our ability to detect the effects of DAergic drugs varies depending on whether subjective, behavioral, imaging measures are used.

Behavioral and dopamine transporter binding properties of the modafinil analog (S, S)-CE-158: reversal of the motivational effects of tetrabenazine and enhancement of progressive ratio responding

RATIONALE

Atypical dopamine (DA) transport blockers such as modafinil and its analogs may be useful for treating motivational symptoms of depression and other disorders. Previous research has shown that the DA depleting agent tetrabenazine can reliably induce motivational deficits in rats, as evidenced by a shift towards a low-effort bias in effort-based choice tasks. This is consistent with human studies showing that people with major depression show a bias towards low-effort activities.

OBJECTIVES

Recent studies demonstrated that the atypical DA transport (DAT) inhibitor (S)-CE-123 reversed tetrabenazine-induced motivational deficits, increased progressive ratio (PROG) lever pressing, and increased extracellular DA in the nucleus accumbens. In the present studies, a recently synthesized modafinil analog, (S, S)-CE-158, was assessed in a series of neurochemical and behavioral studies in rats.

RESULTS

(S, S)-CE-158 demonstrated the ability to reverse the effort-related effects of tetrabenazine and increase selection of high-effort PROG lever pressing in rats tested on PROG/chow feeding choice task. (S, S)-CE-158 showed a high selectivity for inhibiting DAT compared with other monoamine transporters, and systemic administration of (S, S)-CE-158 increased extracellular DA in the nucleus accumbens during the behaviorally active time course, which is consistent with the effects of (S)-CE-123 and other DAT inhibitors that enhance high-effort responding.

CONCLUSIONS

These studies provide an initial neurochemical characterization of a novel atypical DAT inhibitor, and demonstrate that this compound is active in models of effort-related choice. This research could contribute to the development of novel compounds for the treatment of motivational dysfunctions in humans.

Dissociating the effects of dopamine D2 receptors on effort-based versus value-based decision making using a novel behavioral approach

Cost-benefit decision making is essential for organisms to adapt to their ever-changing environment. Most studies of cost-benefit decision making involve choice conditions in which effort and value are varied simultaneously. This prevents identification of the aspects of cost-benefit decision making that are affected by experimental manipulations. We developed operant assays to isolate the individual impacts of effort and value manipulations on cost-benefit decision making. In the concurrent effort choice (CEC) task, mice choose between exerting two distinct types of effort: the number of responses and the duration of a response, to earn the same reward. By parametrically varying response cost, psychometric functions are obtained that reflect how the two types of effort scale against one another. Direct manipulations of effort shift the functions. Because reward value is held constant in this task, differences in scaling of the two response types must be related to the effort manipulations. In the concurrent value choice (CVC) task, mice make the same type of response to earn rewards of different value (e.g., pellets vs. sucrose solutions). Here the effort required to earn one reward type is parametrically varied to obtain the psychometric function that scales the value of the two rewards into the number of responses subjects will pay to earn one reward over the other. Direct value manipulations shift these functions. We tested the effect of the dopamine D2 receptor antagonist, haloperidol, on performance in the CEC and CVC assays and found that D2R signaling is important for effort-based, but not value-based decision making. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Hippocampal Interaction With Area 25, but not Area 32, Regulates Marmoset Approach-Avoidance Behavior

Affective disorders are associated with increased sensitivity to negative feedback that influences approach-avoidance decision making. Although neuroimaging studies of these disorders reveal dysregulation in primate cingulate areas 25 and 32 and the anterior hippocampus (aHipp), the causal involvement of these structures and their interaction in the primate brain is unknown. We therefore investigated the effects of localized pharmacological manipulations of areas 25 and 32 and/or the aHipp of the marmoset monkey on performance of an anxiolytic-sensitive instrumental decision-making task in which an approach-avoidance conflict is created by pairing a response with reward and punishment. During control infusions animals avoided punishment, but this bias was reduced by increasing glutamate release within the aHipp or area 32, and inactivation or 5-HT1a antagonism within area 25. Conversely, increasing glutamate release in area 25 enhanced punishment avoidance but, in contrast to previous reports, area 32 and aHipp inactivations had no effect. Simultaneous inactivation or 5-HT1a antagonism within area 25, but not area 32, abolished the reduced punishment avoidance seen after increasing aHipp glutamate. Besides providing causal evidence that these primate areas differentially regulate negative feedback sensitivity, this study links the decision-making deficits in affective disorders to aberrant aHipp-area 25 circuit activity.

Chemogenetic Modulation and Single-Photon Calcium Imaging in Anterior Cingulate Cortex Reveal a Mechanism for Effort-Based Decisions

The ACC is implicated in effort exertion and choices based on effort cost, but it is still unclear how it mediates this cost-benefit evaluation. Here, male rats were trained to exert effort for a high-value reward (sucrose pellets) in a progressive ratio lever-pressing task. Trained rats were then tested in two conditions: a no-choice condition where lever-pressing for sucrose was the only available food option, and a choice condition where a low-value reward (lab chow) was freely available as an alternative to pressing for sucrose. Disruption of ACC, via either chemogenetic inhibition or excitation, reduced lever-pressing in the choice, but not in the no-choice, condition. We next looked for value coding cells in ACC during effortful behavior and reward consumption phases during choice and no-choice conditions. For this, we used in vivo miniaturized fluorescence microscopy to reliably track responses of the same cells and compare how ACC neurons respond during the same effortful behavior where there was a choice versus when there was no-choice. We found that lever-press and sucrose-evoked responses were significantly weaker during choice compared with no-choice sessions, which may have rendered them more susceptible to chemogenetic disruption. Together, findings from our interference experiments and neural recordings suggest that a mechanism by which ACC mediates effortful decisions is in the discrimination of the utility of available options. ACC regulates these choices by providing a stable population code for the relative value of different options.

SIGNIFICANCE STATEMENT The ACC is implicated in effort-based decision-making. Here, we used chemogenetics and in vivo calcium imaging to explore its mechanism. Rats were trained to lever press for a high-value reward and tested in two conditions: a no-choice condition where lever-pressing for the high-value reward was the only option, and a choice condition where a low-value reward was also available. Inhibition or excitation of ACC reduced effort toward the high-value option, but only in the choice condition. Neural responses in ACC were weaker in the choice compared with the no-choice condition. A mechanism by which ACC regulates effortful decisions is in providing a stable population code for the discrimination of the utility of available options.

Adaptive learning under expected and unexpected uncertainty

The outcome of a decision is often uncertain, and outcomes can vary over repeated decisions. Whether decision outcomes should substantially affect behaviour and learning depends on whether they are representative of a typically experienced range of outcomes or signal a change in the reward environment. Successful learning and decision-making therefore require the ability to estimate expected uncertainty (related to the variability of outcomes) and unexpected uncertainty (related to the variability of the environment). Understanding the bases and effects of these two types of uncertainty and the interactions between them - at the computational and the neural level - is crucial for understanding adaptive learning. Here, we examine computational models and experimental findings to distil computational principles and neural mechanisms for adaptive learning under uncertainty.

Basolateral amygdala - nucleus accumbens circuitry regulates optimal cue-guided risk/reward decision making

Maladaptive decision making is a characteristic feature of substance use disorder and pathological gambling. Studies in humans and animals have implicated neural circuits that include the basolateral amygdala (BLA) and nucleus accumbens (NAc) in facilitating risk/reward decision making. However, the preclinical literature has focussed primarily on situations where animals use internally-generated information to adapt to changes in reward likelihood, whereas many real-life situations require the use of external stimuli to facilitate context-appropriate behavior. We recently developed the "Blackjack" task, to measure cued risk/reward decision making requiring rats to chose between Small/Certain and Large/Risky rewards, with auditory cues at the start of each trial explicitly informing that the probability of obtaining a large reward was either good (50%) or poor (12.5%). Here we investigated the contribution of the BLA and its interaction with the NAc in guiding these types of decisions. In well-trained male rats, bilateral inactivation of the BLA induced suboptimal decision making, primarily by reducing risky choice on good-odds trials. In comparison, pharmacological disconnection of the BLA and NAc-shell also induced suboptimal decision making, diverting choice from more preferred option by reducing or increasing risky choice on good vs. poor odds trials respectively. Together, these results suggest that the BLA-NAc circuitry plays a crucial role in integrating information provided by discriminative stimuli. Furthermore, this circuitry may aid in guiding action selection of advantageous options in situations to maximize rewards. Finally, they suggest that perturbations in optimal decision making observed in substance abuse and gambling disorders may be driven in part by dysfunction within this circuitry.

Vulnerable and resilient cognitive performance related to early life stress: The potential mediating role of dopaminergic receptors in the medial prefrontal cortex of adult mice

RATIONALE

Exposure to early life stress (ELS) is known to have pronounced effects on the prefrontal cortex (PFC). However, not all individuals exposed to ELS manifest the same neurobiological and cognitive phenotypes when adults. Dopamine signaling could be a key factor in understanding the effects of stress on PFC-related cognitive function.

OBJECTIVES

We aimed to investigate the differential effects of ELS on cognitive performance of adult mice and the dopaminergic receptors expression in the PFC.

METHODS

BALB/c males were exposed to the maternal separation (MS) procedure and their cognitive performance on the eight-arm radial maze (8-RAM) were assessed during adulthood. For molecular-level assessments, we performed mRNA expression analyses for dopamine receptors-DRD1, DRD2, DRD3-and Hers1 expression in the medial PFC.

RESULTS

While MS produced an overall impairment on 8-RAM, the stressed animals could be divided in two groups based on their performance: those with impaired cognitive performance (vulnerable to maternal separation, V-MS) and those without any impairment (resilient to maternal separation, R-MS). V-MS animals showed increased DRD1 and DRD2 expression in comparison with other groups. Errors on 8-RAM were also positively correlated with DRD1 and DRD2 mRNA expression.

CONCLUSIONS

Our findings suggest a potential role of the dopaminergic system in the programming mechanisms of cognitive vulnerability and resilience related to ELS.

Pharmacological studies of effort-related decision making using mouse touchscreen procedures: effects of dopamine antagonism do not resemble reinforcer devaluation by removal of food restriction

RATIONALE

Effort-based decision-making tasks offer animals choices between preferred reinforcers that require high effort to obtain vs. low effort/low reward options. The neural mechanisms of effort-based choice are widely studied in rats, and evidence indicates that mesolimbic dopamine (DA) and related neural systems play a key role. Fewer studies of effort-based choice have been performed in mice.

OBJECTIVES

The present studies used touchscreen operant procedures (Bussey-Saksida boxes) to assess effort-based choice in mice.

METHODS

CD1 mice were assessed on a concurrent fixed ratio 1 panel pressing/choice procedure. Mice were allowed to choose between rearing to press an elevated panel on the touchscreen for a preferred food (strawberry milkshake) vs. consuming a concurrently available less preferred alternative (high carbohydrate pellets).

RESULTS

The DA D₂ antagonist haloperidol (0.05-0.15 mg/kg IP) produced a dose-related decrease in panel pressing. Intake of food pellets was not reduced by haloperidol, and in fact, there was a significant quadratic trend, indicating a tendency for pellet intake to increase at low/moderate doses. In contrast, reinforcer devaluation by removing food restriction substantially decreased both panel pressing and pellet intake. In free-feeding choice tests, mice strongly preferred milkshake vs. pellets. Haloperidol did not affect food intake or preference.

CONCLUSION

Haloperidol reduced the tendency to work for food, but this reduction was not due to decreases in primary food motivation or preference. Mouse touchscreen procedures demonstrate effects of haloperidol that are similar but not identical to those shown in rats. These rodent studies may be relevant for understanding motivational dysfunctions in humans.

Contributions of anterior cingulate cortex and basolateral amygdala to decision confidence and learning under uncertainty

The subjective sense of certainty, or confidence, in ambiguous sensory cues can alter the interpretation of reward feedback and facilitate learning. We trained rats to report the orientation of ambiguous visual stimuli according to a spatial stimulus-response rule that must be learned. Following choice, rats could wait a self-timed delay for reward or initiate a new trial. Waiting times increase with discrimination accuracy, demonstrating that this measure can be used as a proxy for confidence. Chemogenetic silencing of BLA shortens waiting times overall whereas ACC inhibition renders waiting times insensitive to confidence-modulating attributes of visual stimuli, suggesting contribution of ACC but not BLA to confidence computations. Subsequent reversal learning is enhanced by confidence. Both ACC and BLA inhibition block this enhancement but via differential adjustments in learning strategies and consistent use of learned rules. Altogether, we demonstrate dissociable roles for ACC and BLA in transmitting confidence and learning under uncertainty.