Effects of amphetamine and methylphenidate on delay discounting in rats: interactions with order of delay presentation

Aberrant glutamatergic systems underlying impulsive behaviors: Insights from clinical and preclinical research

Impulsivity is a broad construct that often refers to one of several distinct behaviors and can be measured with self-report questionnaires and behavioral paradigms. Several psychiatric conditions are characterized by one or more forms of impulsive behavior, most notably the impulsive/hyperactive subtype of attention-deficit/hyperactivity disorder (ADHD), mood disorders, and substance use disorders. Monoaminergic neurotransmitters are known to mediate impulsive behaviors and are implicated in various psychiatric conditions. However, growing evidence suggests that glutamate, the major excitatory neurotransmitter of the mammalian brain, regulates important functions that become dysregulated in conditions like ADHD. The purpose of the current review is to discuss clinical and preclinical evidence linking glutamate to separate aspects of impulsivity, specifically motor impulsivity, impulsive choice, and affective impulsivity. Hyperactive glutamatergic activity in the corticostriatal and the cerebro-cerebellar pathways are major determinants of motor impulsivity. Conversely, hypoactive glutamatergic activity in frontal cortical areas and hippocampus and hyperactive glutamatergic activity in anterior cingulate cortex and nucleus accumbens mediate impulsive choice. Affective impulsivity is controlled by similar glutamatergic dysfunction observed for motor impulsivity, except a hyperactive limbic system is also involved. Loss of glutamate homeostasis in prefrontal and nucleus accumbens may contribute to motor impulsivity/affective impulsivity and impulsive choice, respectively. These results are important as they can lead to novel treatments for those with a condition characterized by increased impulsivity that are resistant to conventional treatments.

Adult and adolescent antipsychotic exposure increases delay discounting and diminishes behavioral flexibility in male C57BL/6 mice

Second-generation antipsychotics are frequently prescribed to adolescents, but the long-term consequences of their use remain understudied. These medications work via monoamine neurotransmitter systems, especially dopamine and serotonin, which undergo considerable development and pruning during adolescence. Dopamine and serotonin are linked to a wide host of behaviors, including impulsive choice and behavioral plasticity. In a murine model of adolescent antipsychotic use, male C57BL/6 mice were exposed to either 2.5 mg/kg/day risperidone or 5 mg/kg/day olanzapine via drinking water from postnatal days 22-60. To determine whether the adolescent period was uniquely sensitive to antipsychotic exposure, long-term effects on behavior were compared to an equivalently exposed group of adults where mice were exposed to 2.5 mg/kg risperidone from postnatal days 101-138. Motor activity and body weight in adolescent animals were assessed. Thirty days after exposure terminated animal's behavioral flexibility and impulsive choice were assessed using spatial discrimination reversal and delay discounting. Antipsychotic exposure produced a modest change in behavior flexibility during the second reversal. There was a robust and reproducible difference in impulsive choice: exposed animals devalued the delayed alternative reward substantially more than controls. This effect was observed both following adolescent and adult exposure, indicating that an irreversible change in impulsive choice occurs regardless of the age of exposure.

Developmental exposure to 17-α-hydroxyprogesterone caproate disrupts decision-making in adult female rats: A potential role for a dopaminergic mechanism

The synthetic progestin, 17α-hydroxyprogesterone caproate (17-OHPC), is administered to pregnant individuals at risk for preterm birth and is likely transferred from mother to fetus. Yet, there is little information regarding the potential effects of 17-OHPC administration on behavioral and neural development in offspring. In rats, neonatal 17-OHPC exposure altered dopaminergic fiber distribution and density in the prelimbic medial prefrontal cortex (mPFC) in neonates and adolescents, respectively. Additionally, neonatal 17-OHPC exposure in male rats increased response omissions in a delay discounting task of impulsive decision-making. Because developmental 17-OHPC exposure has differential effects in males and females, investigating the effects of 17-OHPC on impulsive decision-making in female rats is necessary. The present study tested the effects of developmental 17-OHPC exposure (P1-P14) in a delay discounting task in which female rats chose between a small immediate reward and a larger delayed (0, 15 30, or 45 s) reward. 17-OHPC-exposed females made more omissions than controls. There was no effect of 17-OHPC on large reward preference nor on response time, and omissions were similar during both free- and forced-choice trials. The present study also aimed to investigate the neural mechanisms underlying omissions in 17-OHPC-exposed female rats. The dopamine transporter inhibitor, methylphenidate (MPH), was administered prior to delay discounting testing. MPH treatment did not reduce omissions in 17-OHPC-exposed females. If anything, MPH increased omissions in control females nearly fourfold during the longest delays. These results suggest that developmental 17-OHPC exposure increased omissions without affecting impulsivity or slowing decision-making. Furthermore, omissions may be regulated, at least in part, by dopaminergic mechanisms.

Effects of delay sequence in a delay discounting task

Delay discounting refers to the decrease in subjective value of a reward as the delay until its receipt increases. In the present study we assessed the effects of the sequence of delay blocks (increasing or decreasing) on discounting and the data systematicity using a titrating procedure with human participants. All participants completed the delay discounting task in both an increasing and decreasing sequence of delays. Delays ranged from one day to ten years. We found steeper discounting when the delays were presented in an increasing sequence compared with when they were presented in a decreasing sequence. We also found steeper discounting when participants completed the increasing sequence condition first. Our results agree with other findings reported in the literature and suggest that delay discounting may be affected by prior and subsequent experience.

Choice impulsivity after repeated social stress is associated with increased perineuronal nets in the medial prefrontal cortex

Repeated stress can predispose to substance abuse. However, behavioral and neurobiological adaptations that link stress to substance abuse remain unclear. This study investigates whether intermittent social defeat (ISD), a stress protocol that promotes drug-seeking behavior, alters intertemporal decision-making and cortical inhibitory function in the medial prefrontal cortex (mPFC). Male long evans rats were trained in a delay discounting task (DDT) where rats make a choice between a fast (1 s) small reward (1 sugar pellet) and a large reward (3 sugar pellets) that comes with a time delay (10 s or 20 s). A decreased preference for delayed rewards was used as an index of choice impulsivity. Rats were exposed to ISD and tested in the DDT 24 h after each stress episode, and one- and two-weeks after the last stress episode. Immunohistochemistry was performed in rat's brains to evaluate perineuronal nets (PNNs) and parvalbumin GABA interneurons (PV) labeling as markers of inhibitory function in mPFC. ISD significantly decreased the preference for delayed large rewards in low impulsive, but not high impulsive, animals. ISD also increased the density of PNNs in the mPFC. These results suggest that increased choice impulsivity and cortical inhibition predispose animals to seek out rewards after stress.

Refining the study of decision-making in animals: differential effects of d-amphetamine and haloperidol in a novel touchscreen-automated Rearing-Effort Discounting (RED) task and the Fixed-Ratio Effort Discounting (FRED) task

Effort-based decision-making is impaired in multiple psychopathologies leading to significant impacts on the daily life of patients. Preclinical studies of this important transdiagnostic symptom in rodents are hampered, however, by limitations present in currently available decision-making tests, including the presence of delayed reinforcement and off-target cognitive demands. Such possible confounding factors can complicate the interpretation of results in terms of decision-making per se. In this study we addressed this problem using a novel touchscreen Rearing-Effort Discounting (RED) task in which mice choose between two single-touch responses: rearing up to touch an increasingly higher positioned stimulus to obtain a High Reward (HR) or touching a lower stimulus to obtain a Low Reward (LR). To explore the putative advantages of this new approach, RED was compared with a touchscreen version of the well-studied Fixed Ratio-based Effort Discounting (FRED) task, in which multiple touches are required to obtain an HR, and a single response is required to obtain an LR. Results from dopaminergic (haloperidol and d-amphetamine), behavioral (changes in the order of effort demand; fixed-ratio schedule in FRED or response height in RED), and dietary manipulations (reward devaluation by pre-feeding) were consistent with the presence of variables that may complicate interpretation of conventional decision-making tasks, and demonstrate how RED appears to minimize such variables.

Selective chemogenetic inactivation of corticoaccumbal projections disrupts trait choice impulsivity

Impulsive choice has enduring trait-like characteristics and is defined by preference for small immediate rewards over larger delayed ones. Importantly, it is a determining factor in the development and persistence of substance use disorder (SUD). Emerging evidence from human and animal studies suggests frontal cortical regions exert influence over striatal reward processing areas during decision-making in impulsive choice or delay discounting (DD) tasks. The goal of this study was to examine how these circuits are involved in decision-making in animals with defined trait impulsivity. To this end, we trained adolescent male rats to stable behavior on a DD procedure and then re-trained them in adulthood to assess trait-like, conserved impulsive choice across development. We then used chemogenetic tools to selectively and reversibly target corticostriatal projections during performance of the DD task. The prelimbic region of the medial prefrontal cortex (mPFC) was injected with a viral vector expressing inhibitory designer receptors exclusively activated by designer drugs (Gi-DREADD), and then mPFC projections to the nucleus accumbens core (NAc) were selectively suppressed by intra-NAc administration of the Gi-DREADD actuator clozapine-n-oxide (CNO). Inactivation of the mPFC-NAc projection elicited a robust increase in impulsive choice in rats with lower vs. higher baseline impulsivity. This demonstrates a fundamental role for mPFC afferents to the NAc during choice impulsivity and suggests that maladaptive hypofrontality may underlie decreased executive control in animals with higher levels of choice impulsivity. Results such as these may have important implications for the pathophysiology and treatment of impulse control, SUDs, and related psychiatric disorders.

Dopamine D2 receptors in nucleus accumbens cholinergic interneurons increase impulsive choice

Impulsive choice, often characterized by excessive preference for small, short-term rewards over larger, long-term rewards, is a prominent feature of substance use and other neuropsychiatric disorders. The neural mechanisms underlying impulsive choice are not well understood, but growing evidence implicates nucleus accumbens (NAc) dopamine and its actions on dopamine D2 receptors (D2Rs). Because several NAc cell types and afferents express D2Rs, it has been difficult to determine the specific neural mechanisms linking NAc D2Rs to impulsive choice. Of these cell types, cholinergic interneurons (CINs) of the NAc, which express D2Rs, have emerged as key regulators of striatal output and local dopamine release. Despite these relevant functions, whether D2Rs expressed specifically in these neurons contribute to impulsive choice behavior is unknown. Here, we show that D2R upregulation in CINs of the mouse NAc increases impulsive choice as measured in a delay discounting task without affecting reward magnitude sensitivity or interval timing. Conversely, mice lacking D2Rs in CINs showed decreased delay discounting. Furthermore, CIN D2R manipulations did not affect probabilistic discounting, which measures a different form of impulsive choice. Together, these findings suggest that CIN D2Rs regulate impulsive decision-making involving delay costs, providing new insight into the mechanisms by which NAc dopamine influences impulsive behavior.

Sequential delay and probability discounting tasks in mice reveal anchoring effects partially attributable to decision noise

Delay discounting and probability discounting decision making tasks in rodent models have high translational potential. However, it is unclear whether the discounted value of the large reward option is the main contributor to variability in animals' choices in either task, which may limit translation to humans. Male and female mice underwent sessions of delay and probability discounting in sequence to assess how choice behavior adapts over experience with each task. To control for "anchoring" (persistent choices based on the initial delay or probability), mice experienced "Worsening" schedules where the large reward was offered under initially favorable conditions that became less favorable during testing, followed by "Improving" schedules where the large reward was offered under initially unfavorable conditions that improved over a session. During delay discounting, both male and female mice showed elimination of anchoring effects over training. In probability discounting, both sexes of mice continued to show some anchoring even after months of training. One possibility is that "noisy", exploratory choices could contribute to these persistent anchoring effects, rather than constant fluctuations in value discounting. We fit choice behavior in individual animals using models that included both a value-based discounting parameter and a decision noise parameter that captured variability in choices deviating from value maximization. Changes in anchoring behavior over time were tracked by changes in both the value and decision noise parameters in delay discounting, but by the decision noise parameter in probability discounting. Exploratory decision making was also reflected in choice response times that tracked the degree of conflict caused by both uncertainty and temporal cost, but was not linked with differences in locomotor activity reflecting chamber exploration. Thus, variable discounting behavior in mice can result from changes in exploration of the decision options rather than changes in reward valuation.

A method for studying reinforcement factors controlling impulsive choice for use in behavioral neuroscience

Although procedures originating within the experimental analysis of behavior commonly are used in behavioral neuroscience to produce behavioral endpoints, they are used less often to analyze the behavioral processes involved, particularly at the level of individual organisms (see Soto, 2020). Concurrent-chains procedures have been used extensively to study choice and to quantify relations between various dimensions of reinforcement and preference. Unfortunately, parametric analysis of those relations using traditional steady-state, single-subject experimental designs can be time-consuming, often rendering these procedures impractical for use in behavioral neuroscience. The purpose of this paper is to describe how concurrent-chains procedures can be adapted to allow for parametric examination of effects of the reinforcement dimensions involved in impulsive choice (magnitude and delay) within experimental sessions in rats. Data are presented indicating that this procedure can produce relatively consistent within-session estimates of sensitivity to reinforcement in individual subjects, and that these estimates can be modified by neurobiological manipulation (drug administration). These data suggest that this type of procedure offers a promising approach to the study of neurobiological mechanisms of complex behavior in individual organisms, which could facilitate a more fruitful relationship between behavior analysis and behavioral neuroscience.

Impulsivity and risk-seeking as Bayesian inference under dopaminergic control

Bayesian models successfully account for several of dopamine (DA)'s effects on contextual calibration in interval timing and reward estimation. In these models, tonic levels of DA control the precision of stimulus encoding, which is weighed against contextual information when making decisions. When DA levels are high, the animal relies more heavily on the (highly precise) stimulus encoding, whereas when DA levels are low, the context affects decisions more strongly. Here, we extend this idea to intertemporal choice and probability discounting tasks. In intertemporal choice tasks, agents must choose between a small reward delivered soon and a large reward delivered later, whereas in probability discounting tasks, agents must choose between a small reward that is always delivered and a large reward that may be omitted with some probability. Beginning with the principle that animals will seek to maximize their reward rates, we show that the Bayesian model predicts a number of curious empirical findings in both tasks. First, the model predicts that higher DA levels should normally promote selection of the larger/later option, which is often taken to imply that DA decreases 'impulsivity,' and promote selection of the large/risky option, often taken to imply that DA increases 'risk-seeking.' However, if the temporal precision is sufficiently decreased, higher DA levels should have the opposite effect-promoting selection of the smaller/sooner option (higher impulsivity) and the small/safe option (lower risk-seeking). Second, high enough levels of DA can result in preference reversals. Third, selectively decreasing the temporal precision, without manipulating DA, should promote selection of the larger/later and large/risky options. Fourth, when a different post-reward delay is associated with each option, animals will not learn the option-delay contingencies, but this learning can be salvaged when the post-reward delays are made more salient. Finally, the Bayesian model predicts correlations among behavioral phenotypes: Animals that are better timers will also appear less impulsive.

The Effects of Drug Treatments for ADHD in Measures of Cognitive Performance

Based on core symptoms of inattention and deficient impulse control, and the identification of effective pharmacotherapies such as amphetamine (AMP; Adderall^®), methylphenidate (MPH; Ritalin^®), and atomoxetine (ATX; Strattera^®), ADHD is a clinical condition which provides opportunity for translational research. Neuropsychological tests such as the 5-Choice and Continuous Performance Tasks, which measure aspects of attention and impulse control in animals and humans, provide scope for both forward (animal to human) and reverse (human to animal) translation. Rodent studies support pro-attentive effects of AMP and MPH and effectiveness in controlling some forms of impulsive behavior. In contrast, any pro-attentive effects of ATX appear to be less consistent, the most reliable effects of ATX are recorded in tests of impulsivity. These differences may account for AMP and MPH being recognized as first-line treatments for ADHD with a higher efficacy relative to ATX. DSM-5 classifies three "presentations" of ADHD: predominantly inattentive type (ADHD-I), predominantly hyperactive/impulsive type (ADHD-HI), or combined (ADHD-C). Presently, it is unclear whether AMP, MPH, or ATX has differential levels of efficacy across these presentation types. Nonetheless, these studies encourage confidence for the forward translation of NCEs in efforts to identify newer pharmacotherapies for ADHD.

Exposure to nicotine vapor produced by an electronic nicotine delivery system causes short-term increases in impulsive choice in adult male rats

INTRODUCTION

Traditional cigarette use influences cost-benefit decision making by promoting impulsive choice. However, the impact of exposure via electronic nicotine delivery systems on impulsive choice remains unclear. Hence, the present study examined the short- and long-term effects of nicotine vapor on impulsive choice.

METHODS

Twenty-four adult male rats were trained in the delay discounting task, to choose between small, immediate food rewards or large, delayed food rewards. After 24 days of training in the task, rats were exposed to vapor containing either 0, 12, or 24 mg/mL of nicotine, for ten days. To validate inhalation of nicotine vapor, serum cotinine levels were analyzed on exposure days 1, 5, and 10 using enzyme-linked immunosorbent assay (ELISA). Following vapor exposure, rats were retrained in the discounting task until rats displayed stable responding, and the effects of nicotine vapor on choice preference were assessed.

RESULTS

Rats exposed to 12 and 24 mg/mL nicotine vapor displayed higher serum cotinine levels than control rats exposed to 0 mg/mL vapor. There were no differences in impulsive choice between any vapor exposure groups when tested 15 days after exposure, across 6 days of stable responding, suggesting that nicotine vapor does not have long lasting effects on impulsive choice. Interestingly, a subsequent nicotine vapor challenge revealed short-term increases in impulsive choice immediately following a single exposure to 24 mg/mL nicotine vapor, relative to choice preference immediately following exposure to 0 mg/mL vapor.

CONCLUSIONS

These results suggest that exposure to nicotine vapor causes immediate, short-term increases in impulsive choice.

IMPLICATIONS

E-cigarette use is increasing at an alarming rate, particularly among adolescents and young adults. This is concerning given the lack of research into the effects of nicotine vapor exposure on the brain and behavior. The present study describes a viable rodent model of human e-cigarette use and suggest that exposure to nicotine vapor produces short-term increases in impulsive choice.

Contrasting effects of d-amphetamine and atomoxetine on measures of impulsive action and choice

Amphetamine (AMP) and atomoxetine (ATX) represent two of the most widely studied drug treatments used in the investigation of impulsive behaviour. While both drugs have relatively well defined effects in tests designed to investigate impulsive action (e.g. 5-choice task; 5-CSRTT), the effects of both drugs in tests of impulsive choice (e.g. delay discounting) are less consistent. In the present study both AMP and ATX were tested in a rodent gambling task (rGT) and delay discounting in rats separately trained to either an ascending or descending delay schedule. Effects of both drugs were compared to measures of impulsive action (premature (PREM) responses) and perseverative (PSV) responses measured in the 5-choice and rGT tasks. Consistent with previous studies, AMP (0.1-1 mg/kg) increased both PREM and PSV responses, and ATX (0.5-2 mg/kg) reduced both measures in the 5-choice and rGT tasks. At equivalent doses ATX had no reliable effect on choice behaviour in either the rGT or delay discounting suggesting a null effect of this drug on impulsive choice and risky decision making. The effects of AMP were more complex, with a subtle shift in preference to a low risk (P1) choice in the rGT, and an effect on discounting that was unrelated to reinforcer value, but instead dependent on delay sequence and baseline choice preference. One aspect to these outcomes is to highlight the importance of multiple methodological factors when assessing drug effects on complex behaviours such as impulsive choice, and question what are the most appropriate test conditions under which to examine these drugs on discounting.

High impulsive choice is accompanied by an increase in dopamine release in rat dorsolateral striatum

Dopamine neurotransmission has been consistently associated with individual differences in impulsive choice. Clinical and preclinical evidence suggests that low striatal dopamine D₂ signaling predisposes to engage in impulsive behaviors. Although dopamine D₂ signaling controls dopamine (DA) extracellular levels, the relationship between striatal dopamine extracellular levels and impulsive choice remains poorly understood. Using quantitative microdialysis, we investigated whether extracellular DA levels in rat dorsolateral striatum (DLS) correlates with preference for an immediate small reward or for a delayed larger reward. Rats were tested in a delay-discounting task and classified as high impulsive (HI) or low impulsive (LI) according to the area under the discounting curve (AUC). No-net flux microdialysis experiments, assessing basal DA release, DA-uptake, and DA extracellular concentration (DA C_ext), were carried out in dorsolateral striatum (DLS) of urethane-anesthetized rats. Rats classified as HI showed a higher DA release compared with LI rats. Differences in DLS DA-uptake and DA C_ext were non-significant. Importantly, a significant negative correlation was observed between AUC and DA release, indicating that the lower the AUC, the higher the DLS DA release. This finding shows that DA release is augmented in the DLS of rats classified as HI, suggesting that a hyper-activated nigro-striatal pathway contributes to impulsive choice.

Effects of a high-fat diet on impulsive choice in rats

INTRODUCTION

Obesity and binge eating disorder are associated with high levels of impulsivity, but the causal role of eating and palatable food in these associations is unclear. Studies in rodents show that a high-fat diet can increase one aspect of impulsivity (impulsive action); it is less clear, however, whether a dissociable aspect of impulsivity (impulsive choice) is similarly affected. Hence, the aim of this study was to ascertain whether chronic exposure to a high-fat diet would alter impulsive choice.

METHODS

Male rats were maintained on either a high-fat or control chow diet for two weeks ad libitum. They then underwent equi-caloric food restriction for the duration of the experiment, with each group maintained on their respective diet. To measure impulsive choice, rats were trained on a delay discounting task (DDT) in which they made discrete choices between a lever that delivered a small food reward immediately and a lever that delivered a large food reward accompanied by systematically increasing delays. Upon reaching stable performance on the DDT, rats were given acute systemic injections of amphetamine prior to testing in the DDT to determine whether increased monoamine transmission affected impulsive choice differently in the two diet groups. Lastly, subjects were tested on a progressive ratio schedule of reinforcement to assess motivation for a sucrose reward.

RESULTS

There was no significant effect of the high-fat diet on impulsive choice. Further, amphetamine decreased choice of the large, delayed reward (increased impulsive choice) to the same extent in both groups. Exposure to the high-fat diet did, however, increase motivation to obtain a sucrose reward.

CONCLUSIONS

These experiments reveal that, under conditions that do not promote weight gain, a chronic high-fat diet does not affect impulsive choice in a delay discounting task. The data are surprising in light of findings showing that this same diet alters impulsive action, and highlight the necessity of further research to elucidate relationships between palatable food consumption and impulsivity.

Effects of benztropine analogs on delay discounting in rats

RATIONALE

Methylphenidate and d-amphetamine, medications used for treatment of attention deficit hyperactivity disorder (ADHD), are used recreationally and self-administered by laboratory animals. Benztropine (BZT) analogs, like those medications, increase synaptic dopamine levels but are less effective in maintaining self-administration, suggesting clinical utility with less abuse liability.

OBJECTIVES

The current study was designed to evaluate potential therapeutic effects of BZT analogs related to ADHD.

METHODS

Rats responded under a delay-discounting procedure in which responses on one lever produced immediate delivery of a single food pellet and alternative responses produced four food pellets either immediately or with various temporal delays, with those delays arranged in ascending or random orders in different groups of rats. Selection of the smaller more immediate reinforcer has been suggested as an aspect of "impulsivity," a trait with suggested involvement in ADHD. Other rats were studied under fixed-interval (FI) 300-s schedules to assess drug effects on behavior under temporal control.

RESULTS

d-Amphetamine, methylphenidate, and the BZT analog AHN 1-055, but not AHN 2-005 or JHW 007, increased selection of the large, delayed reinforcer with either arrangement of delays. All drugs changed the temporal distribution of responses within the FI from one with responses concentrated at the end to a more uniform distribution. Changes in the temporal distribution of FI responding occurred with drugs that did not affect discounting suggesting that discounting does not arise directly from the same temporal control processes controlling FI responding.

CONCLUSIONS

AHN 1-055 may be of clinical utility in the treatment of ADHD.

Pair housing, but not using a controlled reinforcer frequency procedure, attenuates the modulatory effect of probability presentation order on amphetamine-induced changes in risky choice

Probability discounting is often measured with independent schedules. Independent schedules have several limitations, such as confounding preference for one alternative with frequency of reward presentation and generating ceiling/floor effects at certain probabilities. To address this potential caveat, a controlled reinforcer frequency schedule can be used, in which the manipulandum that leads to reinforcement is pseudo-randomly determined before each trial. This schedule ensures subjects receive equal presentations of the small and large magnitude reinforcers across each block of trials. A total of 24 pair-housed and 11 individually housed female Sprague Dawley rats were tested in a controlled reinforcer frequency procedure. For half of the rats, the odds against (OA) receiving the large magnitude reinforcer increased across the session (ascending schedule); the OA decreased across the session for half of the rats (descending schedule). Following training, rats received treatments of amphetamine (AMPH; 0, 0.25, 0.5, 1.0 mg/kg; s.c.). For pair-housed rats, AMPH (0.5 mg/kg) increased risky choice, regardless of probability presentation order, whereas a higher dose of AMPH (1.0 mg/kg) decreased discriminability of reinforcer magnitude for rats trained on the descending schedule only. For individually housed rats, probability presentation order modulated the effects of AMPH on probability discounting, as AMPH (0.25 and 0.5 mg/kg) increased risky choice in rats trained on the ascending schedule but not on the descending schedule. These results show that pair-housing animals, but not using a controlled reinforcer frequency procedure, attenuates the modulatory effects of probability presentation order on drug effects on risky choice.

Modafinil reduces choice impulsivity while increasing motor activity in preadolescent rats treated prenatally with alcohol

Rats exposed prenatally to alcohol show a reduction in the spontaneous activity of dopaminergic neurons of the ventral tegmental area (VTA), as well as greater impulsive behavior and motor activity, behavioral alterations that have been related to dopaminergic dysfunction. Modafinil (MOD) is a dopamine (DA) reuptake blocker prescribed to treat sleep disorders; however, in recent years it has been used for the treatment of ADHD with positive results. Also, studies in humans and rodents show beneficial effects on learning and attention; however, studies evaluating MOD effects on impulsivity are few and show contradictory results. The purpose of this work was to evaluate the effect of a daily dose of MOD (60 mg/kg i.g.) on cognitive (or choice) impulsivity and motor activity in male preadolescent rats exposed prenatally to alcohol or sucrose (isocaloric control). MOD reduced the impulsive responses in a delay discounting task (DDT) at the same time that increased the motor activity, in both healthy and prenatal alcohol treated rats; however, MOD reduced the response latency in DDT only in prenatal alcohol treated rats. This differential effect of DA activation on impulsivity and motor activity show that the MOD dose that improves the impulse control, does not necessarily decrease motor activity, and suggests a possible differential neural mechanism underlying the expression of these behaviors. On the other hand, the changes in the response latency, only in prenatal alcohol treated groups, suggest that decision-making in animals with a dopaminergic dysfunction is more susceptible to be affected by MOD action.

Using a dependent schedule to measure risky choice in male rats: Effects of d-amphetamine, methylphenidate, and methamphetamine

Risky choice is the tendency to choose a large, uncertain reward over a small, certain reward, and is typically measured with probability discounting, in which the probability of obtaining the large reinforcer decreases across blocks of trials. One caveat to traditional procedures is that independent schedules are used, in which subjects can show exclusive preference for one alternative relative to the other. For example, some rats show exclusive preference for the small, certain reinforcer as soon as delivery of the large reinforcer becomes probabilistic. Therefore, determining if a drug increases risk aversion (i.e., decreases responding for the probabilistic alternative) is difficult (due to floor effects). The overall goal of this experiment was to use a concurrent-chains procedure that incorporated a dependent schedule during the initial link, thus preventing animals from showing exclusive preference for one alternative relative to the other. To determine how pharmacological manipulations alter performance in this task, male Sprague-Dawley rats (n = 8) received injections of amphetamine (0, 0.25, 0.5, 1.0 mg/kg), methylphenidate (0, 0.3, 1.0, 3.0 mg/kg), and methamphetamine (0, 0.5, 1.0, 2.0 mg/kg). Amphetamine (0.25 mg/kg) and methylphenidate (3.0 mg/kg) selectively increased risky choice, whereas higher doses of amphetamine (0.5 and 1.0 kg/mg) and each dose of methamphetamine impaired stimulus control (i.e., flattened the discounting function). These results show that dependent schedules can be used to measure risk-taking behavior and that psychostimulants promote suboptimal choice when this schedule is used. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The Neurobiology of Impulsive Decision-Making and Reinforcement Learning in Nonhuman Animals

Impulsive decisions are those that favor immediate over delayed rewards, involve the acceptance of undue risk or uncertainty, or fail to adapt to environmental changes. Pathological levels of impulsive decision-making have been observed in individuals with mental illness, but there may be substantial heterogeneity in the processes that drive impulsive choices. Understanding this behavioral heterogeneity may be critical for understanding associated diverseness in the neural mechanisms that give rise to impulsivity. The application of reinforcement learning algorithms in the deconstruction of impulsive decision-making phenotypes can help bridge the gap between biology and behavior and provide insights into the biobehavioral heterogeneity of impulsive choice. This chapter will review the literature on the neurobiological mechanisms of impulsive decision-making in nonhuman animals; specifically, the role of the amine neuromodulatory systems (dopamine, serotonin, norepinephrine, and acetylcholine) in impulsive decision-making and reinforcement learning processes is discussed. Ultimately, the integration of reinforcement learning algorithms with sophisticated behavioral and neuroscience techniques may be critical for advancing the understanding of the neurochemical basis of impulsive decision-making.

Effects of opioid/cannabinoid mixtures on impulsivity and memory in rhesus monkeys

The opioid epidemic underscores the need for safer and more effective treatments for pain. Combining opioid receptor agonists with drugs that relieve pain through nonopioid mechanisms could be a useful strategy for reducing the dose of opioid needed to treat pain, thereby reducing risks associated with opioids alone. Opioid/cannabinoid mixtures might be useful in this context; individually, opioids and cannabinoids have modest effects on cognition, and it is important to determine whether those effects occur with mixtures. Delay discounting and delayed matching-to-sample tasks were used to examine effects of the mu-opioid receptor agonist morphine (0.32-5.6 mg/kg), the cannabinoid CB1/CB2 receptor agonist CP55940 (0.0032-0.1 mg/kg), and morphine/CP55940 mixtures on impulsivity (n = 3) and memory (n = 4) in rhesus monkeys. Alone, each drug decreased rate of responding without modifying choice in the delay-discounting task, and morphine/CP55940 mixtures reduced choice of one pellet in a delay dependent manner, with monkeys instead choosing delayed delivery of the larger number of pellets. With the exception of one dose in one monkey, accuracy in the delayed matching-to-sample task was not altered by either drug alone. Morphine/CP55940 mixtures decreased accuracy in two monkeys, but the doses in the mixture were equal to or greater than doses that decreased accuracy or response rate with either drug alone. Rate-decreasing effects of morphine/CP55940 mixtures were additive. These data support the notion that opioid/cannabinoid mixtures that might be effective for treating pain do not have greater, and might have less, adverse effects compared with larger doses of each drug alone.

Effects of d-amphetamine and MK-801 on impulsive choice: Modulation by schedule of reinforcement and delay length

Procedural modifications can modulate drug effects in delay discounting, such as signaling the delay to reinforcement and altering the order in which delays are presented. Although the schedule of reinforcement can alter the rate at which animals discount a reinforcer, research has not determined if animals trained on different schedules of reinforcement are differentially affected by pharmacological manipulations. Similarly, research has not determined if using different delays to reinforcement can modulate drug effects in delay discounting. Male Sprague Dawley rats (n = 36) were split into four groups and were trained in a delay-discounting procedure. The schedule of reinforcement (fixed ratio [FR] 1 vs. FR 10) and delays to reinforcement (0, 5, 10, 20, and 50 s vs. 0, 10, 30, 60, 100 s) were manipulated for each group. Following behavioral training, rats were treated with d-amphetamine (0, 0.25, 0.5, and 1.0 mg/kg) and MK-801 (0, 0.03, and 0.06 mg/kg). Results showed that amphetamine decreased impulsive choice when a FR 1 schedule was used, but only when the short delay sequence was used. Conversely, amphetamine decreased impulsive choice when a FR 10 schedule was used, but only when rats were trained on the long delay sequence. MK-801 decreased impulsive choice in rats trained on a FR 1 schedule, regardless of delay sequence, but did not alter choice in rats trained on a FR 10 schedule. These results show that schedule of reinforcement and delay length can modulate drug effects in delay discounting.

Effects of amphetamine, methylphenidate, atomoxetine, and morphine in rats responding under an adjusting stop signal reaction time task

RATIONALE

Stop signal reaction time procedures are used to investigate behavioral and neurobiological processes that contribute to behavioral inhibition and to evaluate potential therapeutics for disorders characterized by disinhibition and impulsivity. The current study examined effects of amphetamine, methylphenidate, atomoxetine, and morphine in rats responding under an adjusting stop signal reaction time task that measures behavioral inhibition, as well as motor impulsivity.

METHODS

Rats (n = 8) completed a two-response sequence to earn food. During most trials, responses following presentation of a visual stimulus (go signal) delivered food. Occasionally, a tone (stop signal) was presented signifying that food would be presented only if the second response was withheld. Responding after the stop signal measured inhibition and responding prior to the start of the trial (premature) measured motor impulsivity. Delay to presentation of the stop signal was adjusted for individual subjects based on performance.

RESULTS

Amphetamine and methylphenidate increased responding after presentation of the stop signal and markedly increased premature responding. Atomoxetine modestly improved accuracy on stop trials and decreased premature responding. Morphine did not alter stop trial accuracy or premature responding up to doses that decreased the number of trials initiated.

CONCLUSIONS

These data demonstrate the sensitivity of an adjusting stop signal reaction time task to a range of drug effects and shows that some drugs that enhance dopaminergic transmission, such as amphetamine, can differentially alter various types of impulsive behavior.

Effects of morphine/CP55940 mixtures on an impulsive choice task in rhesus monkeys

µ-Opioid receptor agonists are commonly used to treat pain despite their adverse effects. In preclinical studies, cannabinoid receptor agonists increase the potency of opioids for producing antinociceptive but not reinforcing effects. It is unknown whether other adverse effects of these drugs, such as impairment of complex behavior, are enhanced by their co-administration. This study characterized the effects of morphine (µ-opioid receptor agonist; 0.32-5.6 mg/kg, subcutaneously) and CP55940 (CB1/CB2 cannabinoid receptor agonist; 0.0032-0.32 mg/kg, subcutaneously), alone and in mixtures, in monkeys (n=3) choosing between one pellet delivered immediately and two pellets delivered after a delay. Two consecutive choices of the immediate or delayed reward decreased or increased, respectively, the delay. The median adjusted delay, indicating indifference between the immediate and delayed reinforcers, was increased by morphine (3.2 mg/kg) and CP55940 (0.01-0.032 mg/kg). Performance after administration of morphine (0.32 and 1 mg/kg)/CP55940 (0.0032-0.032 mg/kg) mixtures was not different from performance after CP55940 alone. Neither morphine, CP55940, nor mixtures decreased the median adjusted delay (i.e. increased impulsivity). These findings failed to confirm previous studies showing that morphine increases impulsivity, perhaps because of procedural differences among studies. Treatment of pain often requires repeated drug administration; thus, it remains to be determined whether the present findings predict the effects of chronically administered morphine/CP5540 mixtures on impulsive choice.

Examining the neurochemical underpinnings of animal models of risky choice: Methodological and analytic considerations

Because risky choice is associated with several psychiatric conditions, recent research has focused on examining the underlying neurochemical processes that control risk-based decision-making. Not surprisingly, several tasks have been developed to study the neural mechanisms involved in risky choice. The current review will briefly discuss the major tasks used to measure risky choice and will summarize the contribution of several major neurotransmitter systems to this behavior. To date, the most common measures of risky choice are the probability discounting task, the risky decision task, and the rat gambling task. Across these three tasks, the contribution of the dopaminergic system has been most studied, although the effects of serotonergic, adrenergic, cholinergic, and glutamatergic ligands will be discussed. Drug effects across these tasks have been inconsistent, which makes determining the precise role of neurotransmitter systems in risky choice somewhat difficult. Furthermore, procedural differences can modulate drug effects in these procedures, and the way data are analyzed can alter the interpretations one makes concerning pharmacological manipulations. By taking these methodological/analytic considerations into account, we may better elucidate the neurochemistry of risky decision-making. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Translational tests involving non-reward: methodological considerations

This review is concerned with methods for assessing the processing of unrewarded responses in experimental animals and the mechanisms underlying performance of these tasks. A number of clinical populations, including Parkinson's disease, depression, compulsive disorders, and schizophrenia demonstrate either abnormal processing or learning from non-rewarded responses in laboratory-based reinforcement learning tasks. These effects are hypothesized to result from disturbances in modulatory neurotransmitter systems, including dopamine and serotonin. Parallel work in experimental animals has revealed consistent behavioral patterns associated with non-reward and, consistent with the human literature, modulatory roles for specific neurotransmitters. Classical tests involving an important reward omission component include appetitive extinction, ratio schedules of responding, reversal learning, and delay and probability discounting procedures. In addition, innovative behavioral tests have recently been developed leverage probabilistic feedback to specifically assay accommodation of, and learning from, non-rewarded responses. These procedures will be described and reviewed with discussion of the behavioral and neural determinants of performance. A final section focusses specifically on the benefits of trial-by-trial analysis of responding during such tasks, and the implications of such analyses for the translation of findings to clinical studies.

Trait determinants of impulsive behavior: a comprehensive analysis of 188 rats

Impulsivity is a naturally occurring behavior that, when accentuated, can be found in a variety of neuropsychiatric disorders. The expression of trait impulsivity has been shown to change with a variety of factors, such as age and sex, but the existing literature does not reflect widespread consensus regarding the influence of modulating effects. We designed the present study to investigate, in a cohort of significant size (188 rats), the impact of four specific parameters, namely sex, age, strain and phase of estrous cycle, using the variable delay-to-signal (VDS) task. This cohort included (i) control animals from previous experiments; (ii) animals specifically raised for this study; and (iii) animals previously used for breeding purposes. Aging was associated with a general decrease in action impulsivity and an increase in delay tolerance. Females generally performed more impulsive actions than males but no differences were observed regarding delay intolerance. In terms of estrous cycle, no differences in impulsive behavior were observed and regarding strain, Wistar Han animals were, in general, more impulsive than Sprague-Dawley. In addition to further confirming, in a substantial study cohort, the decrease in impulsivity with age, we have demonstrated that both the strain and sex influences modulate different aspects of impulsive behavior manifestations.

Effects of GluN2B-selective antagonists on delay and probability discounting in male rats: Modulation by delay/probability presentation order

The contribution of the GluN2B subunit of the NMDA receptor to impulsivity has recently been examined. Ro 63-1908, a highly selective antagonist for the GluN2B, decreases impulsive choice. Because the order in which delays are presented modulates drug effects in discounting procedures, one goal of the current study was to determine the effects of Ro 63-1908 in delay discounting procedures in which the delays to obtaining the large reinforcer either increase or decrease across the session. We also determined if Ro 63-1908 differentially alters risky choice in probability discounting procedures that use ascending/descending schedules. Male rats were trained in either delay (n = 24) or probability (n = 24) discounting in which the delay to/odds against reinforcement were presented in either ascending or descending order (n = 12 each schedule). Following training, rats received the GluN2B antagonists Ro 63-1908 (0-1.0 mg/kg) and CP-101,606 (0-3.0 mg/kg). In delay discounting, Ro 63-1908 (1.0 mg/kg), but not CP-101,606, decreased choice for the large reinforcer, but only when the delays decreased across the session. In probability discounting, Ro 63-1908 (0.3 mg/kg)/CP-101,606 (1.0 mg/kg) increased choice for the large reinforcer when the probability of obtaining this alternative decreased across the session, but Ro 63-1908 (1.0 mg/kg)/CP-101,606 (3.0 mg/kg) decreased choice when the probabilities increased. These results show that the GluN2B is a mediator of impulsive/risky choice, but the effects of GluN2B antagonists are dependent on the order in which delays/probabilities are presented. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Effects of the NMDA receptor antagonists dizocilpine and Ro 63-1908 on delay-discounting and risky decision-making in a gambling task

Previous studies demonstrated that NMDA receptor antagonists such as dizocilpine (MK801) and the GluN2B NMDA antagonist Ro 63-1908 promote impulsive action (motor impulsivity). The effects of these treatments on impulsive choice and decision-making is less well characterized. Two experiments were undertaken. In the first experiment, given evidence for delay order as a factor in choice selection, the effect of dizocilpine was examined in a delay discounting task with separate groups of male Long-Evans rats trained to a schedule of either ascending (i.e. 0-40 s), or descending delays (i.e. 40-0 s). Under the ascending-delay schedule, dizocilpine (0.03-0.06 mg/kg SC) reduced discounting, yet on the descending-delay schedule discounting was increased. Subgrouping rats according to discounting rate under vehicle pretreatment were consistent with a treatment-induced choice perseveration. In a second experiment, male Long-Evans rats were trained to a gambling task (rGT). Neither dizocilpine (0.01-0.06 mg/kg SC) nor Ro 63-1908 (0.1-1 mg/kg SC) shifted choice from the advantageous to the disadvantageous options. However dizocilpine, and marginally Ro 63-1908, increased choice of the least risky, but suboptimal option. This effect was most evident in rats that initially preferred the disadvantageous options. Consistent with previous studies, both treatments increased measures of motor impulsivity. These results demonstrate that dizocilpine has effects on discounting dependent on delay order and likely reflective of perseveration. On the rGT task, neither dizocilpine nor Ro 63-1908 promoted risky choice, yet both NMDA receptor antagonists increased impulsive action.

Durability and generalizability of time-based intervention effects on impulsive choice in rats

Impulsive choice involves choosing a smaller-sooner (SS) reward over a larger-later (LL) reward. Due to the importance of timing processes in impulsive choice, time-based interventions have been developed to decrease impulsive choice. The present set of experiments assessed the durability and generalizability of time-based interventions. Experiment 1 assessed fixed interval (FI) or variable interval (VI) intervention efficacy over 9 months. The FI intervention decreased impulsive choice, and this effect persisted over time, but the VI intervention effects were only apparent when tested immediately after the intervention. Experiment 2 examined the generalizability of the FI and VI interventions on choice tasks manipulating the SS delay, LL delay, or LL magnitude. The FI intervention decreased sensitivity to delay, promoting LL choices in both delay tasks, but the VI intervention only altered choices when manipulating the SS delay. Experiment 3 further examined the FI intervention effects on tasks that manipulated the LL delay or magnitude immediately following the intervention. The intervention decreased sensitivity to both delay and magnitude. The experiments indicate that the FI intervention is effective at decreasing impulsive choice behavior for an extended period across changing delays and magnitudes, suggesting a relatively broad effect on choice behavior.

Dissecting drug effects in preclinical models of impulsive choice: emphasis on glutamatergic compounds

RATIONALE

Impulsive choice is often measured with delay discounting paradigms. Because there are multiple discounting procedures, as well as different statistical analyses that can be applied to data generated from these paradigms, there are some inconsistencies in the literature regarding drug effects on impulsive choice.

OBJECTIVES

The goal of the current paper is to review the methodological and analytic approaches used to measure discounting and to discuss how these differences can account for differential drug effects observed across studies.

RESULTS

Because some procedures/analyses use a single data point as the dependent variable, changes in this value following pharmacological treatment may be interpreted as alterations in sensitivity to delayed reinforcement, but when other procedures/analyses are used, no changes in behavior are observed. Even when multiple data points are included, some studies show that the statistical analysis (e.g., ANOVA on raw proportion of responses vs. using hyperbolic/exponential functions) can lead to different interpretations. Finally, procedural differences (e.g., delay presentation order, signaling the delay to reinforcement, etc.) in the same discounting paradigm can alter how drugs affect sensitivity to delayed reinforcement.

CONCLUSIONS

Future studies should utilize paradigms that allow one to observe alterations in responding at each delay (e.g., concurrent-chains schedules). Concerning statistical analyses, using parameter estimates derived from nonlinear functions or incorporating the generalized matching law can allow one to determine if drugs affect sensitivity to delayed reinforcement or impair discrimination of the large and small magnitude reinforcers. Using these approaches can help further our understanding of the neurochemical underpinnings of delay discounting.

Pharmacotherapies for decreasing maladaptive choice in drug addiction: Targeting the behavior and the drug

Drug addiction can be conceptualized as a disorder of maladaptive decision making in which drugs are chosen at the expense of pro-social, nondrug alternatives. The study of decision making in drug addiction has focused largely on the role of impulsivity as a facilitator of addiction, in particular the tendency for drug abusers to choose small, immediate gains over larger but delayed outcomes (i.e., delay discounting). A parallel line of work, also focused on decision making in drug addiction, has focused on identifying the determinants underlying the choice to take drugs over nondrug alternatives (i.e., drug vs. nondrug choice). Both tracks of research have been valuable tools in the development of pharmacotherapies for treating maladaptive decision making in drug addiction, and a number of common drugs have been studied in both designs. However, we have observed that there is little uniformity in the administration regimens of potential treatments between the designs, which hinders congruence in the development of single treatment strategies to reduce both impulsive behavior and drug choice. The current review provides an overview of the drugs that have been tested in both delay-discounting and drug-choice designs, and focuses on drugs that reduced the maladaptive choice in both designs. Suggestions to enhance congruence between the findings in future studies are provided. Finally, we propose the use of a hybridized, experimental approach that may enable researchers to test the effectiveness of therapeutics at decreasing impulsive and drug choice in a single design.

Effects of a cafeteria diet on delay discounting in adolescent and adult rats: Alterations on dopaminergic sensitivity

Diet-induced obesity is a laboratory procedure in which nonhuman animals are chronically exposed to a high-fat, high-sugar diet (i.e. cafeteria diet), which results in weight gain, altered sensitivity to reward, and alterations in the dopamine D₂ system. To date, few (if any) studies have examined age-related diet-induced obesity effects in a rat model or have used an impulsive choice task to characterize diet-induced behavioral alterations in reward processes. We exposed rats to a cafeteria-style diet for eight weeks starting at age 21 or 70 days. Following the diet exposures, the rats were tested on a delay discounting task - a measure of impulsive choice in which preference for smaller, immediate vs larger, delayed food reinforcers was assessed. Acute injections of haloperidol (0.03-0.3 mg/kg) were administered to assess the extent to which diet-induced changes in dopamine D₂ influence impulsive food choice. Across both age groups, rats fed a cafeteria diet gained the most weight and consumed more calories than rats fed a standard diet, with rats exposed during development showing the highest weight gain. No age- or diet-related baseline differences in delay discounting were revealed, however, haloperidol unmasked subtle diet-related differences by dose-dependently reducing choice for the larger, later reinforcer. Rats fed a cafeteria diet showed a leftward shift in the dose-response curve, suggesting heightened sensitivity to haloperidol, regardless of age, compared to rats fed a standard diet. Results indicate that chronic exposure to a cafeteria diet resulted in changes in underlying dopamine D₂ that manifested as greater impulsivity independent of age at diet exposure.

Effects of intra-accumbal administration of dopamine and ionotropic glutamate receptor drugs on delay discounting performance in rats

Nucleus accumbens core (NAcc) has been implicated in impulsive choice, as measured in delay discounting. The role of dopamine (DA) in impulsive choice has received considerable attention, whereas glutamate (Glu) has recently been shown to be an important mediator of discounting. However, research has not examined how DA or Glu receptors in NAcc mediate different aspects of delay discounting performance, that is, (a) sensitivity to reinforcer magnitude and (b) sensitivity to delayed reinforcement. Adult male Sprague-Dawley rats were first trained in a delay discounting task, in which the delay to a large magnitude food reinforcer increased across blocks of trials. Following behavioral training, rats received bilateral implantation of guide cannulas into NAcc. Half of the rats (n = 12) received infusions of the DA-selective ligands SKF 38393 (D1-like agonist: 0.03 or 0.1 μg), SCH 23390 (D1-like antagonist: 0.3 or 1.0 μg), quinpirole (D2-like agonist: 0.3 or 1.0 μg), and eticlopride (D2-like antagonist: 0.3 or 1.0 μg). The other half received infusions of the ionotropic Glu ligands MK-801 (NMDA uncompetitive antagonist: 0.3 or 1.0 μg), AP-5 (NMDA competitive antagonist: 0.3 or 1.0 μg), ifenprodil (noncompetitive antagonist at NR2B-containing NMDA receptors: 0.3 or 1.0 μg), and CNQX (AMPA competitive antagonist: 0.2 or 0.5 μg). Results showed that SCH 23390 (0.3 μg) decreased sensitivity to reinforcer magnitude without altering impulsive choice, whereas ifenprodil (1.0 μg) decreased sensitivity to delayed reinforcement (i.e., impulsive choice). The current results show that DA and NMDA receptors in NAcc mediate distinct aspects of discounting performance. (PsycINFO Database Record

Investigation of impulsivity in binge-eating rats in a delay-discounting task and its prevention by the d-amphetamine prodrug, lisdexamfetamine

Freely-fed, female, rats were trained in a two-lever, delay-discounting task: one lever delivered a single chocolate-flavoured pellet immediately and the other a three-pellet reward after increasing delay (0, 4, 8, 16, 32 s). Rats were divided into two groups (i.e. binge-eating rats given irregular, limited access to chocolate in addition to normal chow and controls maintained on normal chow). Both groups exhibited increased preference for the immediate reward as the delay interval was lengthened. The discounting rate was significantly greater in binge-eating rats than non-binge-eating controls, especially as the behaviour became more established indicating that increased impulsivity and intolerance of delayed reward are part of the psychopathology of binge-eating. Lisdexamfetamine (0.8 mg/kg, orally ( d-amphetamine base)) reversed the reduced preference of binge-eating rats for larger rewards at delay intervals of 4 s, 8 s and 32 s and across all sessions. Lisdexamfetamine-treated binge-eating rats consumed the same number of pellets as vehicle-treated, binge-eating rats and non-binge-eating controls eliminating the possibility lisdexamfetamine's actions on appetite or satiety mediated its effects on operant responding for food pellets in delay-discounting. In summary, binge-eating rats showed increased impulsive choice compared with non-binge-eating controls that was reversed by lisdexamfetamine, complementing results showing lisdexamfetamine reduced impulsiveness scores in patients with binge-eating disorder.

Gambling disorder: an integrative review of animal and human studies

Gambling disorder (GD), previously called pathological gambling and classified as an impulse control disorder in DSM-III and DSM-IV, has recently been reclassified as an addictive disorder in the DSM-5. It is widely recognized as an important public health problem associated with substantial personal and social costs, high rates of psychiatric comorbidity, poor physical health, and elevated suicide rates. A number of risk factors have been identified, including some genetic polymorphisms. Animal models have been developed in order to study the underlying neural basis of GD. Here, we discuss recent advances in our understanding of the risk factors, disease course, and pathophysiology. A focus on a phenotype-based dissection of the disorder is included in which known neural correlates from animal and human studies are reviewed. Finally, current treatment approaches are discussed, as well as future directions for GD research.

Endogenous Opiates and Behavior: 2015

This paper is the thirty-eighth consecutive installment of the annual review of research concerning the endogenous opioid system. It summarizes papers published during 2015 that studied the behavioral effects of molecular, pharmacological and genetic manipulation of opioid peptides, opioid receptors, opioid agonists and opioid antagonists. The particular topics that continue to be covered include the molecular-biochemical effects and neurochemical localization studies of endogenous opioids and their receptors related to behavior, and the roles of these opioid peptides and receptors in pain and analgesia, stress and social status, tolerance and dependence, learning and memory, eating and drinking, drug abuse and alcohol, sexual activity and hormones, pregnancy, development and endocrinology, mental illness and mood, seizures and neurologic disorders, electrical-related activity and neurophysiology, general activity and locomotion, gastrointestinal, renal and hepatic functions, cardiovascular responses, respiration and thermoregulation, and immunological responses.

Effects of N-methyl-D-aspartate receptor ligands on sensitivity to reinforcer magnitude and delayed reinforcement in a delay-discounting procedure

RATIONALE

The N-methyl-D-aspartate (NMDA) receptor has been recently identified as an important mediator of impulsive choice, as assessed in delay discounting. Although discounting is independently influenced by sensitivity to reinforcer magnitude and delayed reinforcement, few studies have examined how NMDA receptor ligands differentially affect these parameters.

OBJECTIVES

The current study examined the effects of various NMDA receptor ligands on sensitivity to reinforcer magnitude and delayed reinforcement in a delay-discounting procedure.

METHODS

Following behavioral training, rats received treatments of the following NMDA receptor ligands: the uncompetitive antagonists ketamine (0, 1.0, 5.0, or 10.0 mg/kg; i.p.), MK-801 (0, 0.003, 0.01, or 0.03 mg/kg; s.c.), and memantine (0, 2.5, 5.0, or 10.0 mg/kg; i.p.), the competitive antagonist CGS 19755 (0, 5.0, 10.0, or 20.0 mg/kg; s.c.), the non-competitive NR2B subunit-selective antagonist ifenprodil (0, 1.0, 3.0, or 10.0 mg/kg; i.p), and the partial agonist D-cycloserine (0, 3.25, 15.0, or 30.0 mg/kg; s.c.).

RESULTS

When an exponential model was used to describe discounting, CGS 19755 (5.0 mg/kg) increased impulsive choice without altering sensitivity to reinforcer magnitude. Conversely, ketamine (10.0 mg/kg), memantine (5.0 mg/kg), and ifenprodil (10.0 mg/kg) decreased sensitivity to reinforcer magnitude without altering impulsive choice. MK-801 and D-cycloserine did not alter delay-discounting performance, although two-way ANOVA analyses indicated D-cycloserine (15.0 mg/kg) decreased impulsive choice.

CONCLUSIONS

The behavioral changes observed in delay discounting following administration of NMDA receptor antagonists do not always reflect an alteration in impulsive choice. These results emphasize the utility in employing quantitative methods to assess drug effects in delay discounting.

Effects of Group I metabotropic glutamate receptor antagonists on sensitivity to reinforcer magnitude and delayed reinforcement in a delay-discounting task in rats: Contribution of delay presentation order

Metabotropic glutamate receptor 1 (mGluR₁) blockade has been shown to decrease impulsive choice, as measured in delay discounting. However, several variables are known to influence an animal's discounting, including sensitivity to delayed reinforcement and sensitivity to reinforcer magnitude. The goal of this experiment was to determine the effects of mGluR₁, as well as mGluR₅, antagonism on these parameters. Forty Sprague Dawley rats were trained in delay discounting, in which consistently choosing a small, immediate reward reflects impulsive choice. For half of the rats, the delay to the large reinforcer increased across blocks of trials, whereas the delay decreased across the session for half of the rats. Following training, half of the rats received injections of the mGluR₁ antagonist JNJ 16259685 (JNJ; 0, 0.1, 0.3, or 1.0mg/kg; i.p), and half received injections of the mGluR₅ antagonist MPEP (0, 1.0, 3.0, or 10.0mg/kg; i.p.). Administration of JNJ increased sensitivity to delayed reinforcement (i.e., promoted impulsive choice), regardless of which schedule was used. However, the order in which delays were presented modulated the effects of JNJ on sensitivity to reinforcer magnitude. Specifically, JNJ decreased sensitivity to reinforcer magnitude in rats trained on the descending schedule only. MPEP did not alter sensitivity to reinforcer magnitude or sensitivity to delayed reinforcement. These results show that mGluR₁ is an important mediator of impulsive choice, and they provide further evidence that delay order presentation is an important variable that influences drug effects in delay discounting.

Effects of nucleus accumbens amphetamine administration on performance in a delay discounting task

Chronic administration of cocaine can cause pronounced and enduring cognitive alterations such as increases in impulsive choice. Chronic cocaine can also result in enhanced dopamine (DA) release in the nucleus accumbens (NAc) in response to reward-related cues. It is possible that this enhanced DA release in the NAc is a mechanism by which cocaine increases impulsive choice. To date, however, the specific role of DA in the NAc in impulsive choice is unclear. To begin to address this, rats received acute microinjections of the indirect DA agonist amphetamine directly into the NAc prior to testing in a delay discounting task in which rats chose between a small, immediate and a large, delayed food reward. When delays to the large reward increased within test sessions, amphetamine increased choice of the large reward. When delays decreased within test sessions, however, amphetamine decreased choice of the large reward. These findings suggest that, rather than specifically mediating impulsive choice, DA neurotransmission in the NAc is necessary for flexible adaptation of choice strategies in the presence of shifting reward contingencies. These results further indicate that enhancements in NAc DA release likely do not account for lasting increases in impulsive choice caused by chronic cocaine.

Effect of daily morphine administration and its discontinuation on delay discounting of food in rhesus monkeys

Opioid abusers discount delayed reinforcers more rapidly than nonusers; however, it is unclear whether chronic drug administration or its discontinuation impacts discounting. This study examined the impact of daily morphine administration and its discontinuation on delay discounting of food in rhesus monkeys. Responding on one lever delivered one food pellet immediately; responding on another lever delivered two food pellets either immediately or after a delay (30-120 s) that increased within the session. Monkeys (n=3) responded for the large reinforcer when both reinforcers were delivered immediately and more for the smaller, immediately available reinforcer as the delay to delivery of the large reinforcer increased. When administered acutely, morphine (0.032-5.6 mg/kg) increased trial omissions and had variable effects on choice, with small doses decreasing and large doses increasing choice of the large delayed reinforcer. Chronic morphine administration (0.1 mg/kg/day to 3.2 mg/kg twice daily) reduced choice of the large delayed reinforcer in two monkeys, while increasing choice in a third monkey. Despite the development of tolerance to some effects (i.e. rightward shifts in dose-effect curves for the number of trials omitted) and evidence of mild opioid dependence (e.g. decrease in the number of trials completed, as well as body weight), discontinuation of treatment did not appear to systematically impact discounting. Overall, these results suggest that repeated opioid administration causes persistent effects on choice under a delay discounting procedure; however, differences in the direction of effect among individuals suggest that factors other than, or in addition to, changes in discounting might play a role.

Adolescent methylmercury exposure affects choice and delay discounting in mice

The developing fetus is vulnerable to low-level exposure to methylmercury (MeHg), an environmental neurotoxicant, but the consequences of exposure during the adolescent period remain virtually unknown. The current experiments were designed to assess the effects of low-level MeHg exposure during adolescence on delay discounting, preference for small, immediate reinforcers over large, delayed ones, using a mouse model. Thirty-six male C57BL/6n mice were exposed to 0, 0.3, or 3.0ppm mercury (as MeHg) via drinking water from postnatal day 21 through 59, encompassing the murine adolescent period. As adults, mice lever pressed for a 0.01-cc droplet of milk solution delivered immediately or four 0.01-cc droplets delivered after a delay. Delays ranged from 1.26 to 70.79s, and all were presented within a session. A model based on the Generalized Matching Law indicated that sensitivity to reinforcer magnitude was lower for MeHg-exposed mice relative to controls, indicating that responding in MeHg-exposed mice was relatively indifferent to the larger reinforcer. Sensitivity to reinforcer delay was reduced (delay discounting was decreased) in the 0.3-ppm group, but not in the 3.0-ppm group, compared to controls. Adolescence is a developmental period during which the brain and behavior may be vulnerable to MeHg exposure. As with gestational MeHg exposure, the effects are reflected in the impact of reinforcing stimuli.

Effects of NMDA receptor antagonists on probability discounting depend on the order of probability presentation

Risky decision making can be measured using a probability-discounting procedure, in which animals choose between a small, certain reinforcer and a large, uncertain reinforcer. Recent evidence has identified glutamate as a mediator of risky decision making, as blocking the N-methyl-d-aspartate (NMDA) receptor with MK-801 increases preference for a large, uncertain reinforcer. Because the order in which probabilities associated with the large reinforcer can modulate the effects of drugs on choice, the current study determined if NMDA receptor ligands alter probability discounting using ascending and descending schedules. Sixteen rats were trained in a probability-discounting procedure in which the odds against obtaining the large reinforcer increased (n=8) or decreased (n=8) across blocks of trials. Following behavioral training, rats received treatments of the NMDA receptor ligands MK-801 (uncompetitive antagonist; 0, 0.003, 0.01, or 0.03mg/kg), ketamine (uncompetitive antagonist; 0, 1.0, 5.0, or 10.0mg/kg), and ifenprodil (NR2B-selective non-competitive antagonist; 0, 1.0, 3.0, or 10.0mg/kg). Results showed discounting was steeper (indicating increased risk aversion) for rats on an ascending schedule relative to rats on the descending schedule. Furthermore, the effects of MK-801, ketamine, and ifenprodil on discounting were dependent on the schedule used. Specifically, the highest dose of each drug decreased risk taking in rats in the descending schedule, but only MK-801 (0.03mg/kg) increased risk taking in rats on an ascending schedule. These results show that probability presentation order modulates the effects of NMDA receptor ligands on risky decision making.

Neural substrates underlying effort, time, and risk-based decision making in motivated behavior

All mobile organisms rely on adaptive motivated behavior to overcome the challenges of living in an environment in which essential resources may be limited. A variety of influences ranging from an organism's environment, experiential history, and physiological state all influence a cost-benefit analysis which allows motivation to energize behavior and direct it toward specific goals. Here we review the substantial amount of research aimed at discovering the interconnected neural circuits which allow organisms to carry-out the cost-benefit computations which allow them to behave in adaptive ways. We specifically focus on how the brain deals with different types of costs, including effort requirements, delays to reward and payoff riskiness. An examination of this broad literature highlights the importance of the extended neural circuits which enable organisms to make decisions about these different types of costs. This involves Cortical Structures, including the Anterior Cingulate Cortex (ACC), the Orbital Frontal Cortex (OFC), the Infralimbic Cortex (IL), and prelimbic Cortex (PL), as well as the Baso-Lateral Amygdala (BLA), the Nucleus Accumbens (NAcc), the Ventral Pallidal (VP), the Sub Thalamic Nucleus (STN) among others. Some regions are involved in multiple aspects of cost-benefit computations while the involvement of other regions is restricted to information relating to specific types of costs.

Lorcaserin and CP-809101 reduce motor impulsivity and reinstatement of food seeking behavior in male rats: Implications for understanding the anti-obesity property of 5-HT2C receptor agonists

RATIONALE

The 5-HT2C receptor agonist lorcaserin (Belviq®) has been approved by the FDA for the treatment of obesity. Impulsivity is a contributory feature of some eating disorders.

OBJECTIVE

Experiments investigated the effect of lorcaserin and the highly selective 5-HT2C agonist CP-809101 on measures of impulsivity and on reinstatement of food-seeking behaviour, a model of dietary relapse. The effect of both drugs on 22-h deprivation-induced feeding was also examined, as was the effect of prefeeding in each impulsivity test.

RESULTS

Lorcaserin (0.3-0.6 mg/kg SC) and CP-809101 (0.6-1 mg/kg SC) reduced premature responding in rats trained on the 5-CSRTT and improved accuracy in a Go-NoGo task by reducing false alarms. At equivalent doses, both drugs also reduced reinstatement for food-seeking behaviour. Neither drug altered impulsive choice measured in a delay-discounting task. Lorcaserin (1-3 mg/kg SC) and CP-809101 (3-6 mg/kg SC) reduced deprivation-induced feeding but only at higher doses.

CONCLUSIONS

These results suggest that in addition to previously reported effects on satiety and reward, altered impulse control may represent a contributory factor to the anti-obesity property of 5-HT2C receptor agonists. Lorcaserin may promote weight loss by improving adherence to dietary regimens in individuals otherwise prone to relapse and may be beneficial in cases where obesity is associated with eating disorders tied to impulsive traits, such as binge eating disorder.

Enhanced attention and impulsive action following NMDA receptor GluN2B-selective antagonist pretreatment

NMDA GluN2B (NR2B) subtype selective antagonists are currently in clinical development for a variety of indications, including major depression. We previously reported the selective NMDA GluN2B antagonists Ro 63-1908 and traxoprodil, increase premature responding in a 5-choice serial reaction time task (5-CSRTT) suggesting an effect on impulsive action. The present studies extend these investigations to a Go-NoGo and delay discounting task, and the 5-CSRTT under test conditions of both regular (5s) and short (2-5s) multiple ITI (Intertrial interval). Dizocilpine was included for comparison. Both Ro 63-1908 (0.1-1mg/kg SC) and traxoprodil (0.3-3mg/kg SC) increased premature and perseverative responses in both 5-CSRT tasks and improved attention when tested under a short ITI test condition. Ro 63-1908 but not traxoprodil increased motor impulsivity (false alarms) in a Go-NoGo task. Dizocilpine (0.01-0.06mg/kg SC) affected both measures of motor impulsivity and marginally improved attention. In a delay discounting test of impulsive choice, both dizocilpine and Ro 63-1908 decreased impulsive choice (increased choice for the larger, delayed reward), while traxoprodil showed a similar trend. Motor stimulant effects were evident following Ro 63-1908, but not traxoprodil treatment - although no signs of motor stereotypy characteristic of dizocilpine (>0.1mg/kg) were noted. The findings of both NMDA GluN2B antagonists affecting measures of impulsive action and compulsive behavior may underpin emerging evidence to suggest glutamate signaling through the NMDA GluN2B receptor plays an important role in behavioural flexibility. The profiles between Ro 63-1908 and traxoprodil were not identical, perhaps suggesting differences between members of this drug class.