Sex Differences and Effects of Predictive Cues on Delayed Punishment Discounting

Sex differences in sensitivity to dopamine receptor manipulations of risk-based decision making in rats

Risky decision making involves the ability to weigh risks and rewards associated with different options to make adaptive choices. Previous work has established a necessary role for the basolateral amygdala (BLA) in mediating effective decision making under risk of punishment, but the mechanisms by which the BLA mediates this process are less clear. Because this form of decision making is profoundly sensitive to dopaminergic (DA) manipulations, we hypothesized that DA receptors in the BLA may be involved in risk-taking behavior. To test this hypothesis, male and female Long-Evans rats were trained in a decision-making task in which rats chose between a small, safe food reward and a larger food reward that was associated with a variable risk of footshock punishment. Once behavioral stability emerged, rats received intra-BLA infusions of ligands targeting distinct dopamine receptor subtypes prior to behavioral testing. Intra-BLA infusions of the dopamine D2 receptor (D2R) agonist quinpirole decreased risk taking in females at all doses, and this reduction in risk taking was accompanied by an increase in sensitivity to punishment. In males, decreased risk taking was only observed at the highest dose of quinpirole. In contrast, intra-BLA manipulations of dopamine D1 or D3 receptors (D1R and D3R, respectively) had no effect on risk taking. Considered together, these data suggest that differential D2R sensitivity in the BLA may contribute to the well-established sex differences in risk taking. Neither D1Rs nor D3Rs, however, appear to contribute to risky decision making in either sex.

Understanding sex differences and the translational value of models of persistent substance use despite negative consequences

Persistent substance use despite negative consequences is a key facet of substance use disorder. The last decade has seen the preclinical field adopt the use of punishment to model adverse consequences associated with substance use. This has largely involved the pairing of drug use with either electric foot shock or quinine, a bitter tastant. Whilst at face value, these punishers may model aspects of the physical and psychological consequences of substance use, such models are yet to assist the development of approved medications for treatment. This review discusses progress made with animal models of punishment to understand the behavioral consequences of persistent substance use despite negative consequences. We highlight the importance of examining sex differences, especially when the behavioral response to punishment changes following drug exposure. Finally, we critique the translational value these models provide for the substance use disorder field.

Random interval schedule of reinforcement influences punishment resistance for cocaine in rats

In an animal model of compulsive drug use, a subset of rats continues to self-administer cocaine despite footshock consequences and is considered punishment resistant. We recently found that punishment resistance is associated with habits that persist under conditions that typically encourage a transition to goal-directed control. Given that random ratio (RR) and random interval (RI) schedules of reinforcement influence whether responding is goal-directed or habitual, we investigated the influence of these schedules on punishment resistance for cocaine or food. Male and female Sprague Dawley rats were trained to self-administer either intravenous cocaine or food pellets on a seeking-taking chained schedule of reinforcement, with the seeking lever requiring completion of either an RR20 or RI60 schedule. Rats were then given four days of punishment testing with footshock administered at the completion of seeking on a random one-third of trials. For cocaine-trained rats, the RI60 schedule led to greater punishment resistance (i.e., more trials completed) than the RR20 schedule in males and females. For food-trained rats, the RI60 schedule led to greater punishment resistance (i.e., higher reward rates) than the RR20 schedule in female rats, although male rats showed punishment resistance on both RR20 and RI60 schedules. For both cocaine and food, we found that seeking responses were suppressed to a greater degree than reward rate with the RI60 schedule, whereas response rate and reward rate were equally suppressed with the RR20 schedule. This dissociation between punishment effects on reward rate and response rate with the RI60 schedule can be explained by the nonlinear relation between these variables on RI schedules, but it does not account for the enhanced resistance to punishment. Overall, the results show greater punishment resistance with the RI60 schedule as compared to the RR20 schedule, indicating that schedules of reinforcement are an influencing factor on resistance to negative consequences.

Examining a punishment-related brain circuit with miniature fluorescence microscopes and deep learning

In humans experiencing substance use disorder (SUD), abstinence from drug use is often motivated by a desire to avoid some undesirable consequence of further use: health effects, legal ramifications, etc. This process can be experimentally modeled in rodents by training and subsequently punishing an operant response in a context-induced reinstatement procedure. Understanding the biobehavioral mechanisms underlying punishment learning is critical to understanding both abstinence and relapse in individuals with SUD. To date, most investigations into the neural mechanisms of context-induced reinstatement following punishment have utilized discrete loss-of-function manipulations that do not capture ongoing changes in neural circuitry related to punishment-induced behavior change. Here, we describe a two-pronged approach to analyzing the biobehavioral mechanisms of punishment learning using miniature fluorescence microscopes and deep learning algorithms. We review recent advancements in both techniques and consider a target neural circuit.

Dopaminergic modulation of sensitivity to immediate and delayed punishment during decision-making

Effective decision-making involves careful consideration of all rewarding and aversive outcomes. Importantly, negative outcomes often occur later in time, leading to underestimation, or "discounting," of these consequences. Despite the frequent occurrence of delayed outcomes, little is known about the neurobiology underlying sensitivity to delayed punishment during decision-making. The Delayed Punishment Decision-making Task (DPDT) addresses this by assessing sensitivity to delayed versus immediate punishment in rats. Rats initially avoid punished reinforcers, then select this option more frequently when delay precedes punishment. We used DPDT to examine effects of acute systemic administration of catecholaminergic drugs on sensitivity to delayed punishment in male and female adult rats. Cocaine did not affect choice of rewards with immediate punishment but caused a dose-dependent reduction in choice of delayed punishment. Neither activation nor blockade of D1-like dopamine receptor affected decision-making, but activation of D2-like dopamine receptors reduced choice of delayed punishment. D2 blockade did not attenuate cocaine's effects on decision-making, suggesting that cocaine's effects are not dependent on D2 receptor activation. Increasing synaptic norepinephrine via atomoxetine also reduced choice of delayed (but not immediate) punishment. Notably, when DPDT was modified from ascending to descending pre-punishment delays, these drugs did not affect choice of delayed or immediate punishment, although high-dose quinpirole impaired behavioral flexibility. In summary, sensitivity to delayed punishment is regulated by both dopamine and norepinephrine transmission in task-specific fashion. Understanding the neurochemical modulation of decision-making with delayed punishment is a critical step toward treating disorders characterized by aberrant sensitivity to negative consequences.

Delay of punishment highlights differential vulnerability to developing addiction-like behavior toward sweet food

Resistance to punishment is commonly used to measure the difficulty in refraining from rewarding activities when negative consequences ensue, which is a hallmark of addictive behavior. We recently developed a progressive shock strength (PSS) procedure in which individual rats can titrate the amount of punishment that they are willing to tolerate to obtain food rewards. Here, we investigated the effects of a range of delays (0-12 s) on resistance to punishment measured by PSS break points. As expected from delay discounting principles, we found that delayed shock was less effective as a punisher, as revealed by higher PSS breakpoints. However, this discounting effect was not equally distributed in the population of rats, and the introduction of a delay highlighted the existence of two populations: rats that were sensitive to immediate punishment were also sensitive to delayed shock, whereas rats that were resistant to immediate punishment showed strong temporal discounting of delayed punishment. Importantly, shock-sensitive rats suppressed responding even in subsequent non-punishment sessions, and they differed from shock-resistant rats in anxiety-like behavior, but not in sensitivity to pain. These results show that manipulation of temporal contingencies of punishment in the PSS procedure provides a valuable tool to identify individuals with a double vulnerability to addiction: low sensitivity to aversion and excessive discounting of negative future consequences. Conversely, the shock-sensitive population may provide a model of humans who are vulnerable to opportunity loss due to excessive anxiety.

Making and breaking habits: Revisiting the definitions and behavioral factors that influence habits in animals

Habits have garnered significant interest in studies of associative learning and maladaptive behavior. However, habit research has faced scrutiny and challenges related to the definitions and methods. Differences in the conceptualizations of habits between animal and human studies create difficulties for translational research. Here, we review the definitions and commonly used methods for studying habits in animals and humans and discuss potential alternative ways to assess habits, such as automaticity. To better understand habits, we then focus on the behavioral factors that have been shown to make or break habits in animals, as well as potential mechanisms underlying the influence of these factors. We discuss the evidence that habitual and goal-directed systems learn in parallel and that they seem to interact in competitive and cooperative manners. Finally, we draw parallels between habitual responding and compulsive drug seeking in animals to delineate the similarities and differences in these behaviors.

Effects of fentanyl self-administration on risk-taking behavior in male rats

RATIONALE

Individuals with opioid use disorder (OUD) exhibit impaired decision making and elevated risk-taking behavior. In contrast to the effects of natural and semi-synthetic opioids, however, the impact of synthetic opioids on decision making is still unknown.

OBJECTIVES

The objective of the current study was to determine how chronic exposure to the synthetic opioid fentanyl alters risk-based decision making in adult male rats.

METHODS

Male rats underwent 14 days of intravenous fentanyl or oral sucrose self-administration. After 3 weeks of abstinence, rats were tested in a decision-making task in which they chose between a small, safe food reward and a large food reward accompanied by variable risk of footshock punishment. Following testing in the decision-making task, rats were tested in control assays that assessed willingness to work for food and shock reactivity. Lastly, rats were tested on a probabilistic reversal learning task to evaluate enduring effects of fentanyl on behavioral flexibility.

RESULTS

Relative to rats in the sucrose group, rats in the fentanyl group displayed greater choice of the large, risky reward (risk taking), an effect that was present as long as 7 weeks into abstinence. This increased risk taking was driven by enhanced sensitivity to the large rewards and diminished sensitivity to punishment. The fentanyl-induced elevation in risk taking was not accompanied by alterations in food motivation or shock reactivity or impairments in behavioral flexibility.

CONCLUSIONS

Results from the current study reveal that the synthetic opioid fentanyl leads to long-lasting increases in risk taking in male rats. Future experiments will extend this work to females and identify neural mechanisms that underlie these drug-induced changes in risk taking.

Behavioral mechanisms of oxycodone's effects in female and male rats: Reinforcement delay and impulsive choice

μ-Opioid agonists (e.g., morphine) typically increase impulsive choice, which has been interpreted as an opioid-induced increase in sensitivity to reinforcement delay. Relatively little research has been done with opioids other than morphine (e.g., oxycodone), or on sex differences in opioid effects, on impulsive choice. The present study investigated the effects of acute (0.1-1.0 mg/kg) and chronic (1.0 mg/kg twice/day) administration of oxycodone on choice controlled by reinforcement delay, a primary mechanism implicated in impulsive choice, in female and male rats. Rats responded under a concurrent-chains procedure designed to quantify the effects of reinforcement delay on choice within each session. For both sexes, choice was sensitive to delay under this procedure. Sensitivity to delay under baseline was slightly higher for males than females, suggesting more impulsive choice with males. When given acutely, intermediate and higher doses of oxycodone decreased sensitivity to delay; this effect was larger and more reliable in males than females. When given chronically, sex differences were also observed: tolerance developed to the sensitivity-decreasing effects in females, whereas sensitization developed in males. These data suggest that reinforcement delay may play an important role in sex differences in impulsive choice, as well as in the effects of acute and chronic administration of opioids in impulsive choice. However, drug-induced changes in impulsive choice could be related to at least two potential behavioral mechanisms: reinforcement delay and/or reinforcement magnitude. Effects of oxycodone on sensitivity to reinforcement magnitude remain to be fully characterized. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Pathways to the persistence of drug use despite its adverse consequences

The persistence of drug taking despite its adverse consequences plays a central role in the presentation, diagnosis, and impacts of addiction. Eventual recognition and appraisal of these adverse consequences is central to decisions to reduce or cease use. However, the most appropriate ways of conceptualizing persistence in the face of adverse consequences remain unclear. Here we review evidence that there are at least three pathways to persistent use despite the negative consequences of that use. A cognitive pathway for recognition of adverse consequences, a motivational pathway for valuation of these consequences, and a behavioral pathway for responding to these adverse consequences. These pathways are dynamic, not linear, with multiple possible trajectories between them, and each is sufficient to produce persistence. We describe these pathways, their characteristics, brain cellular and circuit substrates, and we highlight their relevance to different pathways to self- and treatment-guided behavior change.

Reward maximization assessed using a sequential patch depletion task in a large sample of heterogeneous stock rats

Choice behavior requires animals to evaluate both short- and long-term advantages and disadvantages of all potential alternatives. Impulsive choice is traditionally measured in laboratory tasks by utilizing delay discounting (DD), a paradigm that offers a choice between a smaller immediate reward, or a larger more delayed reward. This study tested a large sample of Heterogeneous Stock (HS) male (n = 896) and female (n = 898) rats, part of a larger genetic study, to investigate whether measures of reward maximization overlapped with traditional models of delay discounting via the patch depletion model using a Sequential Patch Depletion procedure. In this task, rats were offered a concurrent choice between two water "patches" and could elect to "stay" in the current patch or "leave" for an alternative patch. Staying in the current patch resulted in decreasing subsequent reward magnitudes, whereas the choice to leave a patch was followed by a delay and a resetting to the maximum reward magnitude. Based on the delay in a given session, different visit durations were necessary to obtain the maximum number of rewards. Visit duration may be analogous to an indifference point in traditional DD tasks. Males and females did not significantly differ on traditional measures of DD (e.g. delay gradient; AUC). When examining measures of patch utilization, females made fewer patch changes at all delays and spent more time in the patch before leaving for the alternative patch compared to males. Consistent with this, there was some evidence that females deviated from reward maximization more than males. However, when controlling for body weight, females had a higher normalized rate of reinforcement than males. Measures of reward maximization were only weakly associated with traditional DD measures and may represent distinctive underlying processes. Taken together, females performance differed from males with regard to reward maximization that were not observed utilizing traditional measures of DD, suggesting that the patch depletion model was more sensitive to modest sex differences when compared to traditional DD measures in a large sample of HS rats.

Cognitive mechanisms underlying decision making involving risk of explicit punishment in male and female rats

Individuals engage in the process of risk-based decision making on a daily basis to navigate various aspects of life. There are, however, individual differences in this form of decision making, with some individuals exhibiting preference for riskier choices (risk taking) and others exhibiting preference for safer choices (risk aversion). Recent work has shown that extremes in risk taking (e.g., excessive risk taking or risk aversion) are not only cognitive features of neuropsychiatric diseases, but may in fact predispose individuals to the development of such diseases. To better understand individual differences in risk taking, and thus the mechanisms by which they confer disease vulnerability, the current study investigated the cognitive contributions to risk taking in both males and females. Rats were first behaviorally characterized in a decision-making task involving risk of footshock punishment and then tested on a battery of cognitive behavioral assays. Individual variability in risk taking was compared with performance on these tasks. Consistent with prior work, females were more risk averse than males. With the exception of the Set-shifting Task, there were no sex differences in performance on other cognitive assays. There were, however, sex-dependent associations between risk taking and specific cognitive measures. Greater risk taking was associated with better cognitive flexibility in males whereas greater risk aversion was associated with better working memory in females. Collectively, these findings reveal that distinct cognitive mechanisms are associated with risk taking in males and females, which may account for sex differences in this form of decision making.

Effects of the estrous cycle on impulsive choice and interval timing in female rats

Research in humans and animals shows differences in impulsive choice, which is a failure to wait for larger, delayed rewards, when comparing males and females. It is possible that fluctuations in sex hormones (estradiol and progesterone) across the reproductive cycle contribute to sex differences in impulsive choice. The current study delivered an impulsive choice task with peak interval trials to female rats while estrous cycles, the rodent reproductive cycle, were tracked over the course of the task. Female rats were more sensitive to changes in delay in the proestrus phase of the estrous cycle and made more larger-later choices when in estrus, particularly when the delay to the smaller reward was short. Estradiol increases dramatically during proestrus while progesterone peaks during estrus, suggesting that estradiol and progesterone may affect impulsive choice through mechanisms such as delay discounting, delay aversion, and/or timing processes. Analyses of timing of the choice task delays showed inconsistent effects of the estrous cycle across delays, suggesting that reward-timing interactions may have complicated how hormone fluctuations affected interval timing. Further research is needed to determine the mechanism underlying increased larger-later choices during the estrus phase, increased delay sensitivity during the proestrus phase, and variability in interval timing across delays and estrous cycle stages.

Reward Maximization Assessed Using a Sequential Patch Depletion Task in a Large Sample of Heterogeneous Stock Rats

Choice behavior requires animals to evaluate both short- and long-term advantages and disadvantages of all potential alternatives. Impulsive choice is traditionally measured in laboratory tasks by utilizing delay discounting (DD), a paradigm that offers a choice between a smaller immediate reward, or a larger more delayed reward. This study tested a large sample of Heterogeneous Stock (HS) male (n = 896) and female (n = 898) rats, part of a larger genetic study, to investigate whether measures of reward maximization overlapped with traditional models of delay discounting via the patch depletion model using a Sequential Patch Depletion procedure. In this task, rats were offered a concurrent choice between two water "patches" and could elect to "stay" in the current patch or "leave" for an alternative patch. Staying in the current patch resulted in decreasing subsequent reward magnitudes, whereas the choice to leave a patch was followed by a delay and a resetting to the maximum reward magnitude. Based on the delay in a given session, different visit durations were necessary to obtain the maximum number of rewards. Visit duration may be analogous to an indifference point in traditional DD tasks. While differences in traditional DD measures (e.g., delay gradient) have been detected between males and females, these effects were small and inconsistent. However, when examining measures of reward maximization, females made fewer patch changes at all delays and spent more time in the patch before leaving for the alternative patch compared to males. This pattern of choice resulted in males having a higher rate of reinforcement than females. Consistent with this, there was some evidence that females deviated from the optimal more, leading to less reward. Measures of reward maximization were only weakly associated with traditional DD measures and may represent distinctive underlying processes. Taken together, females performance differed from males with regard to reward maximization that were not observed utilizing traditional measures of DD, suggesting that the patch depletion model was more sensitive to modest sex differences when compared to traditional DD measures in a large sample of HS rats.

Age-Related Changes in Risky Decision Making and Associated Neural Circuitry in a Rat Model

Altered decision making at advanced ages can have a significant impact on an individual's quality of life and the ability to maintain personal independence. Relative to young adults, older adults make less impulsive and less risky choices; although these changes in decision making could be considered beneficial, they can also lead to choices with potentially negative consequences (e.g., avoidance of medical procedures). Rodent models of decision making have been invaluable for dissecting cognitive and neurobiological mechanisms that contribute to age-related changes in decision making, but they have predominantly used costs related to timing or probability of reward delivery and have not considered other equally important costs, such as the risk of adverse consequences. The current study therefore used a rat model of decision making involving risk of explicit punishment to examine age-related changes in this form of choice behavior in male rats, and to identify potential cognitive and neurobiological mechanisms that contribute to these changes. Relative to young rats, aged rats displayed greater risk aversion, which was not attributable to reduced motivation for food, changes in shock sensitivity, or impaired cognitive flexibility. Functional MRI analyses revealed that, overall, functional connectivity was greater in aged rats compared with young rats, particularly among brain regions implicated in risky decision making such as basolateral amygdala, orbitofrontal cortex, and ventral tegmental area. Collectively, these findings are consistent with greater risk aversion found in older humans, and reveal age-related changes in brain connectivity.

Lateral Orbitofrontal Cortex and Basolateral Amygdala Regulate Sensitivity to Delayed Punishment during Decision-making

In real-world decision-making scenarios, negative consequences do not always occur immediately after a choice. This delay between action and outcome drives the underestimation, or "delay discounting", of punishment. While the neural substrates underlying sensitivity to immediate punishment have been well-studied, there has been minimal investigation of delayed consequences. Here, we assessed the role of lateral orbitofrontal cortex (LOFC) and basolateral amygdala (BLA), two regions implicated in cost/benefit decision-making, in sensitivity to delayed vs immediate punishment. The delayed punishment decision-making task (DPDT) was used to measure delay discounting of punishment in rodents. During DPDT, rats choose between a small, single pellet reward and a large, three pellet reward accompanied by a mild foot shock. As the task progresses, the shock is preceded by a delay that systematically increases or decreases throughout the session. We observed that rats avoid choices associated with immediate punishment, then shift preference toward these options when punishment is delayed. LOFC inactivation did not influence choice of rewards with immediate punishment, but decreased choice of delayed punishment. We also observed that BLA inactivation reduced choice of delayed punishment for ascending but not descending delays. Inactivation of either brain region produced comparable effects on decision-making in males and females, but there were sex differences observed in omissions and latency to make a choice. In summary, both LOFC and BLA contribute to the delay discounting of punishment and may serve as promising therapeutic targets to improve sensitivity to delayed punishment during decision-making.Significance StatementNegative consequences occurring after a delay are often underestimated, which can lead to maladaptive decision-making. While sensitivity to immediate punishment during reward-seeking has been well-studied, the neural substrates underlying sensitivity to delayed punishment remain unclear. Here, we used the Delayed Punishment Decision-making Task to determine that lateral orbitofrontal cortex and basolateral amygdala both regulate the discounting of delayed punishment, suggesting that these regions may be potential targets to improve decision-making in psychopathology.

Regulation of sex differences in risk-based decision making by gonadal hormones: Insights from rodent models

Men and women differ in their ability to evaluate options that vary in their rewards and the risks that are associated with these outcomes. Most studies have shown that women are more risk averse than men and that gonadal hormones significantly contribute to this sex difference. Gonadal hormones can influence risk-based decision making (i.e., risk taking) by modulating the neurobiological substrates underlying this cognitive process. Indeed, estradiol, progesterone and testosterone modulate activity in the prefrontal cortex, amygdala and nucleus accumbens associated with reward and risk-related information. The use of animal models of decision making has advanced our understanding of the intersection between the behavioral, neural and hormonal mechanisms underlying sex differences in risk taking. This review will outline the current state of this literature, identify the current gaps in knowledge and suggest the neurobiological mechanisms by which hormones regulate risky decision making. Collectively, this knowledge can be used to understand the potential consequences of significant hormonal changes, whether endogenously or exogenously induced, on risk-based decision making as well as the neuroendocrinological basis of neuropsychiatric diseases that are characterized by impaired risk taking, such as substance use disorder and schizophrenia.

Instrumental aversion coding in the basolateral amygdala and its reversion by a benzodiazepine

Punishment involves learning the relationship between actions and their adverse consequences. Both the acquisition and expression of punishment learning depend on the basolateral amygdala (BLA), but how BLA supports punishment remains poorly understood. To address this, we measured calcium (Ca²⁺) transients in BLA principal neurons during punishment. Male rats were trained to press two individually presented levers for food; when one of these levers also yielded aversive footshock, responding on this punished lever decreased relative to the other, unpunished lever. In rats with the Ca²⁺ indicator GCaMP6f targeted to BLA principal neurons, we observed excitatory activity transients to the footshock punisher and inhibitory transients to lever-presses earning a reward. Critically, as rats learned punishment, activity around the punished response transformed from inhibitory to excitatory and similarity analyses showed that these punished lever-press transients resembled BLA transients to the punisher itself. Systemically administered benzodiazepine (midazolam) selectively alleviated punishment. Moreover, the degree to which midazolam alleviated punishment was associated with how much punished response-related BLA transients reverted to their pre-punishment state. Together, these findings show that punishment learning is supported by aversion-coding of instrumental responses in the BLA and that the anti-punishment effects of benzodiazepines are associated with a reversion of this aversion coding.

Impairments in Fear Extinction Memory and Basolateral Amygdala Plasticity in the TgF344-AD Rat Model of Alzheimer's Disease Are Distinct from Nonpathological Aging

Fear-based disorders such as post-traumatic stress disorder (PTSD) steepen age-related cognitive decline and double the risk for developing Alzheimer's disease (AD). Because of the seemingly hyperactive properties of fear memories, PTSD symptoms can worsen with age. Perturbations in the synaptic circuitry supporting fear memory extinction are key neural substrates of PTSD. The basolateral amygdala (BLA) is a medial temporal lobe structure that is critical in the encoding, consolidation, and retrieval of fear memories. As little is known about fear extinction memory and BLA synaptic dysfunction within the context of aging and AD, the goal of this study was to investigate how fear extinction memory deficits and basal amygdaloid nucleus (BA) synaptic dysfunction differentially associate in nonpathologic aging and AD in the TgF344AD (TgAD) rat model of AD. Young, middle-aged, and older-aged WT and TgAD rats were trained on a delay fear conditioning and extinction procedure before ex vivo extracellular field potential recording experiments in the BA. Relative to young WT rats, long-term extinction memory was impaired, and in general, was associated with a hyperexcitable BA and impaired LTP in TgAD rats at all ages. In contrast, long-term extinction memory was impaired in aged WT rats and was associated with impaired LTP but not BA hyperexcitability. Interestingly, the middle-aged TgAD rats showed intact short-term extinction and BA LTP, which is suggestive of a compensatory mechanism, whereas differential neural recruitment in older-aged WT rats may have facilitated short-term extinction. As such, associations between fear extinction memory and amygdala deficits in nonpathologic aging and AD are dissociable.

Spontaneously hypertensive rats show higher impulsive action, but equal impulsive choice with both positive and aversive consequences

Both positive and aversive delayed consequences play an important role in decision making. However, most of research has studied the temporal discounting of the positive consequences, while the study of the aversives is scarce in general and null in some areas. This is the case of research on impulsivity in spontaneously hypertensive rats (SHR), an animal model of ADHD. To evaluate SHRs' sensitivity to aversive delayed consequences, we employed a choice procedure in which subjects chose between a smaller-amount alternative and a larger-amount alternative plus a shock; when preference for the smaller-amount alternative stabilized, five different delays to the shock were presented with the objective of analyzing the recovery of preference for the larger-amount alternative, which is related to the sensitivity to the delayed aversive consequence. To analyze the sensitivity to delayed positive consequences we employed a procedure that evaluated the preference between a smaller-amount alternative and a larger-amount alternative as a function of the delay to the later. Finally, to evaluate impulsive action we employed a DRL 10s schedule. In all tasks, Wistar rats were evaluated as control strain. The results indicated that choice impulsivity was equivalent between strains, both for positive and for aversive consequences. In contrast, we found a higher level of impulsive action in SHR.

Effects of the psychoactive compounds in green tea on risky decision-making

Epigallocatechin-3-gallate (EGCG) and caffeine are the two primary compounds found in green tea. While EGCG has anxiolytic and anti-inflammatory effects, its acute effects on cognition are not well understood. Furthermore, despite widespread green tea consumption, little is known about how EGCG and caffeine co-administration impacts behavior. Here, we investigated the effects of multiple doses of either EGCG or caffeine on a rat model of risk-taking. This was assessed using the risky decision-making task (RDT), in which rats choose between a small, well-tolerated reward and a large reward with escalating risk of mild footshock. Rats were tested in RDT after acute systemic administration of EGCG, caffeine or joint EGCG and caffeine. EGCG caused a dose-dependent reduction in risk-taking without affecting reward discrimination or task engagement. Caffeine did not impact risk-taking, but elevated locomotor activity and reduced task engagement at high doses. Finally, exposure to both EGCG and caffeine had no effect on risk-taking, suggesting that low-dose caffeine is sufficient to mask the risk-aversion caused by EGCG. These data suggest EGCG as a potential therapeutic treatment for psychological disorders that induce compulsive risky decision-making.

Basic Behavioral Processes Involved in Procrastination

Procrastination involves an irrational putting off of engaging in a course of action, in spite of expecting to be worse off for the delay. I suggest that to understand the processes underlying procrastination one should examine its relation to several behavioral procedures that have been studied in humans and other animals. For example, in delay discounting, smaller rewards that come sooner are often preferred over larger rewards that come later. In the context of delay discounting, procrastination can be viewed as the preference for an immediate competing activity over the delay to work on a required task. Another process similar to procrastination can be seen in free operant, temporal avoidance (or Sidman avoidance) in which an animal will receive a shock (a deadline not met) if an interval passes without a specified response (task completion). Once animals learn about the interval, they often procrastinate by waiting until the interval has almost passed before responding. Finally, research with animals suggests that the persistence of procrastination may involve a form of negative reinforcement associated with the sudden decline in anxiety or fear (relief) when the task is completed prior to the deadline. Research with animals suggests that the mechanisms responsible for human procrastination may involve systems that derive from several procedures known to produce similar behavior animals.

Juvenile stress increases cocaine-induced impulsivity in female rats

In humans, adverse childhood experiences are associated with an increased risk of developing a neuropsychiatric disorder. Changes in social behavior and cognitive function are hallmarks of numerous neuropsychiatric disorders. Here we examined the effects of exposure to variable stress during the juvenile period on social behavior, reward, and cognitive function (as measured in the 5-choice serial reaction time task (5CSRTT)) in rats. From postnatal days (PND) 25-29 male and female rats were exposed to a variable stress protocol. In adulthood, social interactions and sucrose preference were assessed prior to training on the 5CSRTT. Once successfully trained, rats were challenged with different task versions, and then the effects of cocaine (0, 10, or 20 mg/kg, IP) on performance were assessed. A follow-up experiment examined the ability of the D2 receptor antagonist eticlopride (0.0, 0.025, 0.05 mg/kg, IP) to block the effects of cocaine on 5CSRTT performance in female rats. Male rats exposed to juvenile stress tended to engage in less social behavior and had an increased correct response latency in the 5CSRTT following cocaine administration. Female rats exposed to juvenile stress exhibited a trend towards increased social behavior and demonstrated increased cocaine-induced impulsivity. The increase in impulsivity was not blocked by co-administration of eticlopride. Juvenile stress had minimal effects on adult behavior in male rats, but increased cocaine-induced impulsivity in female rats. Such an effect could contribute to the enhanced escalation of drug-use observed in females that experience juvenile stress. This possibility awaits further testing.

Reward/Punishment-Based Decision Making in Rodents

Deficits in decision making are at the heart of many psychiatric diseases, such as substance abuse disorders and attention deficit hyperactivity disorder. Consequently, rodent models of decision making are germane to understanding the neural mechanisms underlying adaptive choice behavior and how such mechanisms can become compromised in pathological conditions. A critical factor that must be integrated with reward value to ensure optimal decision making is the occurrence of consequences, which can differ based on probability (risk of punishment) and temporal contiguity (delayed punishment). This article will focus on two models of decision making that involve explicit punishment, both of which recapitulate different aspects of consequences during human decision making. We will discuss each behavioral protocol, the parameters to consider when designing an experiment, and finally how such animal models can be utilized in studies of psychiatric disease. © 2020 Wiley Periodicals LLC. Basic Protocol 1: Behavioral training Support Protocol: Equipment testing Alternate Protocol: Reward discrimination Basic Protocol 2: Risky decision-making task (RDT) Basic Protocol 3: Delayed punishment decision-making task (DPDT).

Neurochemical substrates linked to impulsive and compulsive phenotypes in addiction: A preclinical perspective

Drug compulsion manifests in some but not all individuals and implicates multifaceted processes including failures in top-down cognitive control as drivers for the hazardous pursuit of drug use in some individuals. As a closely related construct, impulsivity encompasses rash or risky behaviour without foresight and underlies most forms of drug taking behaviour, including drug use during adverse emotional states (i.e., negative urgency). While impulsive behavioural dimensions emerge from drug-induced brain plasticity, burgeoning evidence suggests that impulsivity also predates the emergence of compulsive drug use. Although the neural substrates underlying the apparently causal relationship between trait impulsivity and drug compulsion are poorly understood, significant advances have come from the interrogation of defined limbic cortico-striatal circuits involved in motivated behaviour and response inhibition, together with chemical neuromodulatory influences from the ascending neurotransmitter systems. We review what is presently known about the neurochemical mediation of impulsivity, in its various forms, and ask whether commonalities exist in the neurochemistry of compulsive drug-motivated behaviours that might explain individual risk for addiction.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.

Regulation of risky decision making by gonadal hormones in males and females

Psychiatric diseases characterized by dysregulated risky decision making are differentially represented in males and females. The factors that govern such sex differences, however, remain poorly understood. Using a task in which rats make discrete trial choices between a small, "safe" food reward and a large food reward accompanied by varying probabilities of footshock punishment, we recently showed that females are more risk averse than males. The objective of the current experiments was to test the extent to which these sex differences in risky decision making are mediated by gonadal hormones. Male and female rats were trained in the risky decision-making task, followed by ovariectomy (OVX), orchiectomy (ORX), or sham surgery. Rats were then retested in the task, under both baseline conditions and following administration of estradiol and/or testosterone. OVX increased choice of the large, risky reward (increased risky choice), an effect that was attenuated by estradiol administration. In contrast, ORX decreased risky choice, but testosterone administration was without effect in either ORX or sham males. Estradiol, however, decreased risky choice in both groups of males. Importantly, none of the effects of hormonal manipulation on risky choice were due to altered shock sensitivity or food motivation. These data show that gonadal hormones are required for maintaining sex-typical profiles of risk-taking behavior in both males and females, and that estradiol is sufficient to promote risk aversion in both sexes. The findings provide novel information about the mechanisms supporting sex differences in risk taking and may prove useful in understanding sex differences in the prevalence of psychiatric diseases associated with altered risk taking.

Testicular hormones mediate robust sex differences in impulsive choice in rats

Impairments in choosing optimally between immediate and delayed rewards are associated with numerous psychiatric disorders. Such ‘intertemporal’ choice is influenced by genetic and experiential factors; however, the contributions of biological sex are understudied and data to date are largely inconclusive. Rats were used to determine how sex and gonadal hormones influence choices between small, immediate and large, delayed rewards. Females showed markedly greater preference than males for small, immediate over large, delayed rewards (greater impulsive choice). This difference was neither due to differences in food motivation or reward magnitude perception, nor was it affected by estrous cycle. Ovariectomies did not affect choice in females, whereas orchiectomies increased impulsive choice in males. These data show that male rats exhibit less impulsive choice than females and that this difference is at least partly maintained by testicular hormones. These differences in impulsive choice could be linked to gender differences across multiple neuropsychiatric conditions.

Involvement of noradrenergic and serotonergic systems in risk-based decisions between options of equivalent expected value in rats

Risk perception is an important factor that may mediate risk-based decision-making processes regulated by noradrenergic (NA) and serotonergic (5-HT) systems. Most risk-based decision-making models involve complex factors, such as risk perception or reward value, such that the final decision is the result of the interactions among these factors. However, the contribution of risk perception per se in risk decisions has remained unclear. Therefore, in the present study, we made some modifications to the classical probabilistic discounting task (PDT) to focus on the impact of risk perception and noradrenergic/serotonergic systems on decision-making behavior. Meanwhile, we conducted an elevated plus-maze (EPM) test to detect the correlation between anxiety and choice behavior. In the current study, rats had to choose between a "certain" lever that delivered a certain number of pellets and a "risky" lever that delivered eight pellets in a probabilistic manner (descending: 50%, 25%, 12.5% or ascending 12.5%, 25%, 50% of the time). The long-term rewarding values of the two levers were always identical in each block within each session. According to their baseline performances in choosing the risky lever, rats were divided into the risk-prefer group and risk-averse group. The results showed that there was a significant correlation between open arm time in EPM and risky choice for both descending order and ascending order, indicating that highly anxious rats more often preferred the safe option under risk. Pharmacological stimulation of α2-adrenergic receptors via dexmedetomidine (0.01 mg/kg) decreased the preference of probabilistic rewards in the risk-prefer group, while blocking α2 receptors by atipamezole (0.3 mg/kg) also reduced risky choices. The NA reuptake inhibitor, atomoxetine, increased the preference for risky choices in the risk-prefer group, the effect of which was attained via multiple superimposed doses. Administration of the 5-HT_2A receptor agonist, DOI (0.1 mg/kg), increased risk-taking behavior in the risk-prefer group. Taken together, these results suggest that NA may be more inclined to process negative information such as loss or uncertainty in the regulation of risk-related decision making, whereas 5-HT may function primarily to increase risk-taking behavior. Our findings may help to further elucidate how noradrenergic and serotonergic systems differentially affect individuals with different risk preferences in terms of regulating risk perception in risk-related decision making.

Translating concepts of risk and loss in rodent models of gambling and the limitations for clinical applications

Gambling involves placing something of value at risk in exchange for the opportunity to potentially gain something of greater value in return. A variety of gambling paradigms have been designed to study the maladaptive decision-making that underlies problematic gambling. Central to these gambling models are the definitions of "risk" and "loss", especially when translating the results from rodent studies to clinical applications. Risk and loss are not mutually exclusive but rather share some overlap. With careful interpretation and consideration of the limitations of these behavioral paradigms, results from rodent models may provide insights into the neurobiology of risky decision-making that leads to problematic gambling in humans.