Reward/Punishment-Based Decision Making in Rodents

Effects of psilocybin on uncertain punishment learning

Psilocybin may provide a useful treatment for mood disorders including anxiety and depression but its mechanisms of action for these effects are not well understood. While recent preclinical work has begun to assess psilocybin's role in affective behaviors through innate anxiety or fear conditioning, there is scant evidence for its role in conflict between reward and punishment. The current study was designed to determine the impact of psilocybin on the learning of reward-punishment conflict associations, as well as its effects after learning, in male and female rats. We utilized a chained schedule of reinforcement that involved execution of safe and risky reward-guided actions under uncertain punishment. Different patterns of behavioral suppression by psilocybin emerged during learning versus after learning of risky action-reward associations. Psilocybin increased behavioral suppression in female rats as punishment associations were learned. After learning, psilocybin decreased behavioral suppression in both sexes. Thus, psilocybin produces divergent effects on action suppression during approach-avoidance conflict depending on when the conflict is experienced. This observation may have implications for its therapeutic mechanism of action.

Lateral Orbitofrontal Cortex Encodes Presence of Risk and Subjective Risk Preference During Decision-Making

Adaptive decision-making requires consideration of objective risks and rewards associated with each option, as well as subjective preference for risky/safe alternatives. Inaccurate risk/reward estimations can engender excessive risk-taking, a central trait in many psychiatric disorders. The lateral orbitofrontal cortex (lOFC) has been linked to many disorders associated with excessively risky behavior and is ideally situated to mediate risky decision-making. Here, we used single-unit electrophysiology to measure neuronal activity from lOFC of freely moving rats performing in a punishment-based risky decision-making task. Subjects chose between a small, safe reward and a large reward associated with either 0% or 50% risk of concurrent punishment. lOFC activity repeatedly encoded current risk in the environment throughout the decision-making sequence, signaling risk before, during, and after a choice. In addition, lOFC encoded reward magnitude, although this information was only evident during action selection. A Random Forest classifier successfully used neural data accurately to predict the risk of punishment in any given trial, and the ability to predict choice via lOFC activity differentiated between and risk-preferring and risk-averse rats. Finally, risk preferring subjects demonstrated reduced lOFC encoding of risk and increased encoding of reward magnitude. These findings suggest lOFC may serve as a central decision-making hub in which external, environmental information converges with internal, subjective information to guide decision-making in the face of punishment risk.

Dopaminergic modulation of sensitivity to immediate and delayed punishment during decision-making

Effective decision-making involves careful consideration of all rewarding and aversive outcomes. Importantly, negative outcomes often occur later in time, leading to underestimation, or "discounting," of these consequences. Despite the frequent occurrence of delayed outcomes, little is known about the neurobiology underlying sensitivity to delayed punishment during decision-making. The Delayed Punishment Decision-making Task (DPDT) addresses this by assessing sensitivity to delayed versus immediate punishment in rats. Rats initially avoid punished reinforcers, then select this option more frequently when delay precedes punishment. We used DPDT to examine effects of acute systemic administration of catecholaminergic drugs on sensitivity to delayed punishment in male and female adult rats. Cocaine did not affect choice of rewards with immediate punishment but caused a dose-dependent reduction in choice of delayed punishment. Neither activation nor blockade of D1-like dopamine receptor affected decision-making, but activation of D2-like dopamine receptors reduced choice of delayed punishment. D2 blockade did not attenuate cocaine's effects on decision-making, suggesting that cocaine's effects are not dependent on D2 receptor activation. Increasing synaptic norepinephrine via atomoxetine also reduced choice of delayed (but not immediate) punishment. Notably, when DPDT was modified from ascending to descending pre-punishment delays, these drugs did not affect choice of delayed or immediate punishment, although high-dose quinpirole impaired behavioral flexibility. In summary, sensitivity to delayed punishment is regulated by both dopamine and norepinephrine transmission in task-specific fashion. Understanding the neurochemical modulation of decision-making with delayed punishment is a critical step toward treating disorders characterized by aberrant sensitivity to negative consequences.

Lateral Orbitofrontal Cortex and Basolateral Amygdala Regulate Sensitivity to Delayed Punishment during Decision-making

In real-world decision-making scenarios, negative consequences do not always occur immediately after a choice. This delay between action and outcome drives the underestimation, or "delay discounting", of punishment. While the neural substrates underlying sensitivity to immediate punishment have been well-studied, there has been minimal investigation of delayed consequences. Here, we assessed the role of lateral orbitofrontal cortex (LOFC) and basolateral amygdala (BLA), two regions implicated in cost/benefit decision-making, in sensitivity to delayed vs immediate punishment. The delayed punishment decision-making task (DPDT) was used to measure delay discounting of punishment in rodents. During DPDT, rats choose between a small, single pellet reward and a large, three pellet reward accompanied by a mild foot shock. As the task progresses, the shock is preceded by a delay that systematically increases or decreases throughout the session. We observed that rats avoid choices associated with immediate punishment, then shift preference toward these options when punishment is delayed. LOFC inactivation did not influence choice of rewards with immediate punishment, but decreased choice of delayed punishment. We also observed that BLA inactivation reduced choice of delayed punishment for ascending but not descending delays. Inactivation of either brain region produced comparable effects on decision-making in males and females, but there were sex differences observed in omissions and latency to make a choice. In summary, both LOFC and BLA contribute to the delay discounting of punishment and may serve as promising therapeutic targets to improve sensitivity to delayed punishment during decision-making.Significance StatementNegative consequences occurring after a delay are often underestimated, which can lead to maladaptive decision-making. While sensitivity to immediate punishment during reward-seeking has been well-studied, the neural substrates underlying sensitivity to delayed punishment remain unclear. Here, we used the Delayed Punishment Decision-making Task to determine that lateral orbitofrontal cortex and basolateral amygdala both regulate the discounting of delayed punishment, suggesting that these regions may be potential targets to improve decision-making in psychopathology.

Spontaneously hypertensive rats show higher impulsive action, but equal impulsive choice with both positive and aversive consequences

Both positive and aversive delayed consequences play an important role in decision making. However, most of research has studied the temporal discounting of the positive consequences, while the study of the aversives is scarce in general and null in some areas. This is the case of research on impulsivity in spontaneously hypertensive rats (SHR), an animal model of ADHD. To evaluate SHRs' sensitivity to aversive delayed consequences, we employed a choice procedure in which subjects chose between a smaller-amount alternative and a larger-amount alternative plus a shock; when preference for the smaller-amount alternative stabilized, five different delays to the shock were presented with the objective of analyzing the recovery of preference for the larger-amount alternative, which is related to the sensitivity to the delayed aversive consequence. To analyze the sensitivity to delayed positive consequences we employed a procedure that evaluated the preference between a smaller-amount alternative and a larger-amount alternative as a function of the delay to the later. Finally, to evaluate impulsive action we employed a DRL 10s schedule. In all tasks, Wistar rats were evaluated as control strain. The results indicated that choice impulsivity was equivalent between strains, both for positive and for aversive consequences. In contrast, we found a higher level of impulsive action in SHR.

Effects of the psychoactive compounds in green tea on risky decision-making

Epigallocatechin-3-gallate (EGCG) and caffeine are the two primary compounds found in green tea. While EGCG has anxiolytic and anti-inflammatory effects, its acute effects on cognition are not well understood. Furthermore, despite widespread green tea consumption, little is known about how EGCG and caffeine co-administration impacts behavior. Here, we investigated the effects of multiple doses of either EGCG or caffeine on a rat model of risk-taking. This was assessed using the risky decision-making task (RDT), in which rats choose between a small, well-tolerated reward and a large reward with escalating risk of mild footshock. Rats were tested in RDT after acute systemic administration of EGCG, caffeine or joint EGCG and caffeine. EGCG caused a dose-dependent reduction in risk-taking without affecting reward discrimination or task engagement. Caffeine did not impact risk-taking, but elevated locomotor activity and reduced task engagement at high doses. Finally, exposure to both EGCG and caffeine had no effect on risk-taking, suggesting that low-dose caffeine is sufficient to mask the risk-aversion caused by EGCG. These data suggest EGCG as a potential therapeutic treatment for psychological disorders that induce compulsive risky decision-making.

Dopamine receptors regulate preference between high-effort and high-risk rewards

RATIONALE

Optimal decision-making necessitates evaluation of multiple rewards that are each offset by distinct costs, such as high effort requirement or high risk of failure. The neurotransmitter dopamine is fundamental toward these cost-benefit analyses, and D1-like and D2-like dopamine receptors differently modulate the reward-discounting effects of both effort and risk. However, measuring the role of dopamine in regulating decision-making between options associated with distinct costs exceeds the scope of traditional rodent economic decision-making paradigms.

OBJECTIVES

We developed the effort vs probability economic conflict task (EvP) to model multimodal economic decision-making in rats. This task measures choice between two rewards of uniform magnitude associated with either a high effort requirement or risk of reward omission. We then tested the modulatory effects of systemic cocaine and D1/D2 blockade or activation on the preference between high-effort and high-risk alternatives.

METHODS

In the EvP, two reinforcers of equal magnitude are associated with either (1) an effort requirement that increases throughout the session (1, 5, 10, and 20 lever presses), or (2) a low probability of reward receipt (25% of probabilistic choices). Critically, the reinforcer for each choice is comparable (one pellet), which eliminates the influence of magnitude discrimination on the decision-making process. After establishing the task, the dopamine transporter blocker cocaine and D1/D2 antagonists and agonists were administered prior to EvP performance.

RESULTS

Preference shifted away from either effortful or probabilistic choice when either option became more costly, and this preference was highly variable between subjects and stable over time. Cocaine, D1 activation, and D2 blockade produced limited, dose-dependent shifts in choice preference contingent on high or low effort conditions. In contrast, D2 activation across multiple doses evoked a robust shift from effortful to risky choice that was evident even when clearly disadvantageous.

CONCLUSIONS

The EvP clearly demonstrates that rats can evaluate distinct effortful or risky costs associated with rewards of comparable magnitude, and shift preference away from either option with increasing cost. This preference is more tightly linked to D2 than D1 receptor manipulation, suggesting D2-like receptors as a possible therapeutic target for maladaptive biases toward risk-taking over effort.