Opposing effects of acute and chronic d-amphetamine on decision-making in rats

Activity in the Dorsomedial Striatum Underlies Serial Reversal Learning Performance Under Probabilistic Uncertainty

Background

Corticostriatal circuits, particularly the dorsomedial striatum (DMS) and lateral orbitofrontal cortex, are critical for navigating reversal learning under probabilistic uncertainty. These same areas are implicated in the reversal learning impairments observed in individuals with psychosis as well as their psychotic symptoms, suggesting that they may share a common neurobiological substrate. To address this question, we used psychostimulant exposure and specific activation of the DMS during reversal learning in mice to assess corticostriatal activity.

Methods

We used amphetamine treatment to induce psychosis-relevant neurobiology in male mice during reversal learning and to examine pathway-specific corticostriatal activation. To determine the causal role of DMS activity, we used chemogenetics to drive midbrain inputs during a range of probabilistic contingencies.

Results

Mice treated with amphetamine showed altered punishment learning, which was associated with decreased shifting after losses and increased perseverative errors after reversals. Reversal learning performance and strategies were dependent on increased activity in lateral orbitofrontal cortex to DMS circuits as well as in the DMS itself. Specific activation of midbrain to DMS circuits also decreased shifting after losses and reversal learning performance. However, these alterations were dependent on the probabilistic contingency.

Conclusions

Our work suggests that the DMS plays a multifaceted role in reversal learning. Increasing DMS activity impairs multiple reversal learning processes dependent on the level of uncertainty, confirming its role in the maintenance and selection of incoming cortical inputs. Together, these outcomes suggest that elevated dopamine levels in the DMS could contribute to decision-making impairments in individuals with psychosis.

Cannabis use is associated with sexually dimorphic changes in executive control of visuospatial decision-making

When the outcome of a choice is less favorable than expected, humans and animals typically shift to an alternate choice option on subsequent trials. Several lines of evidence indicate that this “lose-shift” responding is an innate sensorimotor response strategy that is normally suppressed by executive function. Therefore, the lose-shift response provides a covert gauge of cognitive control over choice mechanisms. We report here that the spatial position, rather than visual features, of choice targets drives the lose-shift effect. Furthermore, the ability to inhibit lose-shift responding to gain reward is different among male and female habitual cannabis users. Increased self-reported cannabis use was concordant with suppressed response flexibility and an increased tendency to lose-shift in women, which reduced performance in a choice task in which random responding is the optimal strategy. On the other hand, increased cannabis use in men was concordant with reduced reliance on spatial cues during decision-making, and had no impact on the number of correct responses. These data (63,600 trials from 106 participants) provide strong evidence that spatial-motor processing is an important component of economic decision-making, and that its governance by executive systems is different in men and women who use cannabis frequently.

Neural Circuitry of Salience and Reward Processing in Psychosis

The processing of salient and rewarding stimuli is integral to engaging our attention, stimulating anticipation for future events, and driving goal-directed behaviors. Widespread impairments in these processes are observed in psychosis, which may be associated with worse functional outcomes or mechanistically linked to the development of symptoms. Here, we summarize the current knowledge of behavioral and functional neuroimaging in salience, prediction error, and reward. Although each is a specific process, they are situated in multiple feedback and feedforward systems integral to decision making and cognition more generally. We argue that the origin of salience and reward processing dysfunctions may be centered in the subcortex during the earliest stages of psychosis, with cortical abnormalities being initially more spared but becoming more prominent in established psychotic illness/schizophrenia. The neural circuits underpinning salience and reward processing may provide targets for delaying or preventing progressive behavioral and neurobiological decline.

Amphetamine reduces reward encoding and stabilizes neural dynamics in rat anterior cingulate cortex

Psychostimulants such as d-amphetamine (AMPH) often have behavioral effects that appear paradoxical within the framework of optimal choice theory. AMPH typically increases task engagement and the effort animals exert for reward, despite decreasing reward valuation. We investigated neural correlates of this phenomenon in the anterior cingulate cortex (ACC), a brain structure implicated in signaling cost-benefit utility. AMPH decreased signaling of reward, but not effort, in the ACC of freely-moving rats. Ensembles of simultaneously recorded neurons generated task-specific trajectories of neural activity encoding past, present, and future events. Low-dose AMPH contracted these trajectories and reduced their variance, whereas high-dose AMPH expanded both. We propose that under low-dose AMPH, increased network stability balances moderately increased excitability, which promotes accelerated unfolding of a neural 'script' for task execution, despite reduced reward valuation. Noise from excessive excitability at high doses overcomes stability enhancement to drive frequent deviation from the script, impairing task execution.

Lesions of lateral habenula attenuate win-stay but not lose-shift responses in a competitive choice task

Multiple neural systems contribute to choice adaptation following reinforcement. Recent evidence suggests that the lateral habenula (LHb) plays a key role in such adaptations, particularly when reinforcements are worse than expected. Here, we investigated the effects of bilateral LHb lesions on responding in a binary choice task with no discriminatory cues. LHb lesions in rats decreased win-stay responses but surprisingly left lose-shift responses intact. This same dissociated effect was also observed after systemic administration of d-amphetamine in a separate cohort of animals. These results suggest that at least some behavioural responses triggered by reward omission do not depend on an intact LHb.

Implicit Valuation of the Near-Miss is Dependent on Outcome Context

Gambling studies have described a "near-miss effect" wherein the experience of almost winning increases gambling persistence. The near-miss has been proposed to inflate the value of preceding actions through its perceptual similarity to wins. We demonstrate here, however, that it acts as a conditioned stimulus to positively or negatively influence valuation, dependent on reward expectation and cognitive engagement. When subjects are asked to choose between two simulated slot machines, near-misses increase valuation of machines with a low payout rate, whereas they decrease valuation of high payout machines. This contextual effect impairs decisions and persists regardless of manipulations to outcome feedback or financial incentive provided for good performance. It is consistent with proposals that near-misses cause frustration when wins are expected, and we propose that it increases choice stochasticity and overrides avoidance of low-valued options. Intriguingly, the near-miss effect disappears when subjects are required to explicitly value machines by placing bets, rather than choosing between them. We propose that this task increases cognitive engagement and recruits participation of brain regions involved in cognitive processing, causing inhibition of otherwise dominant systems of decision-making. Our results reveal that only implicit, rather than explicit strategies of decision-making are affected by near-misses, and that the brain can fluidly shift between these strategies according to task demands.

Sex differences in rat decision-making: The confounding role of extraneous feeder sampling between trials

Although male and female rats appear to perform differently in some tasks, a clear picture of sex differences in decision-making has yet to develop. This is in part due to significant variability arising from differences in strains and tasks. The aim of this study was to characterize the effects of sex on specific response elements in a reinforcement learning task so as to help identify potential explanations for this variability. We found that the primary difference between sexes was the propensity to approach feeders out of the task context. This extraneous feeder sampling affects choice on subsequent trials in both sexes by promoting a lose-shift response away from the last feeder sampled. Female rats, however, were more likely to engage in this extraneous feeder sampling, and therefore exhibited a greater rate of this effect. Once trials following extraneous sampling were removed, there were no significant sex differences in any of the tested measures. These data suggest that feeder approach outside of the task context, which is often not recorded, could produce a confound in sex-based differences of reinforcement sensitivity in some tasks.

Feeder Approach between Trials Is Increased by Uncertainty and Affects Subsequent Choices

Animals quickly learn to approach sources of food. Here, we report on a form of approach in which rats made volitional orofacial contact with inactive feeders between trials of a self-paced operant task. This extraneous feeder sampling (EFS) was never reinforced and therefore imposed an opportunity and effort cost. EFS decreased during initial training but persisted thereafter. The relative rate of EFS to operant responding increased with novel changes to the operant chamber, reward devaluation by prefeeding, or lesions to the dorsolateral striatum. We speculate that this may function to increase exploration when the task is uncertain (early in learning or introduction of novel apparatus components), when the opportunity cost is low, or when the learned sensorimotor solution is compromised. Moreover, EFS strongly affected subsequent choices by triggering a lose-shift response away from the sampled feeder, even though it occurred outside of the trial context. This indicates that at least some behaviors occurring between trials impact future behaviors and should be considered in decision-making studies.

Acute Δ-9-tetrahydrocannabinol administration in female rats attenuates immediate responses following losses but not multi-trial reinforcement learning from wins

Δ-9-Tetrahydrocannabinol (THC) is the main psychoactive component of marijuana and has potent effects on decision-making, including a proposed reduction in cognitive flexibility. We demonstrate here that acute THC administration differentially affects some of the processes that contribute to cognitive flexibility. Specifically, THC reduces lose-shift responding in which female rats tend to immediately shift choice responses away from options that result in reward omission on the previous trial. THC, however, did not impair the ability of rats to flexibly bias responses toward feeders with higher probability of reward in a reversal task. This response adaptation developed over several trials, suggesting that THC did not impair slower forms of reinforcement learning needed to choose among options with unequal utility. This dissociation of THC's effects on innate/rapid and learned/gradual decision-making processes was unexpected, but is supported by emerging evidence that lose-shift responding is mediated by neural mechanisms distinct from those involved in other forms of reinforcement learning. The present data suggest that, at least in some tasks, the apparent reductions in cognitive flexibility by THC may be explained by the immediate effects on loss sensitivity, rather than impairments of all processes used for choice adaptation.

The Role of Dopamine D1 and D3 Receptors in N-Methyl-D-Aspartate (NMDA)/GlycineB Site-Regulated Complex Cognitive Behaviors following Repeated Morphine Administration

BACKGROUND

Opiate addiction is associated with complex cognitive impairment, which contributes to the development of compulsive drug use and relapses. Dopamine and N-methyl-D-aspartate receptors play critical roles in opiate-induced cognitive deficits. However, the roles of D1 and D3 receptors in the N-methyl-D-aspartate/glycineB receptor-regulated cognitive behaviors induced by morphine remain unknown.

METHODS

The 5-choice serial reaction time task was used to investigate the cognitive profiles associated with repeated morphine administration in D1 (D1-/-)- and D3 (D3-/-)-receptor knockout mice. The expression of phosphorylated NR1, Ca2+/calmodulin-dependent protein kinase II (CaMKII), and cAMP response element-binding protein (CREB) in the brain was examined by western blotting. D1-/- and D3-/- mice were treated with the N-methyl-D-aspartate/glycineB site agonist l-aminocyclopropanecarboxylic acid and the antagonist L-701,324 to chronically disrupt N-methyl-D-aspartate receptor function and investigate their effects on morphine-induced cognitive changes.

RESULTS

Repeated morphine administration impaired attentional function and caused impulsive and compulsive behaviors. D1-/- mice exhibited hardly any premature nosepokes. D3-/- mice showed robustly increased morphine-induced impulsive behavior. The numbers of premature responses were decreased by L-701,324 administration and increased by ACPC administration; these effects were completely abolished in D1-/- mice due to their inability to perform reward-based tasks. In contrast, the inhibitory effects of L-701,324 on impulsive behavior were significantly augmented in D3-/- mice.

CONCLUSIONS

N-methyl-D-aspartate/glycineB site functions may contribute to morphine-induced cognitive deficits, especially those related to impulsive behavior. D1 and D3 receptors may have contrasting effects with respect to modulating impulsive behavior. D3 receptors have inhibitory effects on impulsive behaviors, and these effects are clearly mediated by N-methyl-D-aspartate/glycineB receptor and μ-opioid receptor interactions.