Cocaine makes actions insensitive to outcomes but not extinction: implications for altered orbitofrontal-amygdalar function

Cocaine disrupts action flexibility via glucocorticoid receptors

Many addictive drugs increase stress hormone levels. They also alter the propensity of organisms to prospectively select actions based on long-term consequences. We hypothesized that cocaine causes inflexible action by increasing circulating stress hormone levels, activating the glucocorticoid receptor (GR). We trained mice to generate two nose pokes for food and then required them to update action-consequence associations when one response was no longer reinforced. Cocaine delivered in adolescence or adulthood impaired the capacity of mice to update action strategies, and inhibiting CORT synthesis rescued action flexibility. Next, we reduced Nr3c1, encoding GR, in the orbitofrontal cortex (OFC), a region of the brain responsible for interlacing new information into established routines. Nr3c1 silencing preserved action flexibility and dendritic spine abundance on excitatory neurons, despite cocaine. Spines are often considered substrates for learning and memory, leading to the discovery that cocaine degrades the representation of new action memories, obstructing action flexibility.

Impairments in expression of devaluation in a Pavlovian goal-tracking task, but not a free operant devaluation task, after fentanyl exposure in female rats

In laboratory animals, there are numerous demonstrations that past exposure to drugs of abuse can lead to devaluation impairments weeks after the final drug exposure, with the majority of these demonstrations examining effects of exposure to psychostimulants. There has been minimal investigation into whether prior exposure to opiates can lead to devaluation impairments. Here, we first trained female rats that two separate cuelights predicted two different foods and measured Pavlovian goal-tracking responses (Experiment 1) or trained female rats to press two levers to earn two different foods and measured this operant response (Experiment 2). In both experiments, we subsequently gave the rats injections of fentanyl twice daily for 6 days, and then tested rats for conditioned responses after satiation on one of the foods 48-h after the final injection. We found that rats were impaired in the expression of devaluation in the Pavlovian task after fentanyl exposure, but were unimpaired in the expression of devaluation in the operant task. The pattern of results is most consistent with an impairment in lateral orbitofrontal cortex function, but additional research is needed to determine the neurobiological cause of this pattern of results.

Sex-biased effects of outcome devaluation by sensory-specific satiety on Pavlovian-conditioned behavior

Goal-directed behavior relies on accurate mental representations of the value of expected outcomes. Disruptions to this process are a central feature of several neuropsychiatric disorders, including addiction. Goal-directed behavior is most frequently studied using instrumental paradigms paired with outcome devaluation, but cue-evoked behaviors in Pavlovian settings can also be goal-directed and therefore sensitive to changes in outcome value. Emerging literature suggests that male and female rats may differ in the degree to which their Pavlovian-conditioned responses are goal-directed, but interpretation of these findings is complicated by the tendency of female and male rats to engage in distinct types of Pavlovian responses when trained with localizable cues. Here, we used outcome devaluation via sensory-specific satiety to assess the behavioral responses in male and female Long Evans rats trained to respond to an auditory CS (conditioned stimulus) in a Pavlovian-conditioning paradigm. We found that satiety-induced devaluation led to a decrease in behavioral responding to the reward-predictive CS, with males showing an effect on both port entry latency and probability and females showing an effect only on port entry probability. Overall, our results suggest that outcome devaluation affects Pavlovian-conditioned responses in both male and female rats, but that females may be less sensitive to outcome devaluation.

Subunit-selective PI3-kinase control of action strategies in the medial prefrontal cortex

PI3-kinase (PI3K) is an intracellular signaling complex that is stimulated upon cocaine exposure and linked with the behavioral consequences of cocaine. We recently genetically silenced the PI3K p110β subunit in the medial prefrontal cortex following repeated cocaine in mice, reinstating the capacity of these mice to engage in prospective goal-seeking behavior. In the present short report, we address two follow-up hypotheses: 1) The control of decision-making behavior by PI3K p110β is attributable to neuronal signaling, and 2) PI3K p110β in the healthy (i.e., drug-naïve) medial prefrontal cortex has functional consequences in the control of reward-related decision-making strategies. In Experiment 1, we found that silencing neuronal p110β improved action flexibility following cocaine. In Experiment 2, we reduced PI3K p110β in drug-naïve mice that were extensively trained to respond for food reinforcers. Gene silencing caused mice to abandon goal-seeking strategies, unmasking habit-based behaviors that were propelled by interactions with the nucleus accumbens. Thus, PI3K control of goal-directed action strategies appears to act in accordance with an inverted U-shaped function, with "too much" (following cocaine) or "too little" (following p110β subunit silencing) obstructing goal seeking and causing mice to defer to habit-like response sequences.

Intrusive thinking: Circuit and synaptic mechanisms of a transdiagnostic psychiatric symptom

Spontaneous thought is an adaptive cognitive process that can produce novel and insightful thought sequences useful in guiding future behavior. In many psychiatric disorders, spontaneous thinking becomes intrusive and uncontrolled, and can trigger symptoms such as craving, repetitive negative thinking and trauma-related memories. We link studies using clinical imaging and rodent modeling towards understanding the neurocircuitry and neuroplasticity of intrusive thinking. We propose a framework in which drugs or stress change the homeostatic set point of brain reward circuitry, which then impacts subsequent plasticity induced by drug/stress conditioned cues (metaplastic allostasis). We further argue for the importance of examining not only the canonical pre- and postsynapse, but also the adjacent astroglial protrusions and extracellular matrix that together form the tetrapartite synapse and that plasticity throughout the tetrapartite synapse is necessary for cue-induced drug or stress behaviors. This analysis reveals that drug use or trauma cause long-lasting allostatic brain plasticity that sets the stage for subsequent drug/trauma-associated cues to induce transient plasticity that can lead to intrusive thinking.

Prelimbic cortex neural encoding dynamically tracks expected outcome value

Animals must modify their behavior based on updated expected outcomes in a changing environment. Prelimbic cortex (PrL) neural encoding during learning predicts, and is necessary for, appropriately altering behavior based on a new expected outcome value following devaluation. We aimed to determine how PrL neural activity encodes reward predictive cues after the expected outcome value of those cues is decreased following conditioned taste aversion. In one post-devaluation session, rats were tested under extinction to determine their ability to alter their behavior to the expected outcome values (i.e., extinction test). In a second post-devaluation session, rats were tested with the newly devalued outcome delivered so that the rats experienced the updated outcome value within the session (i.e., re-exposure test). We found that PrL neural encoding of the cue associated with the devalued reward predicted the ability of rats to suppress behavior in the extinction test session, but not in the re-exposure test session. While all rats were able to successfully devalue the outcome during conditioned taste aversion, a subset of rats continued to consume the devalued outcome in the re-exposure test session. We found differential patterns of PrL neural encoding in the population of rats that did not avoid the devalued outcome during the re-exposure test compared to the rats that successfully avoided the devalued outcome. Our findings suggest that PrL neural encoding dynamically tracks expected outcome values, and differential neural encoding in the PrL to reward predictive cues following expected outcome value changes may contribute to distinct behavioral phenotypes.

Striatal dopamine signals are region specific and temporally stable across action-sequence habit formation

Habits are automatic, inflexible behaviors that develop slowly with repeated performance. Striatal dopamine signaling instantiates this habit-formation process, presumably region specifically and via ventral-to-dorsal and medial-to-lateral signal shifts. Here, we quantify dopamine release in regions implicated in these presumed shifts (ventromedial striatum [VMS], dorsomedial striatum [DMS], and dorsolateral striatum [DLS]) in rats performing an action-sequence task and characterize habit development throughout a 10-week training. Surprisingly, all regions exhibited stable dopamine dynamics throughout habit development. VMS and DLS signals did not differ between habitual and non-habitual animals, but DMS dopamine release increased during action-sequence initiation and decreased during action-sequence completion in habitual rats, whereas non-habitual rats showed opposite effects. Consistently, optogenetic stimulation of DMS dopamine release accelerated habit formation. Thus, we demonstrate that dopamine signals do not shift regionally during habit formation and that dopamine in DMS, but not VMS or DLS, determines habit bias, attributing "habit functions" to a region previously associated exclusively with non-habitual behavior.

Linking drug and food addiction via compulsive appetite

BACKGROUND AND PURPOSE

"Food addiction" is the subject of intense public and research interest. However, this nosology based on neurobehavioral similarities among obese individuals and patients with eating disorders and drug addiction remains controversial. We thus sought to determine which aspects of disordered eating are causally linked to preclinical models of drug addiction. We hypothesized that extensive drug histories, known to cause addiction-like brain changes and drug motivation in rats, would also cause addiction-like food motivation.

EXPERIMENTAL APPROACH

Rats underwent extensive cocaine, alcohol, caffeine or obesogenic diet histories, and were subsequently tested for punishment-resistant food self-administration or "compulsive appetite", as a measure of addiction-like food motivation.

KEY RESULTS

Extensive cocaine and alcohol (but not caffeine) histories caused compulsive appetite that persisted long after the last drug exposure. Extensive obesogenic diet histories also caused compulsive appetite, although neither cocaine nor alcohol histories caused excess calorie intake and bodyweight during abstinence. Hence, compulsive appetite and obesity appear to be dissociable, with the former sharing common mechanisms with preclinical drug addiction models.

CONCLUSION AND IMPLICATIONS

Compulsive appetite, as seen in subsets of obese individuals and patients with binge-eating disorder and bulimia nervosa (eating disorders that do not necessarily result in obesity), appears to epitomize "food addiction". Because different drug and obesogenic diet histories caused compulsive appetite, overlapping dysregulations in the reward circuits, which control drug and food motivation independently of energy homeostasis, may offer common therapeutic targets for treating addictive behaviors across drug addiction, eating disorders and obesity.

Impaired Learning From Negative Feedback in Stimulant Use Disorder: Dopaminergic Modulation

BACKGROUND

Drug-induced alterations to the dopamine system in stimulant use disorder (SUD) are hypothesized to impair reinforcement learning (RL). Computational modeling enables the investigation of the latent processes of RL in SUD patients, which could elucidate the nature of their impairments.

METHODS

We investigated RL in 44 SUD patients and 41 healthy control participants using a probabilistic RL task that assesses learning from reward and punishment separately. In an independent sample, we determined the modulatory role of dopamine in RL following a single dose of the dopamine D2/3 receptor antagonist amisulpride (400 mg) and the agonist pramipexole (0.5 mg) in a randomised, double-blind, placebo-controlled, crossover design. We analyzed task performance using computational modelling and hypothesized that RL impairments in SUD patients would be differentially modulated by a dopamine D2/3 receptor antagonist and agonist.

RESULTS

Computational analyses in both samples revealed significantly reduced learning rates from punishment in SUD patients compared with healthy controls, whilst their reward learning rates were not measurably impaired. In addition, the dopaminergic receptor agents modulated RL parameters differentially in both groups. Both amisulpride and pramipexole impaired RL parameters in healthy participants, but ameliorated learning from punishment in SUD patients.

CONCLUSION

Our findings suggest that RL impairments seen in SUD patients are associated with altered dopamine function.

Choose your path: Divergent basolateral amygdala efferents differentially mediate incentive motivation, flexibility and decision-making

To survive in a complex environment, individuals form associations between environmental stimuli and rewards to organize and optimize reward seeking behaviors. The basolateral amygdala (BLA) uses these learned associations to inform decision-making processes. In this review, we describe functional projections between BLA and its cortical and striatal targets that promote learning and motivational processes central to decision-making. Specifically, we compare and contrast divergent projections from the BLA to the orbitofrontal (OFC) and to the nucleus accumbens (NAc) and examine the roles of these pathways in associative learning, value-guided decision-making, choice behaviors, as well as cue and context-driven drug seeking. Finally, we consider how these projections are involved in disorders of motivation, with a focus on Substance Use Disorder.

Effects of Withdrawal from Cocaine Self-Administration on Rat Orbitofrontal Cortex Parvalbumin Neurons Expressing Cre recombinase: Sex-Dependent Changes in Neuronal Function and Unaltered Serotonin Signaling

The orbitofrontal cortex (OFC) is a brain region involved in higher-order decision-making. Rodent studies show that cocaine self-administration (CSA) reduces OFC contribution to goal-directed behavior and behavioral strategies to avoid drug intake. This change in OFC function persists for many weeks after cocaine withdrawal, suggesting involvement in the process of addiction. The mechanisms underlying impaired OFC function by cocaine are not well-understood. However, studies implicate altered OFC serotonin (5-HT) function in disrupted cognitive processes during addiction and other psychiatric disorders. Thus, it is hypothesized that cocaine impairment of OFC function involves changes in 5-HT signaling, and previous work shows that 5-HT_1A and 5-HT_2A receptor-mediated effects on OFC pyramidal neurons (PyNs) are impaired weeks after cocaine withdrawal. However, 5-HT effects on other contributors to OFC circuit function have not been fully investigated, including the parvalbumin-containing, fast-spiking interneurons (OFC^PV), whose function is essential to normal OFC-mediated behavior. Here, 5-HT function in naive rats and those withdrawn from CSA were evaluated using a novel rat transgenic line in which the rat parvalbumin promoter drives Cre-recombinase expression to permit identification of OFC^PV cells by fluorescent reporter protein expression. We find that whereas CSA altered basal synaptic and membrane properties of the OFC^PV neurons in a sex-dependent manner, the effects of 5-HT on these cells were unchanged by CSA. These data suggest that the behavioral effects of dysregulated OFC 5-HT function caused by cocaine experience are primarily mediated by changes in 5-HT signaling at PyNs, and not at OFC^PV neurons.

Reduced Glutamate Turnover in the Putamen Is Linked With Automatic Habits in Human Cocaine Addiction

BACKGROUND

The balance between goal-directed behavior and habits has been hypothesized to be biased toward the latter in individuals with cocaine use disorder (CUD), suggesting possible neurochemical changes in the putamen, which may contribute to their compulsive behavior.

METHODS

We assessed habitual behavior in 48 patients with CUD and 42 healthy control participants using a contingency degradation paradigm and the Creature of Habit Scale. In a subgroup of this sample (CUD: n = 21; control participants: n = 22), we also measured glutamate and glutamine concentrations in the left putamen using ultra-high-field (7T) magnetic resonance spectroscopy. We hypothesized that increased habitual tendencies in patients with CUD would be associated with abnormal glutamatergic metabolites in the putamen.

RESULTS

Compared with their non-drug-using peers, patients with CUD exhibited greater habitual tendencies during contingency degradation, which correlated with increased levels of self-reported daily habits. We further identified a significant reduction in glutamate concentration and glutamate turnover (glutamate-to-glutamine ratio) in the putamen in patients with CUD, which was significantly related to the level of self-reported daily habits.

CONCLUSIONS

Patients with CUD exhibit enhanced habitual behavior, as assessed both by questionnaire and by a laboratory paradigm of contingency degradation. This automatic habitual tendency is related to a reduced glutamate turnover in the putamen, suggesting a dysregulation of habits caused by chronic cocaine use.

Noninvasive Brain Stimulation Rescues Cocaine-Induced Prefrontal Hypoactivity and Restores Flexible Behavior

BACKGROUND

To obtain desirable goals, individuals must predict the outcome of specific choices, use that information to direct appropriate actions, and adjust behavior accordingly in changing environments (behavioral flexibility). Substance use disorders are marked by impairments in behavioral flexibility along with decreased prefrontal cortical function that limits the efficacy of treatment strategies. Restoring prefrontal hypoactivity, ideally in a noninvasive manner, is an intriguing target for improving flexible behavior and treatment outcomes.

METHODS

A behavioral flexibility task was used in Long-Evans male rats (n = 97) in conjunction with electrophysiology, optogenetics, and a novel rat model of transcranial alternating current stimulation (tACS) to examine the prelimbic cortex (PrL) to nucleus accumbens (NAc) core circuit in behavioral flexibility and determine whether tACS can restore cocaine-induced neural and cognitive dysfunction.

RESULTS

Optogenetic inactivation revealed that the PrL-NAc core circuit is necessary for the ability to learn strategies to flexibly shift behavior. Cocaine self-administration history caused aberrant PrL-NAc core neural encoding and deficits in flexibility. Optogenetics that selectively activated the PrL-NAc core pathway prior to learning rescued cocaine-induced cognitive flexibility deficits. Remarkably, tACS prior to learning the task reestablished adaptive signaling in the PrL-NAc circuit and restored flexible behavior in a relatively noninvasive and frequency-specific manner.

CONCLUSIONS

We establish a role of NAc core-projecting PrL neurons in behavioral flexibility and provide a novel noninvasive brain stimulation method in rats to rescue cocaine-induced frontal hypofunction and restore flexible behavior, supporting a role of tACS as a therapeutic to treat cognitive deficits in substance use disorders.

Leveraging Basic Science for the Clinic-From Bench to Bedside

Importance

The tools and insights of behavioral neuroscience grow apace, yet their clinical application is lagging.

Observations

This article suggests that associative learning theory may be the algorithmic bridge to connect a burgeoning understanding of the brain with the challenges to the mind with which all clinicians and researchers are concerned.

Conclusions and Relevance

Instead of giving up, talking past one another, or resting on the laurels of face validity, a consilient and collaborative approach is suggested: visiting laboratory meetings and clinical rounds and attempting to converse in the language of behavior and cognition to better understand and ultimately treat patients.

Prior Cocaine Use Alters the Normal Evolution of Information Coding in Striatal Ensembles during Value-Guided Decision-Making

Substance use disorders (SUDs) are characterized by maladaptive behavior. The ability to properly adjust behavior according to changes in environmental contingencies necessitates the interlacing of existing memories with updated information. This can be achieved by assigning learning in different contexts to compartmentalized "states." Though not often framed this way, the maladaptive behavior observed in individuals with SUDs may result from a failure to properly encode states because of drug-induced neural alterations. Previous studies found that the dorsomedial striatum (DMS) is important for behavioral flexibility and state encoding, suggesting the DMS may be an important substrate for these effects. Here, we recorded DMS neural activity in cocaine-experienced male rats during a decision-making task where blocks of trials represented distinct states to probe whether the encoding of state and state-related information is affected by prior drug exposure. We found that DMS medium spiny neurons (MSNs) and fast-spiking interneurons (FSIs) encoded such information and that prior cocaine experience disrupted the evolution of representations both within trials and across recording sessions. Specifically, DMS MSNs and FSIs from cocaine-experienced rats demonstrated higher classification accuracy of trial-specific rules, defined by response direction and value, compared with those drawn from sucrose-experienced rats, and these overly strengthened trial-type representations were related to slower switching behavior and reaction times. These data show that prior cocaine experience paradoxically increases the encoding of state-specific information and rules in the DMS and suggest a model in which abnormally specific and persistent representation of rules throughout trials in DMS slows value-based decision-making in well trained subjects.SIGNIFICANCE STATEMENT Substance use disorders (SUDs) may result from a failure to properly encode rules guiding situationally appropriate behavior. The dorsomedial striatum (DMS) is thought to be important for such behavioral flexibility and encoding that defines the situation or "state." This suggests that the DMS may be an important substrate for the maladaptive behavior observed in SUDs. In the current study, we show that prior cocaine experience results in over-encoding of state-specific information and rules in the DMS, which may impair normal adaptive decision-making in the task, akin to what is observed in SUDs.

Habitual Preference for the Nondrug Reward in a Drug Choice Setting

For adaptive and efficient decision making, it must be possible to select between habitual alternative courses of action. However, research in rodents suggests that, even in the context of simple decision-making, choice behavior remains goal-directed. In contrast, we recently found that during discrete trial choice between cocaine and water, water-restricted rats preferred water and this preference was habitual and inflexible (i.e., resistant to water devaluation by satiation). Here we sought to test the reproducibility and generality of this surprising finding by assessing habitual control of preference for saccharin over cocaine in non-restricted rats. Specifically, after the acquisition of preference for saccharin, saccharin was devalued and concurrent responding for both options was measured under extinction. As expected, rats responded more for saccharin than for cocaine during extinction, but this difference was unaffected by saccharin devaluation. Together with our previous research, this result indicates that preference for nondrug alternatives over cocaine is under habitual control, even under conditions that normally support goal-directed control of choice between nondrug options. The possible reasons for this difference are discussed.

Substance use is associated with reduced devaluation sensitivity

Substance use has been linked to impairments in reward processing and decision-making, yet empirical research on the relationship between substance use and devaluation of reward in humans is limited. We report findings from two studies that tested whether individual differences in substance use behavior predicted reward learning strategies and devaluation sensitivity in a nonclinical sample. Participants in Experiment 1 (N = 66) and Experiment 2 (N = 91) completed subscales of the Externalizing Spectrum Inventory and then performed a two-stage reinforcement learning task that included a devaluation procedure. Spontaneous eye blink rate was used as an indirect proxy for dopamine functioning. In Experiment 1, correlational analysis revealed a negative relationship between substance use and devaluation sensitivity. In Experiment 2, regression modeling revealed that while spontaneous eyeblink rate moderated the relationship between substance use and reward learning strategies, substance use alone was related to devaluation sensitivity. These results suggest that once reward-action associations are established during reinforcement learning, substance use predicted reduced sensitivity to devaluation independently of variation in eyeblink rate. Thus, substance use is not only related to increased habit formation but also to difficulty disengaging from learned habits. Implications for the role of the dopaminergic system in habitual responding in individuals with substance use problems are discussed.

Addiction is driven by excessive goal-directed drug choice under negative affect: translational critique of habit and compulsion theory

Drug addiction may be a goal-directed choice driven by excessive drug value in negative affective states, a habit driven by strong stimulus-response associations, or a compulsion driven by insensitivity to costs imposed on drug seeking. Laboratory animal and human evidence for these three theories is evaluated. Excessive goal theory is supported by dependence severity being associated with greater drug choice/economic demand. Drug choice is demonstrably goal-directed (driven by the expected value of the drug) and can be augmented by stress/negative mood induction and withdrawal-effects amplified in those with psychiatric symptoms and drug use coping motives. Furthermore, psychiatric symptoms confer risk of dependence, and coping motives mediate this risk. Habit theory of addiction has weaker support. Habitual behaviour seen in drug-exposed animals often does not occur in complex decision scenarios, or where responding is rewarded, so habit is unlikely to explain most human addictive behaviour where these conditions apply. Furthermore, most human studies have not found greater propensity to habitual behaviour in drug users or as a function of dependence severity, and the minority that have can be explained by task disengagement producing impaired explicit contingency knowledge. Compulsion theory of addiction also has weak support. The persistence of punished drug seeking in animals is better explained by greater drug value (evinced by the association with economic demand) than by insensitivity to costs. Furthermore, human studies have provided weak evidence that propensity to discount cost imposed on drug seeking is associated with dependence severity. These data suggest that human addiction is primarily driven by excessive goal-directed drug choice under negative affect, and less by habit or compulsion. Addiction is pathological because negative states powerfully increase expected drug value acutely outweighing abstinence goals.

Withdrawal from escalated cocaine self-administration impairs reversal learning by disrupting the effects of negative feedback on reward exploitation: a behavioral and computational analysis

Addiction is regarded as a disorder of inflexible choice with behavior dominated by immediate positive rewards over longer-term negative outcomes. However, the psychological mechanisms underlying the effects of self-administered drugs on behavioral flexibility are not well understood. To investigate whether drug exposure causes asymmetric effects on positive and negative outcomes we used a reversal learning procedure to assess how reward contingencies are utilized to guide behavior in rats previously exposed to intravenous cocaine self-administration (SA). Twenty-four rats were screened for anxiety in an open field prior to acquisition of cocaine SA over six daily sessions with subsequent long-access cocaine SA for 7 days. Control rats (n = 24) were trained to lever-press for food under a yoked schedule of reinforcement. Higher rates of cocaine SA were predicted by increased anxiety and preceded impaired reversal learning, expressed by a decrease in lose-shift as opposed to win-stay probability. A model-free reinforcement learning algorithm revealed that rats with high, but not low cocaine escalation failed to exploit previous reward learning and were more likely to repeat the same response as the previous trial. Eight-day withdrawal from high cocaine escalation was associated, respectively, with increased and decreased dopamine receptor D2 (DRD2) and serotonin receptor 2C (HTR2C) expression in the ventral striatum compared with controls. Dopamine receptor D1 (DRD1) expression was also significantly reduced in the orbitofrontal cortex of high cocaine-escalating rats. These findings indicate that withdrawal from escalated cocaine SA disrupts how negative feedback is used to guide goal-directed behavior for natural reinforcers and that trait anxiety may be a latent variable underlying this interaction.

Hippocampal BDNF regulates a shift from flexible, goal-directed to habit memory system function following cocaine abstinence

The transition from recreational drug use to addiction involves pathological learning processes that support a persistent shift from flexible, goal-directed to habit behavioral control. Here, we examined the molecular mechanisms supporting altered function in hippocampal (HPC) and dorsolateral striatum (DLS) memory systems following abstinence from repeated cocaine. After 3 weeks of cocaine abstinence (experimenter- or self-administered), we tested new behavioral learning in male rats using a dual-solution maze task, which provides an unbiased approach to assess HPC- versus DLS-dependent learning strategies. Dorsal hippocampus (dHPC) and DLS brain tissues were collected after memory testing to identify transcriptional adaptations associated with cocaine-induced shifts in behavioral learning. Our results demonstrate that following prolonged cocaine abstinence rats show a bias toward the use of an inflexible, habit memory system (DLS) in lieu of a more flexible, easily updated memory system involving the HPC. This memory system bias was associated with upregulation and downregulation of brain-derived neurotrophic factor (BDNF) gene expression and transcriptionally permissive histone acetylation (acetylated histone H3, AcH3) in the DLS and dHPC, respectively. Using viral-mediated gene transfer, we overexpressed BDNF in the dHPC during cocaine abstinence and new maze learning. This manipulation restored HPC-dependent behavioral control. These findings provide a system-level understanding of altered plasticity and behavioral learning following cocaine abstinence and inform mechanisms mediating the organization of learning and memory more broadly.

Expectancy-Related Changes in Dopaminergic Error Signals Are Impaired by Cocaine Self-Administration

Addiction is a disorder of behavioral control and learning. While this may reflect pre-existing propensities, drug use also clearly contributes by causing changes in outcome processing in prefrontal and striatal regions. This altered processing is associated with behavioral deficits, including changes in learning. These areas provide critical input to midbrain dopamine neurons regarding expected outcomes, suggesting that effects on learning may result from changes in dopaminergic error signaling. Here, we show that dopamine neurons recorded in rats that had self-administered cocaine failed to suppress firing on omission of an expected reward and exhibited lower amplitude and imprecisely timed increases in firing to an unexpected reward. Learning also appeared to have less of an effect on reward-evoked and cue-evoked firing in the cocaine-experienced rats. Overall, the changes are consistent with reduced fidelity of input regarding the expected outcomes, such as their size, timing, and overall value, because of cocaine use.

The role of the orbitofrontal cortex in alcohol use, abuse, and dependence

One of the major functions of the orbitofrontal cortex (OFC) is to promote flexible motivated behavior. It is no surprise, therefore, that recent work has demonstrated a prominent impact of chronic drug use on the OFC and a potential role for OFC disruption in drug abuse and addiction. Among drugs of abuse, the use of alcohol is particularly salient with respect to OFC function. Although a number of studies in humans have implicated OFC dysregulation in alcohol use disorders, animal models investigating the association between OFC and alcohol use are only beginning to be developed, and there is still a great deal to be revealed. The goal of this review is to consider what is currently known regarding the role of the OFC in alcohol use and dependence. I will first provide a brief, general overview of current views of OFC function and its contributions to drug seeking and addiction. I will then discuss research to date related to the OFC and alcohol use, both in human clinical populations and in non-human models. Finally I will consider issues and strategies to guide future study that may identify this brain region as a key player in the transition from moderated to problematic alcohol use and dependence.

Corticolimbic circuitry in the modulation of chronic pain and substance abuse

The transition from acute to chronic pain is accompanied by increased engagement of emotional and motivational circuits. Adaptations within this corticolimbic circuitry contribute to the cellular and behavioral maladaptations associated with chronic pain. Central regions within the corticolimbic brain include the mesolimbic dopamine system, the amygdala, and the medial prefrontal cortex. The evidence reviewed herein supports the notion that chronic pain induces significant changes within these corticolimbic regions that contribute to the chronicity and intractability of pain. In addition, pain-induced changes in corticolimbic circuitry are poised to impact motivated behavior and reward responsiveness to environmental stimuli, and may modulate the addiction liability of drugs of abuse, such as opioids.

Repeated cocaine exposure dysregulates cognitive control over cue-evoked reward-seeking behavior during Pavlovian-to-instrumental transfer

Drug-paired cues acquire powerful motivational properties, but only lead to active drug-seeking behavior if they are potent enough to overwhelm the cognitive control processes that serve to suppress such urges. Studies using the Pavlovian-to-instrumental transfer (PIT) task have shown that rats pretreated with cocaine or amphetamine exhibit heightened levels of cue-motivated food-seeking behavior, suggesting that exposure to these drugs sensitizes the incentive motivational system. However, the PIT testing protocol can also create conflict between two competing behavioral responses to the reward-paired cue: active reward seeking (e.g., lever pressing) and passive conditioned food-cup approach behavior. We therefore investigated whether repeated cocaine exposure alters the way in which rats use cue-based reward expectations to resolve such conflict. In-depth analysis of previously published and new research confirmed that when drug-naïve rats are given a cue that signals the timing of a delayed noncontingent reward, they adaptively transition from reward seeking to conditioned approach behavior, facilitating efficient collection of the predicted reward. In contrast, cocaine-exposed rats exhibit pronounced behavioral dysregulation, increasing, rather than suppressing, their reward-seeking behavior over time, disrupting their ability to passively collect reward. Such findings speak to the important and sometimes overlooked role that cognitive control plays in determining the motivational impact of cues associated with drug and nondrug rewards.

Enduring Loss of Serotonergic Control of Orbitofrontal Cortex Function Following Contingent and Noncontingent Cocaine Exposure

Clinical descriptions of cocaine addiction include compulsive drug seeking and maladaptive decision-making despite substantial aversive consequences. Research suggests that this may result from altered orbitofrontal cortex (OFC) function and its participation in outcome-based behavior. Clinical and animal studies also implicate serotonin in the regulation of OFC function in addiction and other neuropsychiatric disorders. Here we test the hypothesis that exposure to cocaine, through self-administration (CSA) or yoked-administration (CYA), alters the regulation of OFC function by 5-HT. Using whole-cell electrophysiology in brain slices from naïve rats we find that 5-HT1A receptors generate hyperpolarizing outward currents in layer-V OFC pyramidal neurons, and that 5-HT2A receptors increase glutamate release onto these cells. Following extended withdrawal from CSA or CYA, this 5-HT regulation of OFC activity is largely lost. In-situ hybridization of 5-HT receptor transcripts reveals that 5-HT1A receptor mRNA is unaffected and 5-HT2A receptor mRNA is significantly elevated after CSA or CYA. These results demonstrate that 5-HT control of OFC neurons is disrupted for extended periods following cocaine exposure. We hypothesize that this dysregulation of 5-HT signaling leads to enduring disruptions of OFC network activity that this is involved in impaired decision-making associated with cocaine addiction.

The long-term effects of cocaine use on cognitive functioning: A systematic critical review

BACKGROUND

The predominant view of chronic cocaine use maintains that it causes a broad range of cognitive deficits. However, concerns about the possibly deleterious impact of cocaine on cognitive functioning have yet to be thoroughly vetted. This review addresses the impact of cocaine use on such cognitive domains as executive function, memory, language, and psychomotor speed. Additionally, relevant neuroimaging data is considered to understand the neural basis underlying cocaine-related effects on cognitive functioning.

METHODS

We searched PubMed, Google Scholar, and Embase using the search terms "cocaine and cognition," "cocaine and cognitive functioning," and "cocaine and cognitive deficits or impairment." To meet inclusion criteria we evaluated only cognitive and neuroimaging studies describing the long-term effects of cocaine on cognitive functioning published from 1999 to 2016.

RESULTS

The majority of studies reported statistically significant differences between cocaine users and non-drug-using controls in brain structures, blood-oxygen-level dependent signals, and brain metabolism. However, differences in cognitive performance were observed on a minority of measures. Additionally, the majority of studies were not compared against normative data.

CONCLUSIONS

The current evidence does not support the view that chronic cocaine use is associated with broad cognitive deficits. The view that cocaine users have broad cognitive deficits is inaccurate based upon current evidence, and the perpetuation of this view may have negative implications for treatment programs and development of public policies.

Inhibiting Rho kinase promotes goal-directed decision making and blocks habitual responding for cocaine

The prelimbic prefrontal cortex is necessary for associating actions with their consequences, enabling goal-directed decision making. We find that the strength of action–outcome conditioning correlates with dendritic spine density in prelimbic cortex, suggesting that new action–outcome learning involves dendritic spine plasticity. To test this, we inhibited the cytoskeletal regulatory factor Rho kinase. We find that the inhibitor fasudil enhances action–outcome memory, resulting in goal-directed behavior in mice that would otherwise express stimulus-response habits. Fasudil transiently reduces prelimbic cortical dendritic spine densities during a period of presumed memory consolidation, but only when paired with new learning. Fasudil also blocks habitual responding for cocaine, an effect that persists over time, across multiple contexts, and depends on actin polymerization. We suggest that Rho kinase inhibition promotes goal-oriented action selection by augmenting the plasticity of prelimbic cortical dendritic spines during the formation of new action–outcome memories.

Action-outcome learning requires the prelimbic prefrontal cortex. Here the authors report that fasudil, a Rho kinase inhibitor, reduces dendritic spine densities on prelimbic neurons in an activity-dependent manner, stimulating goal-directed actions, and reducing habitual responding for cocaine.

Escalated cocaine "binges" in rats: enduring effects of social defeat stress or intra-VTA CRF

RATIONALE

Exposure to intermittent social defeat stress elicits corticotropin releasing factor (CRF) release into the VTA and induces long-term modulation of mesocorticolimbic dopamine activity in rats. These adaptations are associated with an intense cocaine-taking phenotype, which is prevented by CRF receptor antagonists.

OBJECTIVE

The present studies examine whether infusion of CRF into the VTA is sufficient to escalate cocaine-taking behavior, in the absence of social defeat experience. Additionally, we aimed to characterize changes in cocaine valuation that may promote binge-like cocaine intake.

METHODS

Male Long-Evans rats were microinjected into the VTA with CRF (50 or 500 ng/side), vehicle, or subjected to social defeat stress, intermittently over 10 days. Animals were then trained to self-administer IV cocaine (FR5). Economic demand for cocaine was evaluated using a within-session behavioral-economics threshold procedure, which was followed by a 24-h extended access "binge."

RESULTS

Rats that experienced social defeat or received intra-VTA CRF microinfusions (50 ng) both took significantly more cocaine than controls over the 24-h binge but showed distinct patterns of intake. Behavioral economic analysis revealed that individual demand for cocaine strongly predicts binge-like consumption, and demand elasticity (i.e. α) is augmented by intra-VTA CRF, but not by social defeat. The effects of CRF on cocaine-taking were also prevented by intra-VTA pretreatment with CP376395, but not Astressin-2B.

CONCLUSIONS

Repeated infusion of CRF into the VTA persistently alters cocaine valuation and intensifies binge-like drug intake in a CRF-R1-dependent manner. Conversely, the persistent pattern of cocaine bingeing induced by social defeat stress may suggest impaired inhibitory control, independent of reward valuation.

Prior Cocaine Self-Administration Increases Response-Outcome Encoding That Is Divorced from Actions Selected in Dorsal Lateral Striatum

Dorsal lateral striatum (DLS) is a highly associative structure that encodes relationships among environmental stimuli, behavioral responses, and predicted outcomes. DLS is known to be disrupted after chronic drug abuse; however, it remains unclear what neural signals in DLS are altered. Current theory suggests that drug use enhances stimulus-response processing at the expense of response-outcome encoding, but this has mostly been tested in simple behavioral tasks. Here, we investigated what neural correlates in DLS are affected by previous cocaine exposure as rats performed a complex reward-guided decision-making task in which predicted reward value was independently manipulated by changing the delay to or size of reward associated with a response direction across a series of trial blocks. After cocaine self-administration, rats exhibited stronger biases toward higher-value reward and firing in DLS more strongly represented action-outcome contingencies independent from actions subsequently taken rather than outcomes predicted by selected actions (chosen-outcome contingencies) and associations between stimuli and actions (stimulus-response contingencies). These results suggest that cocaine self-administration strengthens action-outcome encoding in rats (as opposed to chosen-outcome or stimulus-response encoding), which abnormally biases behavior toward valued reward when there is a choice between two options during reward-guided decision-making.SIGNIFICANCE STATEMENT Current theories suggest that the impaired decision-making observed in individuals who chronically abuse drugs reflects a decrease in goal-directed behaviors and an increase in habitual behaviors governed by neural representations of response-outcome (R-O) and stimulus-response associations, respectively. We examined the impact that prior cocaine self-administration had on firing in dorsal lateral striatum (DLS), a brain area known to be involved in habit formation and affected by drugs of abuse, during performance of a complex reward-guided decision-making task. Surprisingly, we found that previous cocaine exposure enhanced R-O associations in DLS. This suggests that there may be more complex consequences of drug abuse than current theories have explored, especially when examining brain and behavior in the context of a complex two-choice decision-making task.

Cognitive control in young adults with cannabis use disorder: An event-related brain potential study

Contemporary models of substance use disorders emphasize the role of cognitive control, which has been linked to difficulties in resisting the use of substances. In the present study, we measured two aspects of cognitive control, response inhibition (operationalized by a Go/NoGo Task) and performance monitoring (operationalized by an Eriksen Flanker Task), in a group of young cannabis-use disorder (CUD) patients and compared these functions with two control groups (i.e. a group of cigarette smokers and a group of non-smokers). We employed both behavioural and electrophysiological measures. The results indicate that CUD patients displayed reduced NoGo-P3 event-related potentials compared with non-smoking controls, but not compared with smoking controls. In addition, CUD patients were slower on Go trials than both control groups. No other between-group electrophysiological or behavioural differences were observed. These results seem to suggest that CUD patients have problems related to response inhibition, but performance monitoring seems relatively unaffected.

Prior Cocaine Experience Impairs Normal Phasic Dopamine Signals of Reward Value in Accumbens Shell

Dopamine signals have repeatedly been linked to associative learning and motivational processes. However, there is considerably less agreement on a role for dopamine in reward processing, and therefore whether neuroplastic changes in dopamine function following chronic exposure to drugs of abuse such as cocaine may impair appropriate valuation of rewarding stimuli. To quantify this, we voltammetrically measured real-time dopamine release in the nucleus accumbens (NAc) core or shell while rats received unsignaled deliveries of either a small (1 pellet) or large (2 pellets) reward. In drug-naive controls, core dopamine signals did not discriminate between reward size at any point, while in the shell dopamine encoded magnitude differences only in a slower postpeak period. Despite this lack of discrimination between rewards by the peak DA response, controls easily discriminated between reward options in a subsequent choice task. In contrast, phasic dopamine reward signals were strongly altered by cocaine experience; core dopamine decreased peak response but increased discrimination between reward magnitudes while shell lost phasic responses to reward receipt altogether. Notably, animals with cocaine-associated alterations in dopamine signals for reward magnitude failed to subsequently discriminate between reward options. These findings suggest that cocaine self-administration alters the ability for dopamine signals to appropriately assign value to rewards and thus may in part contribute to later deficits in behaviors that depend on appropriate outcome valuation.

Subchronic anesthetic ketamine injections in rats impair choice reversal learning, but have no effect on reinforcer devaluation

Previous exposure to a variety of drugs of abuse has been shown to cause long-term impairments in reversal learning and reinforcer devaluation tasks. However, there is mixed evidence in the literature for a long-term effect of ketamine exposure on reversal learning and the long-term effect of ketamine exposure on devaluation is not known. We determined whether repeated injections of an anesthetic dose of ketamine would lead to impairments in choice reversal learning after discrimination learning or impairments in reinforcer devaluation. In two experiments, rats received three injections once-daily of ketamine (100 mg/kg, intraperitoneally) or saline and then began behavioral training 19 days later so that the key reversal learning and devaluation tests would occur about 1 month after the final ketamine injection. This ketamine exposure regimen did not impair learning in our discrimination task, but led to an increase in perseverative errors in reversal learning. However, the same ketamine exposure regimen (or injections of a lower 50 mg/kg dose) had no effect on behavior in the devaluation task. The behavioral patterns observed suggest possible neural mechanisms for the effects of ketamine, but future neurobiological investigations will be needed to isolate these mechanisms.

Thinking Outside the Box: Orbitofrontal Cortex, Imagination, and How We Can Treat Addiction

Addiction involves an inability to control drug-seeking behavior. While this may be thought of as secondary to an overwhelming desire for drugs, it could equally well reflect a failure of the brain mechanisms that allow addicts to learn about and mentally simulate non-drug consequences. Importantly, this process of mental simulation draws upon, but is not normally bound by, our past experiences. Rather we have the ability to think outside the box of our past, integrating knowledge gained from a variety of similar and not-so-similar life experiences to derive estimates or imagine what might happen next. These estimates influence our current behavior directly and also affect future behavior by serving as the background against which outcomes are evaluated to support learning. Here we will review evidence, from our own work using a Pavlovian over-expectation task as well as from other sources, that the orbitofrontal cortex is a critical node in the neural circuit that generates these estimates. Further we will offer the specific hypothesis that degradation of this function secondary to drug-induced changes is a critical and likely addressable part of addiction.

Adolescent cocaine self-administration induces habit behavior in adulthood: sex differences and structural consequences

Adolescent cocaine use increases the likelihood of drug abuse and addiction in adulthood, and etiological factors may include a cocaine-induced bias towards so-called 'reward-seeking' habits. To determine whether adolescent cocaine exposure indeed impacts decision-making strategies in adulthood, we trained adolescent mice to orally self-administer cocaine. In adulthood, males with a history of escalating self-administration developed a bias towards habit-based behaviors. In contrast, escalating females did not develop habit biases; rather, low response rates were associated with later behavioral inflexibility, independent of cocaine dose. We focused the rest of our report on understanding how individual differences in young-adolescent females predicted long-term behavioral outcomes. Low, 'stable' cocaine-reinforced response rates during adolescence were associated with cocaine-conditioned object preference and enlarged dendritic spine head size in the medial (prelimbic) prefrontal cortex in adulthood. Meanwhile, cocaine resilience was associated with enlarged spine heads in deep-layer orbitofrontal cortex. Re-exposure to the cocaine-associated context in adulthood energized responding in 'stable responders', which could then be reduced by the GABAB agonist baclofen and the putative tyrosine receptor kinase B (trkB) agonist, 7,8-dihydroxyflavone. Together, our findings highlight resilience to cocaine-induced habits in females relative to males when intake escalates. However, failures in instrumental conditioning in adolescent females may precipitate reward-seeking behaviors in adulthood, particularly in the context of cocaine exposure.

Prefrontal cortical BDNF: A regulatory key in cocaine- and food-reinforced behaviors

Brain-derived neurotrophic factor (BDNF) affects synaptic plasticity and neural structure and plays key roles in learning and memory processes. Recent evidence also points to important, yet complex, roles for BDNF in rodent models of cocaine abuse and addiction. Here we examine the role of prefrontal cortical (PFC) BDNF in reward-related decision making and behavioral sensitivity to, and responding for, cocaine. We focus on BDNF within the medial and orbital PFC, its regulation by cocaine during early postnatal development and in adulthood, and how BDNF in turn influences responding for drug reinforcement, including in reinstatement models. When relevant, we draw comparisons and contrasts with experiments using natural (food) reinforcers. We also summarize findings supporting, or refuting, the possibility that BDNF in the medial and orbital PFC regulate the development and maintenance of stimulus-response habits. Further investigation could assist in the development of novel treatment approaches for cocaine use disorders.

Carrots and sticks fail to change behavior in cocaine addiction

Cocaine addiction is a major public health problem that is particularly difficult to treat. Without medically proven pharmacological treatments, interventions to change the maladaptive behavior of addicted individuals mainly rely on psychosocial approaches. Here we report on impairments in cocaine-addicted patients to act purposefully toward a given goal and on the influence of extended training on their behavior. When patients were rewarded for their behavior, prolonged training improved their response rate toward the goal but simultaneously rendered them insensitive to the consequences of their actions. By contrast, overtraining of avoidance behavior had no effect on patient performance. Our findings illustrate the ineffectiveness of punitive approaches and highlight the potential for interventions that focus on improving goal-directed behavior and implementing more desirable habits to replace habitual drug-taking.

Extended cocaine-seeking produces a shift from goal-directed to habitual responding in rats

Cocaine addiction is often characterized by a rigid pattern of behavior in which cocaine users continue seeking and taking drug despite negative consequences associated with its use. As such, full acquisition and relapse of drug-seeking behavior may be attributed to a shift away from goal-directed responding and a shift towards the maladaptive formation of rigid and habit-like responses. This rigid nature of habitual responding can be developed with extended training and is typically characterized by insensitivity to changes in outcome value. The present study determined whether cocaine (primary reinforcer) and cocaine associated cues (secondary reinforcer) could be devalued in rats with different histories of cocaine self-administration. Specifically, rats were trained on two schedules of cocaine self-administration (long-access vs. short-access). Following training the cocaine reinforcer was devalued through three separate pairings of lithium chloride with cocaine infusions. Cocaine history did not have an impact on devaluation of cocaine-associated cues. However, the reinforcing properties of cocaine were devalued only in rats on a short-access cocaine schedule but not those trained on a long-access schedule. Taken together this pattern of findings suggests that, in short access rats, devaluation is specific to the primary reinforcer and not associative stimuli such as cues. Importantly, rats that received extended training during self-administration displayed insensitivity to outcome devaluation of the primary reinforcer as well as all associative stimuli, thus displaying rigid behavioral responding similar to behavioral patterns found in addiction. Alternatively, long access cocaine exposure may have altered the devaluation threshold.

Going and stopping: Dichotomies in behavioral control by the prefrontal cortex

The rodent dorsal medial prefrontal cortex (PFC), specifically the prelimbic cortex (PL), regulates the expression of conditioned fear and behaviors interpreted as reward-seeking. Meanwhile, the ventral medial PFC, namely the infralimbic cortex (IL), is essential to extinction conditioning in both appetitive and aversive domains. Here we review evidence that supports, or refutes, this "PL-go/IL-stop" dichotomy. We focus on the extinction of conditioned fear and the extinction and reinstatement of cocaine- or heroin-reinforced responding. We then synthesize evidence that the PL is essential for developing goal-directed response strategies, while the IL supports habit behavior. Finally, we propose that some functions of the orbital PFC parallel those of the medial PFC in the regulation of response selection. Integration of these discoveries may provide points of intervention for inhibiting untethered drug seeking in drug use disorders, failures in extinction in Post-traumatic Stress Disorder, or co-morbidities between the two.

Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry

Midbrain dopamine (DA) neurons are proposed to signal reward prediction error (RPE), a fundamental parameter in associative learning models. This RPE hypothesis provides a compelling theoretical framework for understanding DA function in reward learning and addiction. New studies support a causal role for DA-mediated RPE activity in promoting learning about natural reward; however, this question has not been explicitly tested in the context of drug addiction. In this review, we integrate theoretical models with experimental findings on the activity of DA systems, and on the causal role of specific neuronal projections and cell types, to provide a circuit-based framework for probing DA-RPE function in addiction. By examining error-encoding DA neurons in the neural network in which they are embedded, hypotheses regarding circuit-level adaptations that possibly contribute to pathological error signaling and addiction can be formulated and tested.

Cocaine Self-Administration Experience Induces Pathological Phasic Accumbens Dopamine Signals and Abnormal Incentive Behaviors in Drug-Abstinent Rats

Chronic exposure to drugs of abuse is linked to long-lasting alterations in the function of limbic system structures, including the nucleus accumbens (NAc). Although cocaine acts via dopaminergic mechanisms within the NAc, less is known about whether phasic dopamine (DA) signaling in the NAc is altered in animals with cocaine self-administration experience or if these animals learn and interact normally with stimuli in their environment. Here, separate groups of rats self-administered either intravenous cocaine or water to a receptacle (controls), followed by 30 d of enforced abstinence. Next, all rats learned an appetitive Pavlovian discrimination and voltammetric recordings of real-time DA release were taken in either the NAc core or shell of cocaine and control subjects. Cocaine experience differentially impaired DA signaling in the core and shell relative to controls. Although phasic DA signals in the shell were essentially abolished for all stimuli, in the core, DA did not distinguish between cues and was abnormally biased toward reward delivery. Further, cocaine rats were unable to learn higher-order associations and even altered simple conditioned approach behaviors, displaying enhanced preoccupation with cue-associated stimuli (sign-tracking; ST) but diminished time at the food cup awaiting reward delivery (goal-tracking). Critically, whereas control DA signaling correlated with ST behaviors, cocaine experience abolished this relationship. These findings show that cocaine has persistent, differential, and pathological effects on both DA signaling and DA-dependent behaviors and suggest that psychostimulant experience may remodel the very circuits that bias organisms toward repeated relapse.

SIGNIFICANCE STATEMENT

Relapsing to drug abuse despite periods of abstinence and sincere attempts to quit is one of the most pernicious facets of addiction. Unfortunately, little is known about how the dopamine (DA) system functions after periods of drug abstinence, particularly its role in behavior in nondrug situations. Here, rats learned about food-paired stimuli after prolonged abstinence from cocaine self-administration. Using voltammetry, we found that real-time DA signals in cocaine-experienced rats were strikingly altered relative to controls. Further, cocaine-experienced animals found reward-predictive stimuli abnormally salient and spent more time interacting with cues. Therefore, cocaine induces neuroplastic changes in the DA system that biases animals toward salient stimuli (including reward-associated cues), putting addicts at increasing risk to relapse as addiction increases in severity.

A Closer Look at the Effects of Repeated Cocaine Exposure on Adaptive Decision-Making under Conditions That Promote Goal-Directed Control

It has been proposed that compulsive drug seeking reflects an underlying dysregulation in adaptive behavior that favors habitual (automatic and inflexible) over goal-directed (deliberative and highly flexible) action selection. Rodent studies have established that repeated exposure to cocaine or amphetamine facilitates the development of habits, producing behavior that becomes unusually insensitive to a reduction in the value of its outcome. The current study more directly investigated the effects of cocaine pre-exposure on goal-directed learning and action selection using an approach that discourages habitual performance. After undergoing a 15-day series of cocaine (15 or 30 mg/kg, i.p.) or saline injections and a drug withdrawal period, rats were trained to perform two different lever-press actions for distinct reward options. During a subsequent outcome devaluation test, both cocaine- and saline-treated rats showed a robust bias in their choice between the two actions, preferring whichever action had been trained with the reward that retained its value. Thus, it appears that the tendency for repeated cocaine exposure to promote habit formation does not extend to a more complex behavioral scenario that encourages goal-directed control. To further explore this issue, we assessed how prior cocaine treatment would affect the rats' ability to learn about a selective reduction in the predictive relationship between one of the two actions and its outcome, which is another fundamental feature of goal-directed behavior. Interestingly, we found that cocaine-treated rats showed enhanced, rather than diminished, sensitivity to this action-outcome contingency degradation manipulation. Given their mutual dependence on striatal dopamine signaling, we suggest that cocaine's effects on habit formation and contingency learning may stem from a common adaptation in this neurochemical system.

Memory Systems and the Addicted Brain

The view that anatomically distinct memory systems differentially contribute to the development of drug addiction and relapse has received extensive support. The present brief review revisits this hypothesis as it was originally proposed 20 years ago (1) and highlights several recent developments. Extensive research employing a variety of animal learning paradigms indicates that dissociable neural systems mediate distinct types of learning and memory. Each memory system potentially contributes unique components to the learned behavior supporting drug addiction and relapse. In particular, the shift from recreational drug use to compulsive drug abuse may reflect a neuroanatomical shift from cognitive control of behavior mediated by the hippocampus/dorsomedial striatum toward habitual control of behavior mediated by the dorsolateral striatum (DLS). In addition, stress/anxiety may constitute a cofactor that facilitates DLS-dependent memory, and this may serve as a neurobehavioral mechanism underlying the increased drug use and relapse in humans following stressful life events. Evidence supporting the multiple systems view of drug addiction comes predominantly from studies of learning and memory that have employed as reinforcers addictive substances often considered within the context of drug addiction research, including cocaine, alcohol, and amphetamines. In addition, recent evidence suggests that the memory systems approach may also be helpful for understanding topical sources of addiction that reflect emerging health concerns, including marijuana use, high-fat diet, and video game playing.

Individual variability in behavioral flexibility predicts sign-tracking tendency

Sign-tracking rats show heightened sensitivity to food- and drug-associated cues, which serve as strong incentives for driving reward seeking. We hypothesized that this enhanced incentive drive is accompanied by an inflexibility when incentive value changes. To examine this we tested rats in Pavlovian outcome devaluation or second-order conditioning prior to the assessment of sign-tracking tendency. To assess behavioral flexibility we trained rats to associate a light with a food outcome. After the food was devalued by pairing with illness, we measured conditioned responding (CR) to the light during an outcome devaluation probe test. The level of CR during outcome devaluation probe test correlated with the rats' subsequent tracking tendency, with sign-tracking rats failing to suppress CR to the light after outcome devaluation. To assess Pavlovian incentive learning, we trained rats on first-order (CS+, CS-) and second-order (SOCS+, SOCS-) discriminations. After second-order conditioning, we measured CR to the second-order cues during a probe test. Second-order conditioning was observed across all rats regardless of tracking tendency. The behavioral inflexibility of sign-trackers has potential relevance for understanding individual variation in vulnerability to drug addiction.

The basolateral amygdala in reward learning and addiction

Sophisticated behavioral paradigms partnered with the emergence of increasingly selective techniques to target the basolateral amygdala (BLA) have resulted in an enhanced understanding of the role of this nucleus in learning and using reward information. Due to the wide variety of behavioral approaches many questions remain on the circumscribed role of BLA in appetitive behavior. In this review, we integrate conclusions of BLA function in reward-related behavior using traditional interference techniques (lesion, pharmacological inactivation) with those using newer methodological approaches in experimental animals that allow in vivo manipulation of cell type-specific populations and neural recordings. Secondly, from a review of appetitive behavioral tasks in rodents and monkeys and recent computational models of reward procurement, we derive evidence for BLA as a neural integrator of reward value, history, and cost parameters. Taken together, BLA codes specific and temporally dynamic outcome representations in a distributed network to orchestrate adaptive responses. We provide evidence that experiences with opiates and psychostimulants alter these outcome representations in BLA, resulting in long-term modified action.

Too difficult to stop: mechanisms facilitating relapse in alcohol dependence

BACKGROUND

In alcohol and other substance dependencies, patients often suffer relapse despite better knowledge and their intention to remain abstinent. A variety of neurotransmitter systems and their respective alterations due to the chronic drug intake are involved in mechanisms that facilitate relapse. It has been postulated that these neurotransmitter systems are related to changes in motivational and learning mechanisms, and engender a shift from goal-directed to habitual behavior in dependent patients that facilitates drug-seeking behavior.

METHODS

We review learning mechanisms facilitating relapse, as identified and tested to date. We focus on studies examining the interaction between alcohol-related changes in monoaminergic neurotransmission and their respective effects on pavlovian and operant learning mechanisms in alcohol dependence.

RESULTS

Animal experiments and first human studies suggest that chronic alcohol intake impairs goal-directed behavior and facilitates habitual drug intake. Key symptoms of alcohol dependence such as tolerance development, withdrawal, craving and reduced control of alcohol intake can be explained by alcohol-induced alteration of dopaminergic neurotransmission and its GABAergic and glutamatergic modulation and their respective effects on pavlovian and operant conditioning as well as pavlovian-to-instrumental transfer.

CONCLUSION

Chronic alcohol intake impairs neurotransmitter systems that regulate prefrontal-striatal circuits and interfere with goal-directed decision-making and the acquisition of new, non-drug-related behavior patterns. Alcohol craving induced by pavlovian conditioned cues can facilitate habitual drug intake. Such learning mechanisms and their alterations by chronic alcohol intake might be targeted by specific interventions.

Model-based and model-free decisions in alcohol dependence

BACKGROUND

Human and animal work suggests a shift from goal-directed to habitual decision-making in addiction. However, the evidence for this in human alcohol dependence is as yet inconclusive.

METHODS

Twenty-six healthy controls and 26 recently detoxified alcohol-dependent patients underwent behavioral testing with a 2-step task designed to disentangle goal-directed and habitual response patterns.

RESULTS

Alcohol-dependent patients showed less evidence of goal-directed choices than healthy controls, particularly after losses. There was no difference in the strength of the habitual component. The group differences did not survive controlling for performance on the Digit Symbol Substitution Task.

CONCLUSION

Chronic alcohol use appears to selectively impair goal-directed function, rather than promoting habitual responding. It appears to do so particularly after nonrewards, and this may be mediated by the effects of alcohol on more general cognitive functions subserved by the prefrontal cortex.

Synaptic Cytoskeletal Plasticity in the Prefrontal Cortex Following Psychostimulant Exposure

Addiction is characterized by maladaptive decision-making, a loss of control over drug consumption and habit-like drug seeking despite adverse consequences. These cognitive changes may reflect the effects of drugs of abuse on prefrontal cortical neurobiology. Here, we review evidence that amphetamine and cocaine fundamentally remodel the structure of excitatory neurons in the prefrontal cortex. We summarize evidence in particular that these psychostimulants have opposing effects in the medial and orbital prefrontal cortices ('mPFC' and 'oPFC', respectively). For example, amphetamine and cocaine increase dendrite length and spine density in the mPFC, while dendrites are impoverished and dendritic spines are eliminated in the oPFC. We will discuss evidence that certain cytoskeletal regulatory proteins expressed in the oPFC and implicated in postnatal (adolescent) neural development also regulate behavioral sensitivity to cocaine. These findings potentially open a window of opportunity for the identification of novel pharmacotherapeutic targets in the treatment of drug abuse disorders in adults, as well as in drug-vulnerable adolescent populations. Finally, we will discuss the behavioral implications of drug-related dendritic spine elimination in the oPFC, with regard to reversal learning tasks and tasks that assess the development of reward-seeking habits, both used to model aspects of addiction in rodents.