Post-training cocaine administration facilitates habit learning and requires the infralimbic cortex and dorsolateral striatum

Neither Amphetamine nor Sub-Anesthetic Ketamine Treatment during Adolescence Impairs Devaluation in Rats Tested during Adulthood

BACKGROUND

Much of the existing animal literature on the devaluation task suggests that prior repeated exposure to drugs of abuse during adulthood can impair goal-directed action, but the literature on human drug users is mixed. Also, the initiation of drug use often occurs during adolescence, but examinations of the effects of drug exposure during adolescence on behavior in the devaluation task are lacking.

METHODS

We examined whether repeated exposure during adolescence to amphetamine (3 mg/kg injections every-other day from post-natal day 27-45) or ketamine (twice daily 30 mg/kg injections from post-natal day 35-44) would impair behavior in a devaluation test when tested drug-free in adulthood. Rats were trained to press a left lever with a steady cue-light above it for one reinforcer and a right lever with a flashing cue-light above it for a different reinforcer. We tested whether any impairments in goal-directed action could be overcome by compensation between strategies by giving rats information based on lever-location and cue-lights during the test that was either congruent (allowing compensation) or incongruent (preventing compensation between strategies) with the configurations during training.

RESULTS

Our results provided no evidence for impairment of goal-directed action during adulthood after adolescent amphetamine or ketamine exposure.

CONCLUSIONS

We discuss possible reasons for this discrepancy with the prior literature, including (1) the age of exposure and (2) the pattern in the previous literature that most previous demonstrations of drug exposure impairing devaluation in laboratory animals may be attributed to either drug-associated cues present in the testing environment and/or accelerated habit learning in tasks that predispose laboratory animals towards habit formation with extended training (with training procedures that should resist the formation of habits in the current experiment). However, additional research is needed to examine the effects of these factors, as well a potential role for the particular doses and washout periods to determine the cause of our finding of no devaluation impairment after drug exposure.

Computational Mechanisms of Addiction and Anxiety: A Developmental Perspective

A central goal of computational psychiatry is to identify systematic relationships between transdiagnostic dimensions of psychiatric symptomatology and the latent learning and decision-making computations that inform individuals' thoughts, feelings, and choices. Most psychiatric disorders emerge prior to adulthood, yet little work has extended these computational approaches to study the development of psychopathology. Here, we lay out a roadmap for future studies implementing this approach by developing empirically and theoretically informed hypotheses about how developmental changes in model-based control of action and Pavlovian learning processes may modulate vulnerability to anxiety and addiction. We highlight how insights from studies leveraging computational approaches to characterize the normative developmental trajectories of clinically relevant learning and decision-making processes may suggest promising avenues for future developmental computational psychiatry research.

Ghrelin/GHS-R1A antagonism in memory test and its effects on central molecular signaling involved in addiction in rats

Central ghrelin signaling seems to play important role in addiction as well as memory processing. Antagonism of the growth hormone secretagogue receptor (GHS-R1A) has been recently proposed as a promising tool for the unsatisfactory drug addiction therapy. However, molecular aspects of GHS-R1A involvement in specific brain regions remain unclear. The present study demonstrated for the first time that acute as well as subchronic (4 days) administration of the experimental GHS-R1A antagonist JMV2959 in usual intraperitoneal doses including 3 mg/kg, had no influence on memory functions tested in the Morris Water Maze in rats as well as no significant effects on the molecular markers linked with memory processing in selected brain areas in rats, specifically on the β-actin, c-Fos, two forms of the calcium/calmodulin-dependent protein kinase II (CaMKII, p-CaMKII) and the cAMP-response element binding protein (CREB, p-CREB), within the medial prefrontal cortex (mPFC), nucleus accumbens (NAc), dorsal striatum, and hippocampus (HIPP). Furthermore, following the methamphetamine intravenous self-administration in rats, the 3 mg/kg JMV2959 pretreatment significantly reduced or prevented the methamphetamine-induced significant decrease of hippocampal β-actin and c-Fos as well as it prevented the significant decrease of CREB in the NAC and mPFC. These results imply, that the GHS-R1A antagonist/JMV2959 might reduce/prevent some of the memory-linked molecular changes elicited by methamphetamine addiction within brain structures associated with memory (HIPP), reward (NAc), and motivation (mPFC), which may contribute to the previously observed significant JMV2959-induced reduction of the methamphetamine self-administration and drug-seeking behavior in the same animals. Further research is necessary to corroborate these results.

Cocaine falls into oblivion during volitional initiation of choice trials

When facing a choice, most animals quit drugs in favour of a variety of nondrug alternatives. We recently found, rather unexpectedly, that choice of the nondrug alternative is in fact inflexible and habitual. One possible contributing factor to habitual choice is the intermittency and uncontrollability of choice trials in previous studies. Here, we asked whether and to what extent volitional control over the occurrence of choice trials could change animals' preference by preventing habitual choice. To do so, rats were trained to nosepoke in a hole to trigger the presentation of two operant levers: one associated with cocaine, the other with saccharin. Rats were then free to choose among the two levers to obtain the corresponding reward, after which both levers retracted until rats self-initiated the next choice trial. Overall, we found that volitional control over choice trials did not change preference. Most rats preferred saccharin over cocaine and selected this option almost exclusively. Intriguingly, after repeated choice and consumption of saccharin, rats transiently lost interest in this option (i.e., due to sensory-specific satiety), but they did not switch to cocaine, preferring instead to pause during long periods of time before reinitiating a choice trial for saccharin. This finding suggests that during volitional initiation of a choice trial, rats fail to consider cocaine as an option. We discuss a possible associative mechanism to explain this perplexing behaviour.

MicroRNAs signatures associated with vulnerability to food addiction in mice and humans

Food addiction is characterized by a loss of behavioral control over food intake and is associated with obesity and other eating disorders. The mechanisms underlying this behavioral disorder are largely unknown. We aimed to investigate the changes in miRNA expression promoted by food addiction in animals and humans and their involvement in the mechanisms underlying the behavioral hallmarks of this disorder. We found sharp similitudes between miRNA signatures in the medial prefrontal cortex (mPFC) of our animal cohort and circulating miRNA levels in our human cohort, which allowed us to identify several miRNAs of potential interest in the development of this disorder. Tough decoy (TuD) inhibition of miRNA-29c-3p in the mouse mPFC promoted persistence of the response and enhanced vulnerability to developing food addiction, whereas miRNA-665-3p inhibition promoted compulsion-like behavior and also enhanced food addiction vulnerability. In contrast, we found that miRNA-137-3p inhibition in the mPFC did not lead to the development of food addiction. Therefore, miRNA-29c-3p and miRNA-665-3p could be acting as protective factors with regard to food addiction. We believe the elucidation of these epigenetic mechanisms will lead to advances toward identifying innovative biomarkers and possible future interventions for food addiction and related disorders based on the strategies now available to modify miRNA activity and expression.

New functions of the rodent prelimbic and infralimbic cortex in instrumental behavior

The prelimbic and infralimbic cortices of the rodent medial prefrontal cortex mediate the effects of context and goals on instrumental behavior. Recent work from our laboratory has expanded this understanding. Results have shown that the prelimbic cortex is important for the modulation of instrumental behavior by the context in which the behavior is learned (but not other contexts), with context potentially being broadly defined (to include at least previous behaviors). We have also shown that the infralimbic cortex is important in the expression of extensively-trained instrumental behavior, regardless of whether that behavior is expressed as a stimulus-response habit or a goal-directed action. Some of the most recent data suggest that infralimbic cortex may control the currently active behavioral state (e.g., habit vs. action or acquisition vs. extinction) when two states have been learned. We have also begun to examine prelimbic and infralimbic cortex function as key nodes of discrete circuits and have shown that prelimbic cortex projections to an anterior region of the dorsomedial striatum are important for expression of minimally-trained instrumental behavior. Overall, the use of an associative learning perspective on instrumental learning has allowed the research to provide new perspectives on how these two "cognitive" brain regions contribute to instrumental behavior.

Extended operant training increases infralimbic and prelimbic cortex Fos regardless of fear conditioning experience

Extended fear training can lead to initially low fear expression that grows over time, termed fear incubation. Conversely, a single fear conditioning session typically results in high fear initially that is sustained over time. Fear expression decreases across extended training, suggesting that a fear extinction-like process might be responsible for low fear observed soon after training. Because of the prominent role medial prefrontal cortex (mPFC) plays in fear conditioning and extinction, we decided to examine Fos expression resulting from a cued fear retrieval test to gain insight into possible mechanisms involved in extended training fear incubation. Male Long-Evans rats received 1 or 10 days of tone-shock pairings or tone-only exposure (while lever-pressing for food). Two days after the end of fear training, rats received a cued fear test, with perfusions timed to visualize Fos expression during test. As expected, the limited fear conditioning group exhibited higher fear in the test than any of the other groups (as measured with conditioned suppression of lever-pressing). Interestingly, we found that extended training animals (whether they received tone-shock pairings or tone-only exposure) expressed higher levels of Fos in both prelimbic and infralimbic cortices than limited training animals. There was no association between fear expression and mPFC Fos expression. These results suggest we may have visualized Fos expression related to operant overtraining rather than conditioned fear related processes. Further research is needed to determine the neurobiological basis of extended training fear incubation and to determine processes represented by the pattern of Fos expression we observed.

Reduced Glutamate Turnover in the Putamen Is Linked With Automatic Habits in Human Cocaine Addiction

BACKGROUND

The balance between goal-directed behavior and habits has been hypothesized to be biased toward the latter in individuals with cocaine use disorder (CUD), suggesting possible neurochemical changes in the putamen, which may contribute to their compulsive behavior.

METHODS

We assessed habitual behavior in 48 patients with CUD and 42 healthy control participants using a contingency degradation paradigm and the Creature of Habit Scale. In a subgroup of this sample (CUD: n = 21; control participants: n = 22), we also measured glutamate and glutamine concentrations in the left putamen using ultra-high-field (7T) magnetic resonance spectroscopy. We hypothesized that increased habitual tendencies in patients with CUD would be associated with abnormal glutamatergic metabolites in the putamen.

RESULTS

Compared with their non-drug-using peers, patients with CUD exhibited greater habitual tendencies during contingency degradation, which correlated with increased levels of self-reported daily habits. We further identified a significant reduction in glutamate concentration and glutamate turnover (glutamate-to-glutamine ratio) in the putamen in patients with CUD, which was significantly related to the level of self-reported daily habits.

CONCLUSIONS

Patients with CUD exhibit enhanced habitual behavior, as assessed both by questionnaire and by a laboratory paradigm of contingency degradation. This automatic habitual tendency is related to a reduced glutamate turnover in the putamen, suggesting a dysregulation of habits caused by chronic cocaine use.

Habit, choice, and addiction

Addiction was suggested to emerge from the progressive dominance of habits over goal-directed behaviors. However, it is generally assumed that habits do not persist in choice settings. Therefore, it is unclear how drug habits may persist in real-world scenarios where this factor predominates. Here, we discuss the poor translational validity of the habit construct, which impedes our ability to determine its role in addiction. New evidence of habitual behavior in a drug choice setting are then described and discussed. Interestingly, habitual preference did not promote drug choice but instead favored abstinence. Here, we propose several clues to reconcile these unexpected results with the habit theory of addiction, and we highlight the need in experimental research to face the complexity of drug addicts' decision-making environments by investigating drug habits in the context of choice and in the presence of cues. On a theoretical level, we need to consider more complex frameworks, taking into account continuous interactions between goal-directed and habitual systems, and alternative decision-making models more representative of real-world conditions.

Vulnerability to addiction

Addiction is a chronic brain disease that has dramatic health and socioeconomic consequences worldwide. Multiple approaches have been used for decades to clarify the neurobiological basis of this disease and to identify novel potential treatments. This review summarizes the main brain networks involved in the vulnerability to addiction and specific innovative technological approaches to investigate these neural circuits. First, the evolution of the definition of addiction across the Diagnostic and Statistical Manual of Mental Disorders (DSM) is revised. We next discuss several innovative experimental techniques that, combined with behavioral approaches, have allowed recent critical advances in understanding the neural circuits involved in addiction, including DREADDs, calcium imaging, and electrophysiology. All these techniques have been used to investigate specific neural circuits involved in vulnerability to addiction and have been extremely useful to clarify the neurobiological basis of each specific component of the addictive process. These novel tools targeting specific brain regions are of great interest to further understand the different aspects of this complex disease. This article is part of the special issue on 'Vulnerabilities to Substance Abuse.'.

Habitual Preference for the Nondrug Reward in a Drug Choice Setting

For adaptive and efficient decision making, it must be possible to select between habitual alternative courses of action. However, research in rodents suggests that, even in the context of simple decision-making, choice behavior remains goal-directed. In contrast, we recently found that during discrete trial choice between cocaine and water, water-restricted rats preferred water and this preference was habitual and inflexible (i.e., resistant to water devaluation by satiation). Here we sought to test the reproducibility and generality of this surprising finding by assessing habitual control of preference for saccharin over cocaine in non-restricted rats. Specifically, after the acquisition of preference for saccharin, saccharin was devalued and concurrent responding for both options was measured under extinction. As expected, rats responded more for saccharin than for cocaine during extinction, but this difference was unaffected by saccharin devaluation. Together with our previous research, this result indicates that preference for nondrug alternatives over cocaine is under habitual control, even under conditions that normally support goal-directed control of choice between nondrug options. The possible reasons for this difference are discussed.

The transition to compulsion in addiction

Compulsion is a cardinal symptom of drug addiction (severe substance use disorder). However, compulsion is observed in only a small proportion of individuals who repeatedly seek and use addictive substances. Here, we integrate accounts of the neuropharmacological mechanisms that underlie the transition to compulsion with overarching learning theories, to outline how compulsion develops in addiction. Importantly, we emphasize the conceptual distinctions between compulsive drug-seeking behaviour and compulsive drug-taking behaviour (that is, use). In the latter, an individual cannot stop using a drug despite major negative consequences, possibly reflecting an imbalance in frontostriatal circuits that encode reward and aversion. By contrast, an individual may compulsively seek drugs (that is, persist in seeking drugs despite the negative consequences of doing so) when the neural systems that underlie habitual behaviour dominate goal-directed behavioural systems, and when executive control over this maladaptive behaviour is diminished. This distinction between different aspects of addiction may help to identify its neural substrates and new treatment strategies.

Impaired decision making following escalation of cocaine self-administration predicts vulnerability to relapse in rats

Impairments in cost‐benefit decision making represent a cardinal feature of drug addiction. However, whether these alterations predate drug exposure, thereby contributing to facilitating loss of control over drug intake, or alternatively arise as a result of drug use and subsequently confer vulnerability to relapse has yet to be determined. Male Sprague‐Dawley rats were trained to self‐administer (SA) cocaine during 19 daily long‐access (12‐h) sessions; conditions reliably shown to promote escalation. One week after cocaine SA, rats underwent an extinction/relapse test immediately followed by conditioned stimuli–, stress‐, and drug‐primed reinstatement challenges. The influence of escalated cocaine intake on decision making was measured over time by four test sessions of a rodent analogue of the Iowa Gambling Task (rGT), once prior to cocaine exposure and then 1 day, 1 week, and 1 month after the last SA session. Substantial individual variability was observed in the influence of escalated cocaine SA on decision‐making performance. A subset of rats displayed pronounced deficits, while others showed unaffected or even improved performance on the rat Gambling Task (rGT) 24 hours after the last SA session. When challenged with a relapse test after 1 week of forced abstinence, animals that showed impaired decision making following SA displayed an increased propensity to respond for cocaine under extinction. These data suggest that decision‐making deficits in individuals with drug addiction are not antecedent to—but arise as a consequence of—drug exposure. Moreover, these data indicate that susceptibility to the deleterious effects of drugs on decision making confers vulnerability toward relapse.

Addiction is driven by excessive goal-directed drug choice under negative affect: translational critique of habit and compulsion theory

Drug addiction may be a goal-directed choice driven by excessive drug value in negative affective states, a habit driven by strong stimulus-response associations, or a compulsion driven by insensitivity to costs imposed on drug seeking. Laboratory animal and human evidence for these three theories is evaluated. Excessive goal theory is supported by dependence severity being associated with greater drug choice/economic demand. Drug choice is demonstrably goal-directed (driven by the expected value of the drug) and can be augmented by stress/negative mood induction and withdrawal-effects amplified in those with psychiatric symptoms and drug use coping motives. Furthermore, psychiatric symptoms confer risk of dependence, and coping motives mediate this risk. Habit theory of addiction has weaker support. Habitual behaviour seen in drug-exposed animals often does not occur in complex decision scenarios, or where responding is rewarded, so habit is unlikely to explain most human addictive behaviour where these conditions apply. Furthermore, most human studies have not found greater propensity to habitual behaviour in drug users or as a function of dependence severity, and the minority that have can be explained by task disengagement producing impaired explicit contingency knowledge. Compulsion theory of addiction also has weak support. The persistence of punished drug seeking in animals is better explained by greater drug value (evinced by the association with economic demand) than by insensitivity to costs. Furthermore, human studies have provided weak evidence that propensity to discount cost imposed on drug seeking is associated with dependence severity. These data suggest that human addiction is primarily driven by excessive goal-directed drug choice under negative affect, and less by habit or compulsion. Addiction is pathological because negative states powerfully increase expected drug value acutely outweighing abstinence goals.

A specific prelimbic-nucleus accumbens pathway controls resilience versus vulnerability to food addiction

Food addiction is linked to obesity and eating disorders and is characterized by a loss of behavioral control and compulsive food intake. Here, using a food addiction mouse model, we report that the lack of cannabinoid type-1 receptor in dorsal telencephalic glutamatergic neurons prevents the development of food addiction-like behavior, which is associated with enhanced synaptic excitatory transmission in the medial prefrontal cortex (mPFC) and in the nucleus accumbens (NAc). In contrast, chemogenetic inhibition of neuronal activity in the mPFC-NAc pathway induces compulsive food seeking. Transcriptomic analysis and genetic manipulation identified that increased dopamine D2 receptor expression in the mPFC-NAc pathway promotes the addiction-like phenotype. Our study unravels a new neurobiological mechanism underlying resilience and vulnerability to the development of food addiction, which could pave the way towards novel and efficient interventions for this disorder.

Food addiction is linked to obesity and eating disorders. In a mouse model of food addiction, the authors show that a medial prefrontal cortex-nucleus accumbens pathway is involved in vulnerability and resilience against the development of food addiction-like behavior.

Hippocampal BDNF regulates a shift from flexible, goal-directed to habit memory system function following cocaine abstinence

The transition from recreational drug use to addiction involves pathological learning processes that support a persistent shift from flexible, goal-directed to habit behavioral control. Here, we examined the molecular mechanisms supporting altered function in hippocampal (HPC) and dorsolateral striatum (DLS) memory systems following abstinence from repeated cocaine. After 3 weeks of cocaine abstinence (experimenter- or self-administered), we tested new behavioral learning in male rats using a dual-solution maze task, which provides an unbiased approach to assess HPC- versus DLS-dependent learning strategies. Dorsal hippocampus (dHPC) and DLS brain tissues were collected after memory testing to identify transcriptional adaptations associated with cocaine-induced shifts in behavioral learning. Our results demonstrate that following prolonged cocaine abstinence rats show a bias toward the use of an inflexible, habit memory system (DLS) in lieu of a more flexible, easily updated memory system involving the HPC. This memory system bias was associated with upregulation and downregulation of brain-derived neurotrophic factor (BDNF) gene expression and transcriptionally permissive histone acetylation (acetylated histone H3, AcH3) in the DLS and dHPC, respectively. Using viral-mediated gene transfer, we overexpressed BDNF in the dHPC during cocaine abstinence and new maze learning. This manipulation restored HPC-dependent behavioral control. These findings provide a system-level understanding of altered plasticity and behavioral learning following cocaine abstinence and inform mechanisms mediating the organization of learning and memory more broadly.

Inflexible habitual decision-making during choice between cocaine and a nondrug alternative

The concept of compulsive cocaine-seeking habits is difficult to reconcile with other evidence showing that humans and even rats remain able to shift their choice away from the drug and toward an alternative nondrug reward, when available. This paradox could dissolve if preference for the nondrug option reflected in fact inflexible habitual decision-making (i.e., fixed in a habitual control mode, with no return to a goal-directed control mode). Previous research in rats has shown that prior drug use can favor habit formation, but whether the resulting habits are inflexible or not is largely unknown. Here we addressed this question by manipulating the value of water in rats that chose between water and cocaine in a discrete-trials procedure. Rats preferred water when thirsty and maintained this preference despite water devaluation by satiation. Only with repeated daily testing under water satiation did they progressively reverse their preference toward cocaine. Additional evidence showed that this progressive reversal of preference reflected in fact new interoceptive discrimination learning. Overall, this study suggests that rats seem to be stuck in a habitual decision-making mode, unable to return to a goal-directed mode upon experiencing a change in options value. It also reveals that inflexible decision-making does not necessarily promote drug choice, but can also under some circumstances favor abstinence.

Behavioral and neural mechanisms underlying habitual and compulsive drug seeking

Addiction is characterized by compulsive drug use despite negative consequences. Here we review studies that indicate that compulsive drug use, and in particular punishment resistance in animal models of addiction, is related to impaired cortical control over habitual behavior. In humans and animals, instrumental behavior is supported by goal-directed and habitual systems that rely on distinct corticostriatal networks. Chronic exposure to addictive drugs or stress has been shown to bias instrumental response strategies toward habit learning, and impair prefrontal cortical (PFC) control over responding. Moreover, recent work has implicated prelimbic PFC hypofunction in the punishment resistance that has been observed in a subset of animals with an extended history of cocaine self-administration. This may be related to a broader role for prelimbic PFC in mediating adaptive responding and behavioral flexibility, including exerting goal-directed control over behavior. We hypothesize that impaired cortical control and reduced flexibility between habitual and goal-directed systems may be critically involved in the development of maladaptive, compulsive drug use.

An extended history of drug self-administration results in multiple sources of control over drug seeking behavior

It is widely recognized that across the development of drug addiction, cues associated with drug use come to exert increasing control over drug seeking and taking behaviors. However, there remain gaps in our knowledge regarding how the different types of drug related cues affect drug seeking and taking behaviors, and how the emergence of cue control over these behaviors relates to the onset of drug seeking compulsions. This paper reviews the literature on drug self-administration in animals to address these gaps. It first identifies the different types of cues that acquire control over reward seeking behavior generally, and examines whether the same types of cues acquire control over drug seeking behavior specifically. It then examines how the role of drug related cues in motivating and reinforcing drug seeking behavior changes across an extended drug-taking history, with a particular focus on the case of nicotine. The evidence reviewed shows that, after an extended history of drug taking, drug seeking behaviors are controlled by contextual cues associated with the development of drug seeking habits, response contingent cues that accompany delivery of the drug, as well as internal states that correlate with levels of drug intake. These multiple sources of control over drug seeking are discussed in relation to the generation of an addicted phenotype in animal models and the hypothesized progression from internal control over drug use to compulsive drug seeking.

Defining the place of habit in substance use disorders

It has long been suggested that alcohol or substance use disorders could emerge from the progressive development and dominance of drug habits. Like habits, drug-related behaviors are often triggered by drug-associated cues. Like habits, addictive behaviors are strong, rigid and "hard to break". Like habits, these behaviors are insensitive to their outcome and persist despite negative consequences. "Pathological habit" thus appears as a good candidate to explain the transition to compulsive drug use. However, drug use could also be considered as a goal-directed choice, driven by the expectation of drug outcomes. For example, drug addicts may engage in drug-seeking behaviors because they view the drug as more valuable than available alternatives. Substance use disorders therefore may not be all about habit, nor fully intentional, and could be considered as resulting from an imbalance between goal-directed and habitual control. The main objective of this review is to disentangle the relative contribution of habit formation and impairment of goal-directed behavior in this unbalanced control of addictive behaviors. Although deficits in goal-directed behavior have been demonstrated in alcohol and substance use disorders, reliable demonstration of abnormal habit formation has been curtailed by the paucity of paradigms designed to assess habit as a positive result. Refining our animal and human model of habit is therefore required to precisely define the place of habit in substance use disorders and develop appropriate and adapted neurobehavioral treatments.

Inhibiting Rho kinase promotes goal-directed decision making and blocks habitual responding for cocaine

The prelimbic prefrontal cortex is necessary for associating actions with their consequences, enabling goal-directed decision making. We find that the strength of action–outcome conditioning correlates with dendritic spine density in prelimbic cortex, suggesting that new action–outcome learning involves dendritic spine plasticity. To test this, we inhibited the cytoskeletal regulatory factor Rho kinase. We find that the inhibitor fasudil enhances action–outcome memory, resulting in goal-directed behavior in mice that would otherwise express stimulus-response habits. Fasudil transiently reduces prelimbic cortical dendritic spine densities during a period of presumed memory consolidation, but only when paired with new learning. Fasudil also blocks habitual responding for cocaine, an effect that persists over time, across multiple contexts, and depends on actin polymerization. We suggest that Rho kinase inhibition promotes goal-oriented action selection by augmenting the plasticity of prelimbic cortical dendritic spines during the formation of new action–outcome memories.

Action-outcome learning requires the prelimbic prefrontal cortex. Here the authors report that fasudil, a Rho kinase inhibitor, reduces dendritic spine densities on prelimbic neurons in an activity-dependent manner, stimulating goal-directed actions, and reducing habitual responding for cocaine.

Model-Based Control in Dimensional Psychiatry

We use parallel interacting goal-directed and habitual strategies to make our daily decisions. The arbitration between these strategies is relevant to inflexible repetitive behaviors in psychiatric disorders. Goal-directed control, also known as model-based control, is based on an affective outcome relying on a learned internal model to prospectively make decisions. In contrast, habit control, also known as model-free control, is based on an integration of previous reinforced learning autonomous of the current outcome value and is implicit and more efficient but at the cost of greater inflexibility. The concept of model-based control can be further extended into pavlovian processes. Here we describe and compare tasks that tap into these constructs and emphasize the clinical relevance and translation of these tasks in psychiatric disorders. Together, these findings highlight a role for model-based control as a transdiagnostic impairment underlying compulsive behaviors and representing a promising therapeutic target.

Prior Cocaine Self-Administration Increases Response-Outcome Encoding That Is Divorced from Actions Selected in Dorsal Lateral Striatum

Dorsal lateral striatum (DLS) is a highly associative structure that encodes relationships among environmental stimuli, behavioral responses, and predicted outcomes. DLS is known to be disrupted after chronic drug abuse; however, it remains unclear what neural signals in DLS are altered. Current theory suggests that drug use enhances stimulus-response processing at the expense of response-outcome encoding, but this has mostly been tested in simple behavioral tasks. Here, we investigated what neural correlates in DLS are affected by previous cocaine exposure as rats performed a complex reward-guided decision-making task in which predicted reward value was independently manipulated by changing the delay to or size of reward associated with a response direction across a series of trial blocks. After cocaine self-administration, rats exhibited stronger biases toward higher-value reward and firing in DLS more strongly represented action-outcome contingencies independent from actions subsequently taken rather than outcomes predicted by selected actions (chosen-outcome contingencies) and associations between stimuli and actions (stimulus-response contingencies). These results suggest that cocaine self-administration strengthens action-outcome encoding in rats (as opposed to chosen-outcome or stimulus-response encoding), which abnormally biases behavior toward valued reward when there is a choice between two options during reward-guided decision-making.SIGNIFICANCE STATEMENT Current theories suggest that the impaired decision-making observed in individuals who chronically abuse drugs reflects a decrease in goal-directed behaviors and an increase in habitual behaviors governed by neural representations of response-outcome (R-O) and stimulus-response associations, respectively. We examined the impact that prior cocaine self-administration had on firing in dorsal lateral striatum (DLS), a brain area known to be involved in habit formation and affected by drugs of abuse, during performance of a complex reward-guided decision-making task. Surprisingly, we found that previous cocaine exposure enhanced R-O associations in DLS. This suggests that there may be more complex consequences of drug abuse than current theories have explored, especially when examining brain and behavior in the context of a complex two-choice decision-making task.

Subchronic anesthetic ketamine injections in rats impair choice reversal learning, but have no effect on reinforcer devaluation

Previous exposure to a variety of drugs of abuse has been shown to cause long-term impairments in reversal learning and reinforcer devaluation tasks. However, there is mixed evidence in the literature for a long-term effect of ketamine exposure on reversal learning and the long-term effect of ketamine exposure on devaluation is not known. We determined whether repeated injections of an anesthetic dose of ketamine would lead to impairments in choice reversal learning after discrimination learning or impairments in reinforcer devaluation. In two experiments, rats received three injections once-daily of ketamine (100 mg/kg, intraperitoneally) or saline and then began behavioral training 19 days later so that the key reversal learning and devaluation tests would occur about 1 month after the final ketamine injection. This ketamine exposure regimen did not impair learning in our discrimination task, but led to an increase in perseverative errors in reversal learning. However, the same ketamine exposure regimen (or injections of a lower 50 mg/kg dose) had no effect on behavior in the devaluation task. The behavioral patterns observed suggest possible neural mechanisms for the effects of ketamine, but future neurobiological investigations will be needed to isolate these mechanisms.

Drug Addiction: Updating Actions to Habits to Compulsions Ten Years On

A decade ago, we hypothesized that drug addiction can be viewed as a transition from voluntary, recreational drug use to compulsive drug-seeking habits, neurally underpinned by a transition from prefrontal cortical to striatal control over drug seeking and taking as well as a progression from the ventral to the dorsal striatum. Here, in the light of burgeoning, supportive evidence, we reconsider and elaborate this hypothesis, in particular the refinements in our understanding of ventral and dorsal striatal mechanisms underlying goal-directed and habitual drug seeking, the influence of drug-associated Pavlovian-conditioned stimuli on drug seeking and relapse, and evidence for impairments in top-down prefrontal cortical inhibitory control over this behavior. We further review animal and human studies that have begun to define etiological factors and individual differences in the propensity to become addicted to drugs, leading to the description of addiction endophenotypes, especially for cocaine addiction. We consider the prospect of novel treatments for addiction that promote abstinence from and relapse to drug use.

Associative and sensorimotor cortico-basal ganglia circuit roles in effects of abused drugs

The mammalian forebrain is characterized by the presence of several parallel cortico-basal ganglia circuits that shape the learning and control of actions. Among these are the associative, limbic and sensorimotor circuits. The function of all of these circuits has now been implicated in responses to drugs of abuse, as well as drug seeking and drug taking. While the limbic circuit has been most widely examined, key roles for the other two circuits in control of goal-directed and habitual instrumental actions related to drugs of abuse have been shown. In this review we describe the three circuits and effects of acute and chronic drug exposure on circuit physiology. Our main emphasis is on drug actions in dorsal striatal components of the associative and sensorimotor circuits. We then review key findings that have implicated these circuits in drug seeking and taking behaviors, as well as drug use disorders. Finally, we consider different models describing how the three cortico-basal ganglia circuits become involved in drug-related behaviors. This topic has implications for drug use disorders and addiction, as treatments that target the balance between the different circuits may be useful for reducing excessive substance use.

Prefrontal cortical BDNF: A regulatory key in cocaine- and food-reinforced behaviors

Brain-derived neurotrophic factor (BDNF) affects synaptic plasticity and neural structure and plays key roles in learning and memory processes. Recent evidence also points to important, yet complex, roles for BDNF in rodent models of cocaine abuse and addiction. Here we examine the role of prefrontal cortical (PFC) BDNF in reward-related decision making and behavioral sensitivity to, and responding for, cocaine. We focus on BDNF within the medial and orbital PFC, its regulation by cocaine during early postnatal development and in adulthood, and how BDNF in turn influences responding for drug reinforcement, including in reinstatement models. When relevant, we draw comparisons and contrasts with experiments using natural (food) reinforcers. We also summarize findings supporting, or refuting, the possibility that BDNF in the medial and orbital PFC regulate the development and maintenance of stimulus-response habits. Further investigation could assist in the development of novel treatment approaches for cocaine use disorders.

Going and stopping: Dichotomies in behavioral control by the prefrontal cortex

The rodent dorsal medial prefrontal cortex (PFC), specifically the prelimbic cortex (PL), regulates the expression of conditioned fear and behaviors interpreted as reward-seeking. Meanwhile, the ventral medial PFC, namely the infralimbic cortex (IL), is essential to extinction conditioning in both appetitive and aversive domains. Here we review evidence that supports, or refutes, this "PL-go/IL-stop" dichotomy. We focus on the extinction of conditioned fear and the extinction and reinstatement of cocaine- or heroin-reinforced responding. We then synthesize evidence that the PL is essential for developing goal-directed response strategies, while the IL supports habit behavior. Finally, we propose that some functions of the orbital PFC parallel those of the medial PFC in the regulation of response selection. Integration of these discoveries may provide points of intervention for inhibiting untethered drug seeking in drug use disorders, failures in extinction in Post-traumatic Stress Disorder, or co-morbidities between the two.

A Closer Look at the Effects of Repeated Cocaine Exposure on Adaptive Decision-Making under Conditions That Promote Goal-Directed Control

It has been proposed that compulsive drug seeking reflects an underlying dysregulation in adaptive behavior that favors habitual (automatic and inflexible) over goal-directed (deliberative and highly flexible) action selection. Rodent studies have established that repeated exposure to cocaine or amphetamine facilitates the development of habits, producing behavior that becomes unusually insensitive to a reduction in the value of its outcome. The current study more directly investigated the effects of cocaine pre-exposure on goal-directed learning and action selection using an approach that discourages habitual performance. After undergoing a 15-day series of cocaine (15 or 30 mg/kg, i.p.) or saline injections and a drug withdrawal period, rats were trained to perform two different lever-press actions for distinct reward options. During a subsequent outcome devaluation test, both cocaine- and saline-treated rats showed a robust bias in their choice between the two actions, preferring whichever action had been trained with the reward that retained its value. Thus, it appears that the tendency for repeated cocaine exposure to promote habit formation does not extend to a more complex behavioral scenario that encourages goal-directed control. To further explore this issue, we assessed how prior cocaine treatment would affect the rats' ability to learn about a selective reduction in the predictive relationship between one of the two actions and its outcome, which is another fundamental feature of goal-directed behavior. Interestingly, we found that cocaine-treated rats showed enhanced, rather than diminished, sensitivity to this action-outcome contingency degradation manipulation. Given their mutual dependence on striatal dopamine signaling, we suggest that cocaine's effects on habit formation and contingency learning may stem from a common adaptation in this neurochemical system.

Memory Systems and the Addicted Brain

The view that anatomically distinct memory systems differentially contribute to the development of drug addiction and relapse has received extensive support. The present brief review revisits this hypothesis as it was originally proposed 20 years ago (1) and highlights several recent developments. Extensive research employing a variety of animal learning paradigms indicates that dissociable neural systems mediate distinct types of learning and memory. Each memory system potentially contributes unique components to the learned behavior supporting drug addiction and relapse. In particular, the shift from recreational drug use to compulsive drug abuse may reflect a neuroanatomical shift from cognitive control of behavior mediated by the hippocampus/dorsomedial striatum toward habitual control of behavior mediated by the dorsolateral striatum (DLS). In addition, stress/anxiety may constitute a cofactor that facilitates DLS-dependent memory, and this may serve as a neurobehavioral mechanism underlying the increased drug use and relapse in humans following stressful life events. Evidence supporting the multiple systems view of drug addiction comes predominantly from studies of learning and memory that have employed as reinforcers addictive substances often considered within the context of drug addiction research, including cocaine, alcohol, and amphetamines. In addition, recent evidence suggests that the memory systems approach may also be helpful for understanding topical sources of addiction that reflect emerging health concerns, including marijuana use, high-fat diet, and video game playing.

Corticostriatal circuitry and habitual ethanol seeking

The development of alcohol-use disorders is thought to involve a transition from casual alcohol use to uncontrolled alcohol-seeking behavior. This review will highlight evidence suggesting that the shift toward inflexible alcohol seeking that occurs across the development of addiction consists, in part, of a progression from goal-directed to habitual behaviors. This shift in "response strategy" is thought to be largely regulated by corticostriatal network activity. Indeed, specific neuroanatomical substrates within the prefrontal cortex and the striatum have been identified as playing opposing roles in the expression of actions and habits. A majority of the research on the neurobiology of habitual behavior has focused on non-drug reward seeking. Here, we will highlight recent research identifying corticostriatal structures that regulate the expression of habitual alcohol seeking and a comparison will be made when possible to findings for non-drug rewards.

Synaptic Cytoskeletal Plasticity in the Prefrontal Cortex Following Psychostimulant Exposure

Addiction is characterized by maladaptive decision-making, a loss of control over drug consumption and habit-like drug seeking despite adverse consequences. These cognitive changes may reflect the effects of drugs of abuse on prefrontal cortical neurobiology. Here, we review evidence that amphetamine and cocaine fundamentally remodel the structure of excitatory neurons in the prefrontal cortex. We summarize evidence in particular that these psychostimulants have opposing effects in the medial and orbital prefrontal cortices ('mPFC' and 'oPFC', respectively). For example, amphetamine and cocaine increase dendrite length and spine density in the mPFC, while dendrites are impoverished and dendritic spines are eliminated in the oPFC. We will discuss evidence that certain cytoskeletal regulatory proteins expressed in the oPFC and implicated in postnatal (adolescent) neural development also regulate behavioral sensitivity to cocaine. These findings potentially open a window of opportunity for the identification of novel pharmacotherapeutic targets in the treatment of drug abuse disorders in adults, as well as in drug-vulnerable adolescent populations. Finally, we will discuss the behavioral implications of drug-related dendritic spine elimination in the oPFC, with regard to reversal learning tasks and tasks that assess the development of reward-seeking habits, both used to model aspects of addiction in rodents.