Shift from goal-directed to habitual cocaine seeking after prolonged experience in rats

Outcome devaluation as a method for identifying goal-directed behaviors in rats

Goal-directed behaviors allow animals to act to satisfy needs and desires. The outcome devaluation task is an effective method for identifying goal-directed behaviors and distinguishing these from other types of behavior. Rats can be trained to lever-press for one or multiple distinct food rewards. During testing, the previously earned food-or a control food for comparison-is devalued by allowing the animal to freely feed on it until they are sated before testing lever-press performance under extinction conditions (no rewards are delivered). Behavior that adapts to reflect the new value of the outcome is considered goal-directed, whereas behavior that continues as in previous training despite the change in outcome value, is not. As more research groups have used this task, variability in the procedures used has increased. Here, we provide a reliable procedure for conducting the outcome devaluation task with appropriate controls. We describe the most common variants of the task and control conditions and discuss troubleshooting measures such as outcome pre-exposure, habituation to pre-feeding chambers and attention to animals' hunger levels. The method outlined can be executed in ~2 weeks including training (~8 d) and testing (1-4 d) by researchers who are familiar with performing behavioral tasks in laboratory rodents, although longer training may be considered for those who are interested in observing habitual control of behavior. This protocol should facilitate the comparison of results from different studies and laboratories, while allowing flexibility in the application of the outcome devaluation task to different research questions.

Ceftriaxone reverses diet-induced deficits in goal-directed control

RATIONALE

Obesity is associated with numerous health risks and ever-increasing rates are a significant global concern. However, despite weight loss attempts many people have difficulty maintaining weight loss. Previous studies in animals have shown that chronic access to an obesogenic diet can disrupt goal-directed behavior, impairing the ability of animals to flexibly adjust food-seeking behavior following changes in the value of earned outcomes. Changes in behavioral control have been linked to disruption of glutamate transmission in the dorsal medial striatum (DMS), a region critical for the acquisition and expression of goal-directed behavior.

OBJECTIVES

The goal of this study was to test whether ceftriaxone, a beta-lactam antibiotic shown elsewhere to upregulate the expression of the glutamate transporter GLT-1, would improve goal-directed control following long-term exposure to an obesogenic diet.

METHODS

Male and female rats were given access to either standard chow or chow plus sweetened condensed milk (SCM) for 6 weeks. Access to SCM was ended and rats received daily injections of either ceftriaxone or saline for 6 days. Rats were then trained to press a lever to earn a novel food reward and, finally, were assessed for sensitivity to outcome devaluation. Histological analyses examined changes to GLT-1 protein levels and morphological changes to astrocytes, within the DMS.

RESULTS

We found that ceftriaxone robustly restored goal-directed behavior in animals following long-term exposure to SCM. While we did not observe changes in protein levels of GLT-1 in the DMS, we observed that SCM induced changes in the morphology of astrocytes in the DMS, and that ceftriaxone mitigated these changes.

CONCLUSIONS

These results demonstrate that long-term access to a SCM diet impairs goal-directed behavior while also altering the morphology of astrocytes in the DMS. Furthermore, these results suggest that ceftriaxone administration can reverse the impairment of goal-directed behavior potentially through its actions on astrocytes in decision-making circuitry.

A dual-pathway architecture enables chronic stress to disrupt agency and promote habit formation

Chronic stress can change how we learn and, thus, how we make decisions. Here we investigated the neuronal circuit mechanisms that enable this. Using a multifaceted systems neuroscience approach in male and female mice, we reveal a dual pathway, amygdala-striatal neuronal circuit architecture by which a recent history of chronic stress disrupts the action-outcome learning underlying adaptive agency and promotes the formation of inflexible habits. We found that the basolateral amygdala projection to the dorsomedial striatum is activated by rewarding events to support the action-outcome learning needed for flexible, goal-directed decision making. Chronic stress attenuates this to disrupt action-outcome learning and, therefore, agency. Conversely, the central amygdala projection to the dorsomedial striatum mediates habit formation. Following stress this pathway is progressively recruited to learning to promote the premature formation of inflexible habits. Thus, stress exerts opposing effects on two amygdala-striatal pathways to disrupt agency and promote habit. These data provide neuronal circuit insights into how chronic stress shapes learning and decision making, and help understand how stress can lead to the disrupted decision making and pathological habits that characterize substance use disorders and mental health conditions.

Role of dorsal striatum circuits in relapse to opioid seeking after voluntary abstinence

High relapse rate during abstinence is a defining characteristic of drug addiction. We previously found that opioid seeking progressively increases after voluntary abstinence induced by adverse consequences of oxycodone seeking (crossing an electric barrier). Functional MRI revealed that this effect is associated with changes in functional connectivity within medial orbitofrontal cortex (mOFC)- and dorsomedial striatum (DMS)-related circuits. Here, we used a pharmacological manipulation and fMRI to determine the causal role of mOFC and DMS in oxycodone seeking after electric barrier-induced abstinence. We trained rats to self-administer oxycodone (6 h/day, 14 days). Next, we induced voluntary abstinence by exposing them to an electric barrier for 2 weeks. We inactivated the mOFC and DMS with muscimol+baclofen (GABAa and GABAb receptor agonists) and then tested them for relapse to oxycodone seeking on abstinence days 1 or 15 without the electric barrier or oxycodone. Inactivation of DMS (p < 0.001) but not mOFC decreased oxycodone seeking before or after electric barrier-induced abstinence. Functional MRI data revealed that DMS inactivation decreased cerebral blood volume levels in DMS and several distant cortical and subcortical regions (corrected p < 0.05). Furthermore, functional connectivity of DMS with several frontal, sensorimotor, and auditory regions significantly increased after DMS inactivation (corrected p < 0.05). Finally, an exploratory analysis of an existing functional MRI dataset showed that DMS inactivation restored voluntary abstinence-induced longitudinal changes in DMS functional connectivity with these brain regions (p < 0.05). Results indicate a role of DMS and related brain circuits in oxycodone seeking after voluntary abstinence, suggesting potential targets for intervention.

Understanding the shift to compulsion in addiction: insights from personality traits, social factors, and neurobiology

Compulsion stands as a central symptom of drug addiction; however, only a small fraction of individuals who use drugs exhibit compulsive characteristics. Differences observed in Sign-trackers (ST) and Goal-trackers (GT) during Pavlovian conditioning may shed light on individual variances in drug addiction. Here, we focus on the behavioral attributes, formation processes, and neural mechanisms underlying ST and how they drive addiction toward compulsivity in humans. We will explore addiction from three interconnected levels: individual personality traits, social factors, and neurobiology. Furthermore, we distinguish between the processes of sensitization and habituation within ST. These nuanced distinctions across various aspects of addiction will contribute to our understanding of the addiction development process and the formulation of targeted preventive strategies.

Adolescent nicotine exposure promotes adulthood opioid consumption that persists despite adverse consequences and increases the density of insular perineuronal nets

Adolescence marks a sensitive period for neurodevelopment wherein exposure to drugs of abuse may disrupt maturation and induce persistent changes in neurophysiology which may exacerbate the risk for developing substance use disorders in adulthood. Adolescent nicotine exposure (ANE) enhances motivation to obtain drugs of abuse, particularly opioids, and increases vulnerability for the development of opioid use disorder (OUD). Here, we characterized ANE effects on learning about the adverse consequences of opioid consumption in adulthood in the absence of further nicotine administration. First, we show that ANE engenders punishment resistant fentanyl self-administration in a heterogenous seeking-taking chain schedule of reinforcement at least at the tested dose of fentanyl (0.75 μg/kg). We found that ANE rats consumed significantly more fentanyl and contingent foot shock punishment was less efficacious in limiting fentanyl seeking in ANE rats, relative to nicotine-naïve controls. Next, we demonstrated that ANE limits learning about the deleterious consequences of acute opioid intoxication in adulthood. In a combined conditioned taste avoidance and place preference paradigm we found that ANE resulted in significant reductions in the strength of morphine-induced CTA, and a simultaneous enhancement of CPP at a higher dose that was less capable of driving reinforcement in naïve controls. Finally, we examined the expression of perineuronal nets (PNNs) within insular cortex (IC) and found ANE rats to have increased density of PNNs across the anterior IC and significantly more parvalbumin-labeled IC cells relative to naïve controls. Together, these data lay the framework for a mechanistic explanation of the extreme comorbidity between nicotine use and development of OUDs.

Punishment resistance for cocaine is associated with inflexible habits in rats

Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration. We then exposed them to four days of punishment testing in which footshock was delivered randomly on one-third of trials. Before and after punishment testing (four days pre-punishment and ≥ four days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance for cocaine was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment in males, although it was associated with habitual responding both pre- and post-punishment in females, indicating that punishment resistance was predicted by habitual responding in food-seeking females. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.

Pan-striatal reduction in the expression of the astrocytic dopamine transporter precedes the development of dorsolateral striatum dopamine-dependent incentive heroin seeking habits

The emergence of compulsive drug-seeking habits, a hallmark feature of substance use disorder, has been shown to be predicated on the engagement of dorsolateral striatal control over behaviour. This process involves the dopamine-dependent functional coupling of the anterior dorsolateral striatum (aDLS) with the nucleus accumbens core, but the mechanisms by which this coupling occurs have not been fully elucidated. The striatum is tiled by a syncytium of astrocytes that express the dopamine transporter (DAT), the level of which is altered in individuals with heroin use disorder. Astrocytes are therefore uniquely placed functionally to bridge dopamine-dependent mechanisms across the striatum. Here we tested the hypothesis that exposure to heroin influences the expression of DAT in striatal astrocytes across the striatum before the development of DLS-dependent incentive heroin seeking habits. Using Western-blot, qPCR, and RNAscope™, we measured DAT protein and mRNA levels in whole tissue, culture and in situ astrocytes from striatal territories of rats with a well-established cue-controlled heroin seeking habit and rats trained to respond for heroin or food under continuous reinforcement. Incentive heroin seeking habits were associated with a reduction in DAT protein levels in the anterior aDLS that was preceded by a heroin-induced reduction in DAT mRNA and protein in astrocytes across the striatum. Striatal astrocytes were also shown to be susceptible to direct dopamine- and opioid-induced downregulation of DAT expression. These results suggest that astrocytes may critically regulate the striatal dopaminergic adaptations that lead to the development of incentive heroin seeking habits.

Dopamine activity in the nigrostriatal pathway alters cue-induced risky choice patterns in female rats

Deficits in cost/benefit decision making is a critical risk factor for gambling disorder. Reward-paired cues may play an important role, as these stimuli can enhance risk preference in rats. Despite extensive research implicating the dorsal striatum in the compulsive aspects of addiction, the role of nigrostriatal dopaminergic activity in cue-induced risk preference remains unclear, particularly in females. Accordingly, we examined the effects of manipulating the dopaminergic nigrostriatal pathway on cue-induced risky choice in female rats. TH:Cre rats were trained on the cued version of the rat Gambling Task. This task was designed such that maximal reward is attained by avoiding the high-risk, high-reward options and instead favouring the options associated with lower per-trial gains, as they feature less frequent and shorter time-out penalties. Adding reward-paired audiovisual cues to the task leads to greater risky choice on average. To assess the role of the nigrostriatal pathway, a viral vector carrying either Cre-dependent inhibitory or excitatory DREADD was infused into the substantia nigra. Rats then received clozapine-N-oxide either during task acquisition or after a stable performance baseline was reached. Inhibition of this pathway accelerated the development of risk preference in early sessions and increased risky choice during performance, but long-term inhibition actually improved decision making. Activation of this pathway had minimal effects. These results provide evidence for the involvement of the dopaminergic nigrostriatal pathway in cue-induced risk preference in females, therefore shedding light on its role in cost/benefit decision-making deficits and expanding our knowledge of the female dopaminergic system.

Opening new vistas on obsessive-compulsive disorder with the observing response task

Obsessive-compulsive disorder (OCD), a highly prevalent and debilitating disorder, is incompletely understood in terms of underpinning behavioural, psychological, and neural mechanisms. This is attributable to high symptomatic heterogeneity; cardinal features comprise obsessions and compulsions, including clinical subcategories. While obsessive and intrusive thoughts are arguably unique to humans, dysfunctional behaviours analogous to those seen in clinical OCD have been examined in nonhuman animals. Genetic, ethological, pharmacological, and neurobehavioural approaches all contribute to understanding the emergence and persistence of compulsive behaviour. One behaviour of particular interest is maladaptive checking, whereby human patients excessively perform checking rituals despite these serving no purpose. Dysfunctional and excessive checking is the most common symptom associated with OCD and can be readily operationalised in rodents. This review considers animal models of OCD, the neural circuitries associated with impairments in habit-based and goal-directed behaviour, and how these may link to the compulsions observed in OCD. We further review the Observing Response Task (ORT), an appetitive instrumental learning procedure that distinguishes between functional and dysfunctional checking, with translational application in humans and rodents. By shedding light on the psychological and neural bases of compulsive-like checking, the ORT has potential to offer translational insights into the underlying mechanisms of OCD, in addition to being a platform for testing psychological and neurochemical treatment approaches.

Making and breaking habits: Revisiting the definitions and behavioral factors that influence habits in animals

Habits have garnered significant interest in studies of associative learning and maladaptive behavior. However, habit research has faced scrutiny and challenges related to the definitions and methods. Differences in the conceptualizations of habits between animal and human studies create difficulties for translational research. Here, we review the definitions and commonly used methods for studying habits in animals and humans and discuss potential alternative ways to assess habits, such as automaticity. To better understand habits, we then focus on the behavioral factors that have been shown to make or break habits in animals, as well as potential mechanisms underlying the influence of these factors. We discuss the evidence that habitual and goal-directed systems learn in parallel and that they seem to interact in competitive and cooperative manners. Finally, we draw parallels between habitual responding and compulsive drug seeking in animals to delineate the similarities and differences in these behaviors.

Multiple Memory Subsystems: Reconsidering Memory in the Mind and Brain

The multiple-memory-systems framework-that distinct types of memory are supported by distinct brain systems-has guided learning and memory research for decades. However, recent work challenges the one-to-one mapping between brain structures and memory types central to this taxonomy, with key memory-related structures supporting multiple functions across substructures. Here we integrate cross-species findings in the hippocampus, striatum, and amygdala to propose an updated framework of multiple memory subsystems (MMSS). We provide evidence for two organizational principles of the MMSS theory: First, opposing memory representations are colocated in the same brain structures; second, parallel memory representations are supported by distinct structures. We discuss why this burgeoning framework has the potential to provide a useful revision of classic theories of long-term memory, what evidence is needed to further validate the framework, and how this novel perspective on memory organization may guide future research.

The development of compulsive coping behavior depends on dorsolateral striatum dopamine-dependent mechanisms

Humans greatly differ in how they cope with stress, a natural behavior learnt through negative reinforcement. Some individuals engage in displacement activities, others in exercise or comfort eating, and others still in alcohol use. Across species, adjunctive behaviors, such as polydipsic drinking, are used as a form of displacement activity that reduces stress. Some individuals, in particular those that use alcohol to self-medicate, tend to lose control over such coping behaviors, which become excessive and compulsive. However, the psychological and neural mechanisms underlying this individual vulnerability have not been elucidated. Here we tested the hypothesis that the development of compulsive adjunctive behaviors stems from the functional engagement of the dorsolateral striatum (DLS) dopamine-dependent habit system after a prolonged history of adjunctive responding. We measured in longitudinal studies in male Sprague Dawley rats the sensitivity of early established vs compulsive polydipsic water or alcohol drinking to a bilateral infusion into the anterior DLS (aDLS) of the dopamine receptor antagonist α-flupentixol. While most rats acquired a polydipsic drinking response with water, others only did so with alcohol. Whether drinking water or alcohol, the acquisition of this coping response was insensitive to aDLS dopamine receptor blockade. In contrast, after prolonged experience, adjunctive drinking became dependent on aDLS dopamine at a time when it was compulsive in vulnerable individuals. These data suggest that habits may develop out of negative reinforcement and that the engagement of their underlying striatal system is necessary for the manifestation of compulsive adjunctive behaviors.

Punishment resistance for cocaine is associated with inflexible habits in rats

Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration (2 h/day). We then exposed them to 4 days of punishment testing, in which footshock (0.4 mA, 0.3 s) was delivered randomly on one-third of trials, immediately following completion of seeking and prior to extension of the taking lever. Before and after punishment testing (4 days pre-punishment and ≥4 days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.

Glutamate dynamics in the dorsolateral striatum of rats with goal-directed and habitual cocaine-seeking behavior

The shift from drug abuse to addiction is considered to arise from the transition between goal-directed and habitual control over drug behavior. Habitual responding for appetitive and skill-based behaviors is mediated by potentiated glutamate signaling in the dorsolateral striatum (DLS), but the state of the DLS glutamate system in the context of habitual drug-behavior remains undefined. Evidence from the nucleus accumbens of cocaine-experienced rats suggests that decreased transporter-mediated glutamate clearance and enhanced synaptic glutamate release contribute to the potentiated glutamate signaling that underlies the enduring vulnerability to relapse. Preliminary evidence from the dorsal striatum of cocaine-experienced rats suggests that this region exhibits similar alterations to glutamate clearance and release, but it is not known whether these glutamate dynamics are associated with goal-directed or habitual control over cocaine-seeking behavior. Therefore, we trained rats to self-administer cocaine in a chained cocaine-seeking and -taking paradigm, which yielded goal-directed, intermediate, and habitual cocaine-seeking rats. We then assessed glutamate clearance and release dynamics in the DLS of these rats using two different methods: synaptic transporter current (STC) recordings of patch-clamped astrocytes and the intensity-based glutamate sensing fluorescent reporter (iGluSnFr). While we observed a decreased rate of glutamate clearance in STCs evoked with single-pulse stimulation in cocaine-experienced rats, we did not observe any cocaine-induced differences in glutamate clearance rates from STCs evoked with high frequency stimulation (HFS) or iGluSnFr responses evoked with either double-pulse stimulation or HFS. Furthermore, GLT-1 protein expression in the DLS was unchanged in cocaine-experienced rats, regardless of their mode of control over cocaine-seeking behavior. Lastly, there were no differences in metrics of glutamate release between cocaine-experienced rats and yoked-saline controls in either assay. Together, these results suggest that glutamate clearance and release dynamics in the DLS are largely unaltered by a history of cocaine self-administration on this established cocaine seeking-taking paradigm, regardless of whether the control over the cocaine seeking behavior was habitual or goal directed.

Goal-directed and habitual decision making under stress in gambling disorder: An fMRI study

The development of addictive behaviors has been suggested to be related to a transition from goal-directed to habitual decision making. Stress is a factor known to prompt habitual behavior and to increase the risk for addiction and relapse. In the current study, we therefore used functional MRI to investigate the balance between goal-directed 'model-based' and habitual 'model-free' control systems and whether acute stress would differentially shift this balance in gambling disorder (GD) patients compared to healthy controls (HCs). Using a within-subject design, 22 patients with GD and 20 HCs underwent stress induction or a control condition before performing a multistep decision-making task during fMRI. Salivary cortisol levels showed that the stress induction was successful. Contrary to our hypothesis, GD patients did not show impaired goal-directed 'model-based' decision making, which remained similar to HCs after stress induction. Bayes factors provided three times more evidence against a difference between the groups or a group-by-stress interaction on the balance between model-based and model-free decision making. Similarly, no differences were found between groups and conditions on the neural estimates of model-based or model-free decision making. These results challenge the notion that GD is related to an increased reliance on habitual (or decreased goal-directed) control, even during stress.

Computational Mechanisms of Addiction and Anxiety: A Developmental Perspective

A central goal of computational psychiatry is to identify systematic relationships between transdiagnostic dimensions of psychiatric symptomatology and the latent learning and decision-making computations that inform individuals' thoughts, feelings, and choices. Most psychiatric disorders emerge prior to adulthood, yet little work has extended these computational approaches to study the development of psychopathology. Here, we lay out a roadmap for future studies implementing this approach by developing empirically and theoretically informed hypotheses about how developmental changes in model-based control of action and Pavlovian learning processes may modulate vulnerability to anxiety and addiction. We highlight how insights from studies leveraging computational approaches to characterize the normative developmental trajectories of clinically relevant learning and decision-making processes may suggest promising avenues for future developmental computational psychiatry research.

Evaluation of the 5-HT2C receptor drugs RO 60-0175, WAY 161503 and mirtazepine in a preclinical model of comorbidity of depression and cocaine addiction

BACKGROUND

Epidemiological data indicate a high rate of comorbidity of depression and cocaine use disorder (CUD). The role of serotonin 2C (5-HT_2C) receptors in the mechanisms responsible for the coexistence of depression and CUD was not investigated.

METHODS

We combined bilateral olfactory bulbectomy (OBX), an animal model of depression, with intravenous cocaine self-administration and extinction/reinstatement in male rats to investigate two 5-HT_2C receptor agonists (Ro 60-0175 (RO) and WAY 161503 (WAY)) and the 5-HT_2C-receptor preferring antagonist mirtazapine (MIR; an antidepressant), with the goal of determining whether these drugs alter cocaine-induced reinforcement and seeking behaviors. Additionally, neurochemical analyses were performed following cocaine self-administration and its abstinence period in the brain structures in OBX rats and SHAM-operated controls.

RESULTS

Acute administration of RO reduced, while WAY non-significantly attenuated cocaine reinforcement in both rat phenotypes. Moreover, RO or WAY protected against cocaine-seeking behavior after acute or after repeated drug administration during extinction training in OBX and SHAM rats. By contrast, acutely administered MIR did not alter cocaine reinforcement in both rat phenotypes, while it's acute (but not repeated) pretreatment reduced cocaine-seeking in OBX and SHAM rats. In neurochemical analyses, cocaine reinforcement increased 5-HT_2C receptor levels in the ventral hippocampus; a preexisting depression-like phenotype enhanced this effect. The 10-daily cocaine abstinence reduced 5-HT_2C receptor expression in the dorsolateral striatum, while the coexistence of depression and CUD enhanced local receptor expression.

CONCLUSION

The results support a key role of 5-HT_2C receptors for treating CUD and comorbid depression and CUD. They may be backs the further research of pharmacological strategies with drug targeting receptors.

Intermittent access training produces greater motivation for a non-drug reinforcer than long access training

It has recently been proposed that the intermittent access (IntA) drug self-administration procedure better produces behavioral changes relevant to addiction than the long access (LgA) procedure. In this version of the IntA procedure, the drug is made available for a 5-min period during each half hour of a 6-h session. In contrast, on the LgA procedure, the drug is available continuously for 6 h. Previous studies have found that IntA drug self-administration produces greater drug motivation, measured by increased progressive ratio breakpoints, than LgA self-administration. It has been hypothesized that this effect is due to the rapid, "spiking" brain levels of the drug, and consequent neuroadaptations, experienced by rats during IntA sessions. However, no study has compared the effects of IntA versus LgA training on reinforcer motivation when using a non-drug reinforcer. The present study compared motivation for a saccharin reinforcer after IntA or LgA training. In Experiment 1, separate groups of rats lever-pressed for saccharin on the IntA or LgA procedures. In Experiment 2, a within-subjects design was used where rats pressed one lever on the IntA procedure and another lever on the LgA procedure for saccharin. In both experiments, IntA training produced greater breakpoints than LgA training. As no drug was used here, spiking drug levels could not have been responsible for the increased saccharin motivation observed after IntA training. Instead, it is proposed that differences in stimulus-reinforcer associations learned during IntA versus LgA training may be responsible for the effect. Future research is needed to determine the extent to which such learning factors may contribute to the increased motivation observed after IntA training with drug reinforcers.

Discriminating goal-directed and habitual cocaine seeking in rats using a novel outcome devaluation procedure

Habits are theorized to play a key role in compulsive cocaine seeking, yet there is limited methodology for assessing habitual responding for intravenous (IV) cocaine. We developed a novel outcome devaluation procedure to discriminate goal-directed from habitual responding in cocaine-seeking rats. This procedure elicits devaluation temporarily and requires no additional training, allowing repeated testing at different time points. After training male rats to self-administer IV cocaine, we devalued the drug outcome via experimenter-administered IV cocaine (a "satiety" procedure) prior to a 10-min extinction test. Many rats were sensitive to outcome devaluation, a hallmark of goal-directed responding. These animals reduced responding when given a dose of experimenter-administered cocaine that matched or exceeded satiety levels during self-administration. However, other rats were insensitive to experimenter-administered cocaine, suggesting their responding was habitual. Importantly, reinforcement schedules and neural manipulations that produce goal-directed responding (i.e., ratio schedules or dorsolateral striatum lesions) caused sensitivity to outcome devaluation, whereas reinforcement schedules and neural manipulations that produce habitual responding (i.e., interval schedules or dorsomedial striatum lesions) caused insensitivity. Satiety-based outcome devaluation is an innovative new tool to dissect the neural and behavioral mechanisms underlying IV cocaine-seeking behavior.

Making habits measurable beyond what they are not: A focus on associative dual-process models

Habits are the subject of intense international research. Under the associative dual-process model the outcome devaluation paradigm has been used extensively to classify behaviours as being either goal-directed (sensitive to shifts in the value of associated outcomes) or habitual (triggered by stimuli without anticipation of consequences). This has proven to be a useful framework for studying the neurobiology of habit and relevance of habits in clinical psychopathology. However, in recent years issues have been raised about this rather narrow definition of habits in comparison to habitual behaviour experienced in the real world. Specifically, defining habits as the absence of goal-directed control, the very specific set-ups required to demonstrate habit experimentally and the lack of direct evidence for habits as stimulus-response behaviours are viewed as problematic. In this review paper we address key critiques that have been raised about habit research within the framework of the associative dual-process model. We then highlight novel research approaches studying different features of habits with methods that expand beyond traditional paradigms.

Alterations of Dopamine Receptors and the Adaptive Changes of L-Type Calcium Channel Subtypes Regulate Cocaine-Seeking Habit in Tree Shrew

The putamen (Put) is necessary for habitual actions, while the nucleus caudate (Cd) is critical for goal-directed actions. However, compared with the natural reward (such as sucrose)-seeking habit, how drug-related dysfunction or imbalance between the Put and Cd is involved in cocaine-seeking habit, which is not easy to bias behavior to goal-directed actions, is absent. Therefore, in our present study, in comparison with sucrose-habitual behavior, we evaluated the distinctive changes of the two subtypes of dopamine (DA) receptors (D1R and D2R) in cocaine-seeking habitual behavior animals. Moreover, the adaptive changes of Ca_v1.2 and Ca_v1.3, as prime downstream targets of D1R and D2R respectively, were also assessed. Our results showed that a similar percentage of the animals exhibited habitual seeking behavior after cocaine or sucrose variable-interval self-administration (SA) training in tree shrews. In addition, compared with animals with non-habitual behavior, animals with cocaine habitual behavior showed higher D1Rs and Ca_v1.2 expression in the Put accompanied with lower D2Rs and Ca_v1.3 expression in the Cd. However, after sucrose SA training, animals with habitual behavior only showed lower membrane expression of D2R in the Put than animals with non-habitual behavior. These results suggested that the upregulation of D1Rs-Ca_v1.2 signaling may lead to hyper-excitability of the Put, and the inactivation of D2Rs-Ca_v1.3 signaling may result in depressed activity in the Cd. This imbalance function between the Put and Cd, which causes an inability to shift between habits and goal-directed actions, may underlie the compulsive addiction habit.

Evaluating habit formation across pairs of female and male selectively bred alcohol-preferring and non-preferring rats

Some individuals with alcohol use disorder (AUD) continue to drink because they have developed a habit where they do not consider the consequences of their actions. Genetically selected lines of alcohol-preferring and non-preferring rats allow for exploration of how specific endophenotypes, such as tendency to form habits, may be risk factors that interact with a genetic predisposition of AUD. While high alcohol drinking (HAD) and alcohol-preferring (P) rats were selectively bred to consume high amounts of freely available ethanol, they exhibit differences in alcohol-seeking behaviors as well as impulsive behaviors, and may represent different behavioral models of AUD. The goal of the current study was to compare the tendency to develop habitual behaviors across female and male HAD1, HAD2, and P rats and their respective alcohol non-preferring counterparts. Alcohol-naïve rats were trained on a variable interval schedule using a non-ethanol reinforcer and were then tested in two extinction sessions, one prior to a reinforcer devaluation (conditioned taste aversion) procedure and one after. There were no differences in total lever presses between P and alcohol non-preferring (NP) rats, but there were differences between HAD and low-alcohol drinking (LAD) rats. All six strains decreased lever pressing after reinforcer devaluation. However, P and NP females did not increase latency to first lever press after devaluation, suggesting some inclination toward habitual behavior that was not apparent in either the HAD or LAD lines. Selective breeding for alcohol preference does not seem to influence the tendency to form habits, whereas background strain and sex may have an influence on this behavior.

Amyloid-β oligomers in the nucleus accumbens decrease motivation via insertion of calcium-permeable AMPA receptors

It is essential to identify the neuronal mechanisms of Alzheimer's Disease (AD)-associated neuropsychiatric symptoms, e.g., apathy, before improving the life quality of AD patients. Here, we focused on the nucleus accumbens (NAc), a critical brain region processing motivation, also known to display AD-associated pathological changes in human cases. We found that the synaptic calcium permeable (CP)-AMPA receptors (AMPARs), which are normally absent in the NAc, can be revealed by acute exposure to Aβ oligomers (AβOs), and play a critical role in the emergence of synaptic loss and motivation deficits. Blockade of NAc CP-AMPARs can effectively prevent AβO-induced downsizing and pruning of spines and silencing of excitatory synaptic transmission. We conclude that AβO-triggered synaptic insertion of CP-AMPARs is a key mechanism mediating synaptic degeneration in AD, and preserving synaptic integrity may prevent or delay the onset of AD-associated psychiatric symptoms.

Striatal dopamine signals are region specific and temporally stable across action-sequence habit formation

Habits are automatic, inflexible behaviors that develop slowly with repeated performance. Striatal dopamine signaling instantiates this habit-formation process, presumably region specifically and via ventral-to-dorsal and medial-to-lateral signal shifts. Here, we quantify dopamine release in regions implicated in these presumed shifts (ventromedial striatum [VMS], dorsomedial striatum [DMS], and dorsolateral striatum [DLS]) in rats performing an action-sequence task and characterize habit development throughout a 10-week training. Surprisingly, all regions exhibited stable dopamine dynamics throughout habit development. VMS and DLS signals did not differ between habitual and non-habitual animals, but DMS dopamine release increased during action-sequence initiation and decreased during action-sequence completion in habitual rats, whereas non-habitual rats showed opposite effects. Consistently, optogenetic stimulation of DMS dopamine release accelerated habit formation. Thus, we demonstrate that dopamine signals do not shift regionally during habit formation and that dopamine in DMS, but not VMS or DLS, determines habit bias, attributing "habit functions" to a region previously associated exclusively with non-habitual behavior.

On the interrelation between alcohol addiction-like behaviors in rats

RATIONALE

Alcohol use disorder (AUD) is a complex, heterogeneous disorder that only occurs in a minority of alcohol users. Various behavioral constructs, including excessive intake, habit formation, motivation for alcohol and resistance to punishment have been implicated in AUD, but their interrelatedness is unclear.

OBJECTIVE

The aim of this study was therefore to explore the relation between these AUD-associated behavioral constructs in rats. We hypothesised that a subpopulation of animals could be identified that, based on these measures, display consistent AUD-like behavior.

METHODS

Lister Hooded rats (n = 47) were characterised for alcohol consumption, habit formation, motivation for alcohol and quinine-adulterated alcohol consumption. The interrelation between these measures was evaluated through correlation and cluster analyses. In addition, addiction severity scores were computed using different combinations of the behavioral measures, to assess the consistency of the AUD-like subpopulation.

RESULTS

We found that the data was uniformly distributed, as there was no significant tendency of the behavioral measures to cluster in the dataset. On the basis of multiple ranked addiction severity scores, five animals (~ 11%) were classified as displaying AUD-like behavior. The composition of the remaining subpopulation of animals with the highest addiction severity score (9 rats; ~ 19%) varied, depending on the combination of measures included.

CONCLUSION

Consistent AUD-like behavior was detected in a small proportion of alcohol drinking rats. Alcohol consumption, habit formation, motivation for alcohol and punishment resistance contribute in varying degrees to the AUD-like phenotype across the population. These findings emphasise the importance of considering the heterogeneity of AUD-like behavior.

Cocaine cue-induced mesocorticolimbic activation in cocaine users: Effects of personality traits, lifetime drug use, and acute stimulant ingestion

Stimulant drug-paired cues can acquire the ability to activate mesocorticolimbic pathways and lead to new bouts of drug use. Studies in laboratory animals suggest that these effects are augmented by progressively greater drug use histories, impulsive personality traits, and acute drug ingestion. As a preliminary test of these hypotheses in humans, we exposed cocaine users (n = 14) and healthy volunteers (n = 10) to cocaine-related videos during two functional magnetic resonance imaging (fMRI) sessions, once following acute administration of placebo and once following d-amphetamine (0.3 mg/kg, p.o.). Across sessions, cocaine users showed larger cocaine cue-induced responses than healthy controls in the associative striatum and midbrain. Among the cocaine users, larger drug cue-induced responses during the placebo session were correlated with higher Barratt Impulsiveness Scale (BIS-11) nonplanning scores (associative striatum) and greater lifetime use of stimulant drugs (limbic, associative, and sensorimotor striatum). The administration of d-amphetamine did not augment the cue-induced activations, but, in cocaine users, drug cue-induced striatal activations were more widespread following prolonged cocaine cue exposure. Together, these effects of past and present drug use might aggravate the risk for stimulant drug use problems.

Deconstructing and reconstructing behaviour relevant to mental health disorders: The benefits of a psychological approach, with a focus on addiction

RUTHERFORD, L.G. and Milton, A.L. Deconstructing and reconstructing behaviour relevant to mental health disorders: what can psychology offer? NEUROSCI BIOBEHAV REV XX(X)XXX-XXX, 2021. - Current treatments for mental health disorders are successful only for some patients, and there is an unmet clinical need for new treatment development. One challenge for treatment development has been how best to model complex human conditions in animals, where mechanism can be more readily studied with a range of neuroscientific techniques. We suggest that an approach to modelling based on associative animal learning theory provides a good framework for deconstructing complex mental health disorders such that they can be studied in animals. These individual simple models can subsequently be used in combination to 'reconstruct' a more complex model of the mental health disorder of interest. Using examples primarily from the field of drug addiction, we explore the 'psychological approach' and suggest that in addition to facilitating translation and backtranslation of tasks between animal models and patients, it is also highly concordant with the concept of triangulation.

Corticostriatal Activity Driving Compulsive Reward Seeking

BACKGROUND

Activation of the mesolimbic dopamine system is positively reinforcing. After repeated activation, some individuals develop compulsive reward-seeking behavior, which is a core symptom of addiction. However, the underlying neural mechanism remains elusive.

METHODS

We trained mice in a seek-take chain, rewarded by optogenetic dopamine neuron self-stimulation. After compulsivity was evaluated, AMPA/NMDA ratio was measured at three distinct corticostriatal pathways confirmed by retrograde labeling and anterograde synaptic connectivity. Fiber photometry method and chemogenetics were used to parse the contribution of orbitofrontal cortex afferents onto the dorsal striatum (DS) during the behavioral task. We established a causal link between DS activity and compulsivity using optogenetic inhibition.

RESULTS

Mice that persevered when seeking was punished exhibited an increased AMPA/NMDA ratio selectively at orbitofrontal cortex to DS synapses. In addition, an activity peak of spiny projection neurons in the DS at the moment of signaled reward availability was detected. Chemogenetic inhibition of orbitofrontal cortex neurons curbed the activity peak and reduced punished reward seeking, as did optogenetic hyperpolarization of spiny projection neurons time-locked to the cue predicting reward availability.

CONCLUSIONS

Our results suggest that compulsive individuals display stronger neuronal activity in the DS during the cue predicting reward availability even when at the risk of punishment, nurturing further compulsive reward seeking.

Striatonigrostriatal Spirals in Addiction

A biological reward system is integral to all animal life and humans are no exception. For millennia individuals have investigated this system and its influences on human behavior. In the modern day, with the US facing an ongoing epidemic of substance use without an effective treatment, these investigations are of paramount importance. It is well known that basal ganglia contribute to rewards and are involved in learning, approach behavior, economic choices, and positive emotions. This review aims to elucidate the physiological role of striatonigrostriatal (SNS) spirals, as part of basal ganglia circuits, in this reward system and their pathophysiological role in perpetuating addiction. Additionally, the main functions of neurotransmitters such as dopamine and glutamate and their receptors in SNS circuits will be summarized. With this information, the claim that SNS spirals are crucial intermediaries in the shift from goal-directed behavior to habitual behavior will be supported, making this circuit a viable target for potential therapeutic intervention in those with substance use disorders.

Impaired Learning From Negative Feedback in Stimulant Use Disorder: Dopaminergic Modulation

BACKGROUND

Drug-induced alterations to the dopamine system in stimulant use disorder (SUD) are hypothesized to impair reinforcement learning (RL). Computational modeling enables the investigation of the latent processes of RL in SUD patients, which could elucidate the nature of their impairments.

METHODS

We investigated RL in 44 SUD patients and 41 healthy control participants using a probabilistic RL task that assesses learning from reward and punishment separately. In an independent sample, we determined the modulatory role of dopamine in RL following a single dose of the dopamine D2/3 receptor antagonist amisulpride (400 mg) and the agonist pramipexole (0.5 mg) in a randomised, double-blind, placebo-controlled, crossover design. We analyzed task performance using computational modelling and hypothesized that RL impairments in SUD patients would be differentially modulated by a dopamine D2/3 receptor antagonist and agonist.

RESULTS

Computational analyses in both samples revealed significantly reduced learning rates from punishment in SUD patients compared with healthy controls, whilst their reward learning rates were not measurably impaired. In addition, the dopaminergic receptor agents modulated RL parameters differentially in both groups. Both amisulpride and pramipexole impaired RL parameters in healthy participants, but ameliorated learning from punishment in SUD patients.

CONCLUSION

Our findings suggest that RL impairments seen in SUD patients are associated with altered dopamine function.

Dopamine Circuit Mechanisms of Addiction-Like Behaviors

Addiction is a complex disease that impacts millions of people around the world. Clinically, addiction is formalized as substance use disorder (SUD), with three primary symptom categories: exaggerated substance use, social or lifestyle impairment, and risky substance use. Considerable efforts have been made to model features of these criteria in non-human animal research subjects, for insight into the underlying neurobiological mechanisms. Here we review evidence from rodent models of SUD-inspired criteria, focusing on the role of the striatal dopamine system. We identify distinct mesostriatal and nigrostriatal dopamine circuit functions in behavioral outcomes that are relevant to addictions and SUDs. This work suggests that striatal dopamine is essential for not only positive symptom features of SUDs, such as elevated intake and craving, but also for impairments in decision making that underlie compulsive behavior, reduced sociality, and risk taking. Understanding the functional heterogeneity of the dopamine system and related networks can offer insight into this complex symptomatology and may lead to more targeted treatments.

Individual differences in the engagement of habitual control over alcohol seeking predict the development of compulsive alcohol seeking and drinking

Excessive drinking is an important behavioural characteristic of alcohol addiction, but not the only one. Individuals addicted to alcohol crave alcoholic beverages, spend time seeking alcohol despite negative consequences and eventually drink to intoxication. With prolonged use, control over alcohol seeking devolves to anterior dorsolateral striatum, dopamine-dependent mechanisms implicated in habit learning and individuals in whom alcohol seeking relies more on these mechanisms are more likely to persist in seeking alcohol despite the risk of punishment. Here, we tested the hypothesis that the development of habitual alcohol seeking predicts the development of compulsive seeking and that, once developed, it is associated with compulsive alcohol drinking. Male alcohol-preferring rats were pre-exposed intermittently to a two-bottle choice procedure and trained on a seeking-taking chained schedule of alcohol reinforcement until some individuals developed punishment-resistant seeking behaviour. The associative basis of their seeking responses was probed with an outcome-devaluation procedure, early or late in training. After seeking behaviour was well established, subjects that had developed greater resistance to outcome devaluation (were more habitual) were more likely to show punishment-resistant (compulsive) alcohol seeking. These individuals also drank more alcohol, despite quinine adulteration, even though having similar alcohol preference and intake before and during instrumental training. They were also less sensitive to changes in the contingency between seeking responses and alcohol outcome, providing further evidence of recruitment of the habit system. We therefore provide direct behavioural evidence that compulsive alcohol seeking emerges alongside compulsive drinking in individuals who have preferentially engaged the habit system.

Role of Levo-tetrahydropalmatine and its metabolites for management of chronic pain and opioid use disorders

BACKGROUND

Opioids have been prescribed to reduce suffering from pain and to enhance quality of life. Due to the addictive potential and the lack of other effective alternatives to treat severe acute and chronic pains, opioids remain a serious public health issue. While, opioids directly influence the drug-seeking behavior, tolerance and withdrawal processes, through neuroadaptation, the brain's endogenous opioid system also adapts in the presence of chronic pain and could contribute to the difficulty of treatment. Despite the seemingly obvious interaction between the presence of pain and opioid-abuse, little is known about the underlying mechanisms in the brain.

PURPOSE

To review the current understanding of the interaction mechanisms of neurotransmitter circuitries in pain modulation and reward in the brain and the effects of L-tetrahydropalmatine (L-THP) and its metabolites in pain management and opioid use disorder and gain a better insight on the pharmacological profile and in vivo effects of L-THP and its metabolites.

METHOD

A detailed literature search on available (preclinical and clinical) studies about the effects of L-THP and its metabolites against drug addiction and chronic pain has been performed. The data was collected using various search engines such as PubMed, ScienceDirect, Google scholar and articles in English up to December 2020 were included in this review.

RESULTS

L-THP and its metabolites demonstrated analgesic and anti-addiction effects. Due to their dual pharmacological properties (D1 partial agonist and D2 antagonist) these compounds could be used as molecular tools to provide a better understanding of the interactions between pain and addiction.

CONCLUSION

The available data confirms the potential of L-THP and its metabolites to treat both chronic pain and drug addiction. However, further clinical trials are needed to establish safety and efficacy.

Problematic eating as an issue of habitual control

Obesity has reached alarming rates worldwide. Although many people attempt to control weight by modifying their food-related behaviours, this typically only has short-term effects and most dieters regain the weight that was lost. Why do so many people struggle to regulate their food-related behaviours? One possible explanation is that these behaviours have become habits that are not immediately sensitive to their consequences. Here we review experimental evidence for a shift to habitual control over food-related behaviours and the neural systems that control them and how this relates to difficulty changing ones' eating behavior.

Neurobiological correlates of cue-reactivity in alcohol-use disorders: A voxel-wise meta-analysis of fMRI studies

Altered brain responses to alcohol-associated stimuli are a neural hallmark of alcohol-use disorder (AUD) and a promising target for pharmacotherapy. However, findings in cue-reactivity based functional MRI (fMRI) studies are inconclusive. To investigate the neural substrates of cue-reactivity and their relevance to treatment outcomes, alcohol craving and relapse in AUD patients, we performed five meta-analyses using signed differential mapping software. Our meta-analysis revealed that alcohol cues evoke greater cue-reactivity than neutral cues in the mesocorticolimbic circuit and lower reactivity in the parietal and temporal regions in AUD patients. Compared to controls, AUD individuals displayed hyperactivations in the medial prefrontal cortex and anterior/middle part of the cingulate cortex. After receiving AUD treatment, AUD patients exhibited greater activations in the precentral gyrus but reduced activations in the bilateral caudate nucleus, insula, right DLPFC, and left superior frontal gyrus. No significant results were found in cue-reactivity correlates of alcohol craving and relapse. Our results implicate cue-induced abnormalities in corticostriatal-limbic circuits may underline the pathophysiology of AUD, and have translational value for treatment development.

Depression in the Direct Pathway of the Dorsomedial Striatum Permits the Formation of Habitual Action

In order to achieve optimal outcomes in an ever-changing environment, humans and animals generally manage their action control via either goal-directed action or habitual action. These two action strategies are thought to be encoded in distinct parallel circuits in the dorsal striatum, specifically, the posterior dorsomedial striatum (DMS) and the dorsolateral striatum (DLS), respectively. The striatum is primarily composed of two subtypes of medium spiny neurons (MSNs): the direct-pathway striatonigral and the indirect-pathway striatopallidal MSNs. MSN-subtype-specific synaptic plasticity in the DMS and the DLS has been revealed to underlie goal-directed action and habitual action, respectively. However, whether any MSN-subtype-specific synaptic plasticity in the DMS is associated with habitual action, and if so, whether the synaptic plasticity affects the formation of habitual action, are not known. This study demonstrates that postsynaptic depression in the excitatory synapses of the direct-pathway striatonigral MSNs in the DMS is formed after habit learning. Moreover, chemogenetically rescuing this depression compromises the acquisition, but not the expression, of habitual action. These findings reveal that an MSN-subtype-specific synaptic plasticity in the DMS affects habitual action and suggest that plasticity in the DMS as well as in the DLS contributes to the formation of habitual action.

Reduced Glutamate Turnover in the Putamen Is Linked With Automatic Habits in Human Cocaine Addiction

BACKGROUND

The balance between goal-directed behavior and habits has been hypothesized to be biased toward the latter in individuals with cocaine use disorder (CUD), suggesting possible neurochemical changes in the putamen, which may contribute to their compulsive behavior.

METHODS

We assessed habitual behavior in 48 patients with CUD and 42 healthy control participants using a contingency degradation paradigm and the Creature of Habit Scale. In a subgroup of this sample (CUD: n = 21; control participants: n = 22), we also measured glutamate and glutamine concentrations in the left putamen using ultra-high-field (7T) magnetic resonance spectroscopy. We hypothesized that increased habitual tendencies in patients with CUD would be associated with abnormal glutamatergic metabolites in the putamen.

RESULTS

Compared with their non-drug-using peers, patients with CUD exhibited greater habitual tendencies during contingency degradation, which correlated with increased levels of self-reported daily habits. We further identified a significant reduction in glutamate concentration and glutamate turnover (glutamate-to-glutamine ratio) in the putamen in patients with CUD, which was significantly related to the level of self-reported daily habits.

CONCLUSIONS

Patients with CUD exhibit enhanced habitual behavior, as assessed both by questionnaire and by a laboratory paradigm of contingency degradation. This automatic habitual tendency is related to a reduced glutamate turnover in the putamen, suggesting a dysregulation of habits caused by chronic cocaine use.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Neural Substrates and Circuits of Drug Addiction

Drug addiction is a chronic relapsing disorder, and a significant amount of research has been devoted to understand the factors that contribute to the development, loss of control, and persistence of compulsive addictive behaviors. In this review, we provide an overview of various theories of addiction to drugs of abuse and the neurobiology involved in elements of the addiction cycle. Specific focus is devoted to the role of the mesolimbic pathway in acute drug reinforcement and occasional drug use, the role of the mesocortical pathway and associated areas (e.g., the dorsal striatum) in escalation/dependence, and the contribution of these pathways and associated circuits to conditioned responses, drug craving, and loss of behavioral control that may underlie drug relapse. By enhancing the understanding of the neurobiological factors that mediate drug addiction, continued preclinical and clinical research will aid in the development of novel therapeutic interventions that can serve as effective long-term treatment strategies for drug-dependent individuals.

Egr2 induction in spiny projection neurons of the ventrolateral striatum contributes to cocaine place preference in mice

Drug addiction develops due to brain-wide plasticity within neuronal ensembles, mediated by dynamic gene expression. Though the most common approach to identify such ensembles relies on immediate early gene expression, little is known of how the activity of these genes is linked to modified behavior observed following repeated drug exposure. To address this gap, we present a broad-to-specific approach, beginning with a comprehensive investigation of brain-wide cocaine-driven gene expression, through the description of dynamic spatial patterns of gene induction in subregions of the striatum, and finally address functionality of region-specific gene induction in the development of cocaine preference. Our findings reveal differential cell-type specific dynamic transcriptional recruitment patterns within two subdomains of the dorsal striatum following repeated cocaine exposure. Furthermore, we demonstrate that induction of the IEG Egr2 in the ventrolateral striatum, as well as the cells within which it is expressed, are required for the development of cocaine seeking.

The human brain is ever changing, constantly rewiring itself in response to new experiences, knowledge or information from the environment. Addictive drugs such as cocaine can hijack the genetic mechanisms responsible for this plasticity, creating dangerous, obsessive drug-seeking and consuming behaviors.

Cocaine-induced plasticity is difficult to apprehend, however, as brain regions or even cell populations can react differently to the compound. For instance, sub-regions in the striatum – the brain area that responds to rewards and helps to plan movement – show distinct responses during progressive exposure to cocaine. And while researchers know that the drug immediately changes how neurons switch certain genes on and off, it is still unclear how these genetic modifications later affect behavior.

Mukherjee, Gonzales et al. explored these questions at different scales, first focusing on how progressive cocaine exposure changed the way various gene programs were activated across the entire brain. This revealed that programs in the striatum were the most affected by the drug.

Examining this region more closely showed that cocaine switches on genes in specific ‘spiny projection’ neuron populations, depending on where these cells are located and the drug history of the mouse. Finally, Mukherjee, Gonzales et al. used genetically modified mice to piece together cocaine exposure, genetic changes and modifications in behavior. These experiments revealed that the drive to seek cocaine depended on activation of the Egr2 gene in populations of spiny projection neurons in a specific sub-region of the striatum. The gene, which codes for a protein that regulates how genes are switched on and off, was itself strongly activated by cocaine intake.

Cocaine addiction can have devastating consequences for individuals. Grasping how this drug alters the brain could pave the way for new treatments, while also providing information on the basic mechanisms underlying brain plasticity.

Endocannabinoid and dopaminergic system: the pas de deux underlying human motivation and behaviors

Endocannabinoid system (ECS) has been identified ever since cannabinoid, an active substance of Cannabis, was known to interact with endogenous cannabinoid (endocannabinoid/eCB) receptors. It later turned out that eCB was more intricate than previously thought. It has a pervasive role and exerts a multitude of cellular signaling mechanisms, regulating various physiological neurotransmission pathways in the human brain, including the dopaminergic (DA) system. eCB roles toward DA system were robust, clearly delineated, and reproducible with respect to physiological as well as pathological neurochemical and neurobehavioral manifestations of DA system, particularly those involving the nigrostriatal and mesocorticolimbic pathways. The eCB–DA system regulates the basics in the Maslow’s pyramid of hierarchy of needs required for individual survival such as food and sexual activity for reproductive purpose to those of higher needs in the pyramid, including self-actualization behaviors leading to achievement and reward (e.g., academic- and/or work-related performance and achievements). It is, thus, interesting to specifically discuss the eCB–DA system, not only on the molecular level, but also its tremendous potential to be developed as a future therapeutic strategy for various neuropsychiatric problems, including obesity, drug addiction and withdrawal, pathological hypersexuality, or low motivation behaviors.

Habit, choice, and addiction

Addiction was suggested to emerge from the progressive dominance of habits over goal-directed behaviors. However, it is generally assumed that habits do not persist in choice settings. Therefore, it is unclear how drug habits may persist in real-world scenarios where this factor predominates. Here, we discuss the poor translational validity of the habit construct, which impedes our ability to determine its role in addiction. New evidence of habitual behavior in a drug choice setting are then described and discussed. Interestingly, habitual preference did not promote drug choice but instead favored abstinence. Here, we propose several clues to reconcile these unexpected results with the habit theory of addiction, and we highlight the need in experimental research to face the complexity of drug addicts' decision-making environments by investigating drug habits in the context of choice and in the presence of cues. On a theoretical level, we need to consider more complex frameworks, taking into account continuous interactions between goal-directed and habitual systems, and alternative decision-making models more representative of real-world conditions.

Rats exposed to intermittent ethanol during late adolescence exhibit enhanced habitual behavior following reward devaluation

The brain undergoes substantial maturation during adolescence, and repeated exposure to ethanol at this time has been shown to result in long-lasting behavioral and neural consequences. During the broad period of adolescence, different neuronal populations and circuits are refined between early and late adolescence, suggesting the possibility that ethanol exposure at these differing times may lead to differential outcomes. The goal of the current study was to evaluate the impact of adolescent intermittent ethanol (AIE) during early and late adolescence on the formation of goal-directed and habitual behavior in adulthood. Male and female Sprague-Dawley rats were exposed to ethanol via intragastric gavage (4.0 g/kg, 25% v/v) every other day from postnatal day (P) 25-45 or P45-65, considered early and late adolescence, respectively. In adulthood (~P70 early or ~ P90 late), rats were gradually food-restricted and began operant training on a fixed ratio 1 schedule. Rats were then transitioned onto random interval schedules and eventually underwent a sensory-specific satiation procedure as a model of reward devaluation. Few differences as a result of adolescent ethanol exposure were found during instrumental training. Following reward devaluation, rats exposed to water and ethanol during early adolescence exhibited reductions in lever pressing, suggestive of a goal-directed response pattern. In contrast, late AIE males and females demonstrated persistent responding following both devalued and non-devalued trials, findings representative of a habitual behavior pattern. The shifts from goal-directed to habitual behavior noted only following late AIE contribute to the growing literature identifying specific behavioral consequences as a result of ethanol exposure during distinct developmental periods within adolescence. More work is needed to determine whether the greater habit formation following late AIE is also associated with elevated habitual ethanol consumption.

Dorsolateral striatum dopamine-dependent cocaine seeking is resistant to pavlovian cue extinction in male and female rats

Cue exposure therapy (CET) reduces craving induced by drug-associated cues in individuals with substance use disorders. A preclinical model of CET, cue extinction, similarly reduces cue-induced cocaine seeking in rodent self-administration models; however, those models may not capture the habitual or compulsive aspects of drug use. Thus, the effectiveness of cue extinction was tested in male and female rats trained to self-administer cocaine using second-order (SO) or fixed-ratio (FR) schedules of reinforcement to facilitate dorsolateral striatum (DLS) dopamine-dependent or -independent response strategies, respectively. Cue extinction significantly reduced drug seeking in FR-trained rats, replicating prior results, but was ineffective in SO-trained rats. SO-trained rats also showed increased indices of glutamate signaling in the DLS relative to FR-trained rats, despite comparable levels of cocaine intake. Furthermore, in SO-trained rats, antagonism of AMPA receptors in the DLS rescued the efficacy of cue extinction to reduce drug seeking. Finally, the effectiveness of cue extinction was also revealed in SO-trained rats when they were taught to perform a new, non-habitual response for cocaine cue presentation. Overall, our findings indicate that habit-like drug seeking leads to plasticity in the DLS and behavior that is resistant to cue extinction, but that the effects of cue extinction are restored when DLS glutamatergic signaling is blocked. These results have implications for the effectiveness of clinical cue exposure therapy.

The Cerebellum on Cocaine

The traditional cerebellum’s role has been linked to the high computational demands for sensorimotor control. However, several findings have pointed to its involvement in executive and emotional functions in the last decades. First in 2009 and then, in 2016, we raised why we should consider the cerebellum when thinking about drug addiction. A decade later, mounting evidence strongly suggests the cerebellar involvement in this disorder. Nevertheless, direct evidence is still partial and related mainly to drug-induced reward memory, but recent results about cerebellar functions may provide new insights into its role in addiction. The present review does not intend to be a compelling revision on available findings, as we did in the two previous reviews. This minireview focuses on specific findings of the cerebellum’s role in drug-related reward memories and the way ahead for future research. The results discussed here provide grounds for involving the cerebellar cortex’s apical region in regulating behavior driven by drug-cue associations. They also suggest that the cerebellar cortex dysfunction may facilitate drug-induced learning by increasing glutamatergic output from the deep cerebellar nucleus (DCN) to the ventral tegmental area (VTA) and neural activity in its projecting areas.

Improving translation of animal models of addiction and relapse by reverse translation

Critical features of human addiction are increasingly being incorporated into complementary animal models, including escalation of drug intake, punished drug seeking and taking, intermittent drug access, choice between drug and non-drug rewards, and assessment of individual differences based on criteria in the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). Combined with new technologies, these models advanced our understanding of brain mechanisms of drug self-administration and relapse, but these mechanistic gains have not led to improvements in addiction treatment. This problem is not unique to addiction neuroscience, but it is an increasing source of disappointment and calls to regroup. Here we first summarize behavioural and neurobiological results from the animal models mentioned above. We then propose a reverse translational approach, whose goal is to develop models that mimic successful treatments: opioid agonist maintenance, contingency management and the community-reinforcement approach. These reverse-translated 'treatments' may provide an ecologically relevant platform from which to discover new circuits, test new medications and improve translation.

Molecular and circuit mechanisms regulating cocaine memory

Risk of relapse is a major challenge in the treatment of substance use disorders. Several types of learning and memory mechanisms are involved in substance use and have implications for relapse. Associative memories form between the effects of drugs and the surrounding environmental stimuli, and exposure to these stimuli during abstinence causes stress and triggers drug craving, which can lead to relapse. Understanding the neural underpinnings of how these associations are formed and maintained will inform future advances in treatment practices. A large body of research has expanded our knowledge of how associative memories are acquired and consolidated, how they are updated through reactivation and reconsolidation, and how competing extinction memories are formed. This review will focus on the vast literature examining the mechanisms of cocaine Pavlovian associative memories with an emphasis on the molecular memory mechanisms and circuits involved in the consolidation, reconsolidation, and extinction of these memories. Additional research elucidating the specific signaling pathways, mechanisms of synaptic plasticity, and epigenetic regulation of gene expression in the circuits involved in associative learning will reveal more distinctions between consolidation, reconsolidation, and extinction learning that can be applied to the treatment of substance use disorders.