ABA and ABC renewal of conditioned magazine approach are not impaired by dorsal hippocampus inactivation or lesions

Enhancing translation: A need to leverage complex preclinical models of addictive drugs to accelerate substance use treatment options

Preclinical models of addictive drugs have been developed for decades to model aspects of the clinical experience in substance use disorders (SUDs). These include passive exposure as well as volitional intake models across addictive drugs and have been utilized to also measure withdrawal symptomatology and potential neurobehavioral mechanisms underlying relapse to drug seeking or taking. There are a number of Food and Drug Administration (FDA)-approved medications for SUDs, however, many demonstrate low clinical efficacy as well as potential sex differences, and we also note gaps in the continuum of care for certain aspects of clinical experiences in individuals who use drugs. In this review, we provide a comprehensive update on both frequently utilized and novel behavioral models of addiction with a focus on translational value to the clinical experience and highlight the need for preclinical research to follow epidemiological trends in drug use patterns to stay abreast of clinical treatment needs. We then note areas in which models could be improved to enhance the medications development pipeline through efforts to enhance translation of preclinical models. Next, we describe neuroscience efforts that can be leveraged to identify novel biological mechanisms to enhance medications development efforts for SUDs, focusing specifically on advances in brain transcriptomics approaches that can provide comprehensive screening and identification of novel targets. Together, the confluence of this review demonstrates the need for careful selection of behavioral models and methodological parameters that better approximate the clinical experience combined with cutting edge neuroscience techniques to advance the medications development pipeline for SUDs.

Optogenetic stimulation of infralimbic cortex projections to the paraventricular thalamus attenuates context-induced renewal

Contexts associated with prior reinforcement can renew extinguished conditioned responding. The prelimbic (PL) and infralimbic (IL) cortices are thought to mediate the expression and suppression of conditioned responding, respectively. Evidence suggests that PL inputs to the paraventricular nucleus of the thalamus (PVT) drive the expression of cue-induced reinstatement of drug seeking and that IL inputs to the PVT mediate fear extinction retrieval. However, the role of these projections in renewal of appetitive Pavlovian conditioned responding is unknown. We trained male and female Long-Evans rats to associate a conditioned stimulus (CS; 10 s white noise) with delivery of a 10% sucrose unconditioned stimulus (US; .2 ml/CS) to a fluid port in a distinct context (Context A). We then extinguished responding by presenting the CS without the US in a different context (Context B). At test, rats were returned to Context A, and optogenetic stimulation was delivered to either the IL-to-PVT or PL-to-PVT pathway during CS presentations. Optically stimulating the IL-to-PVT, but not the PL-to-PVT pathway, attenuated ABA renewal of CS port entries, and this effect was similar in males and females. Further, rats self-administered optical stimulation of the IL-to-PVT but not the PL-to-PVT pathway suggesting that activation of the IL-to-PVT pathway is reinforcing. The effectiveness of optical stimulation parameters to activate neurons in the IL, PL and PVT was confirmed using Fos immunohistochemistry. These findings provide evidence for novel neural mechanisms in renewal of responding to a sucrose-predictive CS, as well as more generally in contextual processing and appetitive associative learning.

The dorsal hippocampus' role in context-based timing in rodents

To act proactively, we must predict when future events will occur. Individuals generate temporal predictions using cues that indicate an event will happen after a certain duration elapses. Neural models of timing focus on how the brain represents these cue-duration associations. However, these models often overlook the fact that situational factors frequently modulate temporal expectations. For example, in realistic environments, the intervals associated with different cues will often covary due to a common underlying cause. According to the 'common cause hypothesis,' observers anticipate this covariance such that, when one cue's interval changes, temporal expectations for other cues shift in the same direction. Furthermore, as conditions will often differ across environments, the same cue can mean different things in different contexts. Therefore, updates to temporal expectations should be context-specific. Behavioral work supports these predictions, yet their underlying neural mechanisms are unclear. Here, we asked whether the dorsal hippocampus mediates context-based timing, given its broad role in context-conditioning. Specifically, we trained rats with either hippocampal or sham lesions that two cues predicted reward after either a short or long duration elapsed (e.g., tone-8 s/light-16 s). Then, we moved rats to a new context and extended the long cue's interval (e.g., light-32 s). This caused rats to respond later to the short cue, despite never being trained to do so. Importantly, when returned to the initial training context, sham rats shifted back toward both cues' original intervals. In contrast, lesion rats continued to respond at the long cue's newer interval. Surprisingly, they still showed contextual modulation for the short cue, responding earlier like shams. These data suggest the hippocampus only mediates context-based timing if a cue is explicitly paired and/or rewarded across distinct contexts. Furthermore, as lesions did not impact timing measures at baseline or acquisition for the long cue's new interval, our data suggests that the hippocampus only modulates timing when context is relevant.

Conditional Control of Instrumental Avoidance by Context Following Extinction

Using rodents, three training arrangements (i.e., ABB vs. ABA, AAA vs. AAB and ABB vs. ABC) explored whether extinction influences the expression of avoidance in a manner controlled by context. Retention testing following extinction showed that more avoidance responding (i.e., renewal) was observed when extinguished cues were tested outside of the context where they had undergone extinction. In contrast, response rates were significantly lower when stimuli were tested within the context where extinction learning had occurred. These findings add to the emerging literature assessing the role of Pavlovian extinction processes in the development of instrumental avoidance responding by demonstrating conditional control over extinguished responding by context. This study was conducted using a within-subjects approach that minimized the potential for context-outcome associations to bias responding, and thus, reflects hierarchical control over behavior based on the specific associative status of each tested cue in each training context.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Reduced renewal of conditioned suppression following lesions of the dorsal hippocampus in male rats

Extinguished responding will renew when the conditioned stimulus occurs outside the extinction context. Although studies of conditioned freezing have consistently demonstrated a role for the hippocampus in renewal, several studies have demonstrated intact renewal of conditioned suppression despite damage to the hippocampus (Frohardt, Guarraci, & Bouton, 2000; Todd, Jiang, DeAngeli, & Bucci, 2017; Wilson, Brooks, & Bouton, 1995). Because these prior studies have examined renewal when testing occurred in the original conditioning context ("Context A"), the present conditioned suppression experiments examined the role of the hippocampus when testing occurred in a context not associated with prior conditioning ("Context C"). In Experiments 1 and 2, conditioning occurred in Context A, and extinction in Context B. Renewal of conditioned suppression was observed when the extinguished conditioned stimulus (CS) was tested in Context C. However, renewal was attenuated in rats with lesions of the dorsal hippocampus (DH). Summation testing failed to detect conditioned inhibition in the extinction context, suggesting instead that the context acquired negative occasion-setting properties. Attenuated renewal was not due to an inability of DH lesioned rats to discriminate contexts (Experiment 3). These experiments thus demonstrate a role for the DH in renewal of conditioned suppression when testing occurs in a neutral context. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Renewal of conditioned tentacle lowering by circadian contextual cues in snails Cornu aspersum

Previous experiments using tentacle lowering conditioning in terrestrial snails Cornu aspersum have shown extinction and recovery of the conditioned response (CR) as a consequence of both inserting a delay between the extinction and test (spontaneous recovery) and of re-exposing the animal to the unconditioned stimulus after extinction (reinstatement). Two experiments that examined recovery of the CR due to a change in context (renewal effect) were carried out to continue this line of research. In Experiment 1, subjects received conditioning with an odour (CS) followed by extinction in the presence of another odour (CS + C), before being exposed to the original one (CS). In Experiment 2, conditioning and extinction of an odour CS took place in the presence of different circadian contextual cues (hour of the day and presence of light). The results showed that a return to the original context of conditioned training, after the extinction in a different context, either defined by an odour (Experiment 1) or by circadian cues (Experiment 2), produce a recovery of the CR compared to suitable control groups. These results can be interpreted as an instance of ABA renewal effect and they provide information about psychological mechanisms involved in extinction processes.

Comparing ABA, AAB, and ABC Renewal of Appetitive Pavlovian Conditioned Responding in Alcohol- and Sucrose-Trained Male Rats

Conditioned responding can be renewed by re-exposure to the conditioning context following extinction in a different context (ABA renewal) or by removal from the extinction context (AAB or ABC renewal). ABA renewal is robust in Pavlovian and operant conditioning paradigms. However, fewer studies have investigated AAB and ABC renewal of appetitive conditioning, and those that did predominantly used operant conditioning tasks. Renewal has theoretical relevance for extinction and for exposure-based treatments for substance use disorders that aim to extinguish reactivity to drug-predictive cues. We therefore investigated ABA, AAB, and ABC renewal of Pavlovian conditioned responding to cues that predicted either alcohol or sucrose. Male, Long-Evans rats (Charles River) were exposed to either 15% ethanol (Study 1: “alcohol”) or 10% sucrose (Study 2: “sucrose”) in their home cages. Next, they were trained to discriminate between two auditory stimuli (white noise and clicker; 10 s) in conditioning chambers equipped with distinct olfactory, visual, and tactile contextual stimuli (context A). One conditioned stimulus (CS+) was paired with fluid delivery (0.2 ml/CS+; 3.2 ml/session; alcohol or sucrose in separate experiments), and the second CS (CS−) was not. In all sessions (conditioning, extinction, and test), each CS was presented 16 times/session on a variable-time 67-s schedule, and entries into the fluid port were recorded. CS+ port entries were then extinguished by withholding fluid delivery either in context A or in a second, different context (context B). Next, we assessed ABA, AAB, and ABC renewal in the absence of fluid delivery. During extinction, CS+ port entries were initially elevated in context A relative to context B. ABA renewal of CS+ port entries occurred in both alcohol- and sucrose-trained rats. ABC renewal approached statistical significance when data from both experiments were combined. No AAB renewal was observed, and, in fact, alcohol-trained rats showed AAB suppression. These results corroborate the reliability of ABA renewal and suggest that ABC renewal is a modest effect that may require greater statistical power to detect. From a treatment perspective, the lack of AAB renewal suggests that exposure-based treatments for substance use disorders might benefit from implementation in real-world, drug-use contexts.

Chemogenetic Inhibition Reveals That Processing Relative But Not Absolute Threat Requires Basal Amygdala

While our understanding of appetitive motivation has benefited immensely from the use of selective outcome devaluation tools, the same cannot be said about aversive motivation. Findings from appetitive conditioning studies have shown that basal amygdala is required for behaviors that are sensitive to updates in outcome value, but similar results in aversive motivation are difficult to interpret due to a lack of outcome specificity. The studies reported here sought to develop procedures to isolate sensory-specific processes in aversive learning and behavior and to assess the possible contribution of the basal amygdala. Post-training changes to outcome value produced commensurate changes to subsequently tested conditioned responding in male rodents. Specifically, increases in shock intensity (i.e., inflation) augmented, while repeated exposure to (i.e., habituation of) an aversive sound (klaxon-horn) reduced freezing to conditioned stimuli previously paired with these outcomes. This was extended to a discriminative procedure, in which following revaluation of one event, but not the other, responding was found to be dependent on outcome value signaled by each cue. Chemogenetic inactivation of basal amygdala impaired this discrimination between stimuli signaling differently valued outcomes, but did not affect the revaluation process itself. These findings demonstrate a contribution of the basal amygdala to aversive outcome-dependent motivational processes.SIGNIFICANCE STATEMENT The specific content of pavlovian associative learning has been well studied in appetitive motivation, where the value of different foods can be easily manipulated. This has facilitated our understanding of the neural circuits that generate different forms of motivation (i.e., sensory specific vs general). Studies of aversive learning have not produced the same degree of understanding with regard to sensory specificity due to a lack of tools for evaluating sensory-specific processes. Here we use a variant of outcome devaluation procedures with aversive stimuli to study the role of basal amygdala in discriminating between aversive stimuli conveying different degrees of threat. These findings have implications for how we study generalized threat to identify dysregulation that can contribute to generalized anxiety.

Occasion setting

Occasion setting refers to the ability of 1 stimulus, an occasion setter, to modulate the efficacy of the association between another, conditioned stimulus (CS) and an unconditioned stimulus (US) or reinforcer. Occasion setters and simple CSs are readily distinguished. For example, occasion setters are relatively immune to extinction and counterconditioning, and their combination and transfer functions differ substantially from those of simple CSs. Similarly, the acquisition of occasion setting is favored when stimuli are separated by longer intervals, by empty trace intervals, and are of different modalities, whereas the opposite conditions typically favor the acquisition of simple associations. Furthermore, the simple conditioning and occasion setting properties of a single stimulus can be independent, for example, that stimulus may simultaneously predict the occurrence of a reinforcer and indicate that another stimulus will not be reinforced. Many behavioral phenomena that are intractable to simple associative analysis are better understood within an occasion setting framework. Besides capturing the distinction between direct and modulatory control common to many arenas in neuroscience, occasion setting provides a model for the hierarchical organization of memory for events and event relations, and for contextual control more broadly. Although early lesion studies further differentiated between occasion setting and simple conditioning functions, little is known about the neurobiology of occasion setting. Modern techniques for precise manipulation and monitoring of neuronal activity in multiple brain regions are ideally suited for disentangling contributions of simple conditioning and occasion setting in associative learning. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Pathway specific activation of ventral hippocampal cells projecting to the prelimbic cortex diminishes fear renewal

The ability to learn that a stimulus no longer signals danger is known as extinction. A major characteristic of extinction is that it is context-dependent, which means that fear reduction only occurs in the same context as extinction training. In other contexts, there is re-emergence of fear, known as contextual renewal. The ability to properly extinguish fear memories and generalize safety associations to multiple contexts provides therapeutic potential, but little is known about the specific neural pathways that mediate fear renewal and extinction generalization. The ventral hippocampus (VH) is thought to provide a contextual gating mechanism that determines whether fear or safety is expressed in particular contexts through its projections to areas of the fear circuit, including the infralimbic (IL) and prelimbic (PL) cortices. Moreover, VH principal cells fire in large, overlapping regions of the environment, a characteristic that is ideal to support generalization; yet it is unclear how different projection cells mediate this process. Using a pathway-specific (intersectional) chemogenetic approach, we demonstrate that selective activation of VH cells projecting to PL attenuates fear renewal without affecting fear expression. These results have implications for anxiety disorders since they uncover a neural pathway associated with extinction generalization.

Extinction of specific stimulus-outcome (S-O) associations in Pavlovian learning with an extended CS procedure

Three experiments with male and female rats were conducted to examine the effects of Pavlovian extinction training on Pavlovian-to-instrumental transfer (PIT) in a task in which the unconditioned stimulus (US) was presented at an early time point within an extended conditioned stimulus (CS). Two instrumental responses were trained with different reinforcing outcomes (R1-O1, R2-O2) and then, independently, 2 stimuli were trained with those outcomes (S1-O1, S2-O2). One group then underwent an extinction treatment (S1-, S2-) and a second was merely exposed to the experimental contexts without any stimulus events. Finally, the effects of the 2 stimuli on instrumental responding were assessed in PIT tests. Across experiments we varied the number of Pavlovian training trials prior to extinction (8, 16, or 64 trials) and the length of time following extinction prior to test (i.e., 1 or 21 days, in a test for spontaneous recovery). We observed that outcome-specific PIT was reduced by extinction in all of our training conditions and that this extinction effect was durable, surviving a 3-week spontaneous recovery interval even though conditioned magazine approach spontaneously recovered over this interval. Although extinction reduced the magnitude of PIT, the temporal expression of PIT was mostly unaffected. We found these effects in both male and female rats, though in 1 study females were extinction-resistant. These data suggest that under the conditions studied here Pavlovian extinction may permanently weaken the ability of Pavlovian cues to retrieve a representation of their associated outcomes without impacting the temporal organization of responding. This suggests that different features of learning may be differentially sensitive to extinction. (PsycINFO Database Record

Intact renewal after extinction of conditioned suppression with lesions of either the retrosplenial cortex or dorsal hippocampus

Extinction of fear to a Pavlovian conditioned stimulus (CS) is known to be context-specific. When the CS is tested outside the context of extinction, fear returns, or renews. Several studies have demonstrated that renewal depends upon the hippocampus, although there are also studies where renewal was not impacted by hippocampal damage, suggesting that under some conditions context encoding and/or retrieval of extinction depends upon other regions. One candidate region is the retrosplenial cortex (RSC), which is known to contribute to contextual and spatial learning and memory. Using a conditioned-suppression paradigm, Experiment 1 tested the impact of pre-training RSC lesions on renewal of extinguished fear. Consistent with previous studies, lesions of the RSC did not impact acquisition or extinction of conditioned fear to the CS. Further, there was no evidence that RSC lesions impaired renewal, indicating that contextual encoding and/or retrieval of extinction does not depend upon the RSC. In Experiment 2, post-extinction lesions of either the RSC or dorsal hippocampus (DH) also had no impact on renewal. However, in Experiment 3, both RSC and DH lesions did impair performance in an object-in-place procedure, an index of place memory. RSC and DH contributions to extinction and renewal are discussed.

Renewal of conditioned responding to food cues in rats: Sex differences and relevance of estradiol

Cues associated with food can stimulate food anticipation, procurement, and consumption, independently of hunger. These and other behaviors driven by learned cues are persistent and can reappear after extinction, because the original learned associations continue to exist. Renewal, or reinstatement, of extinguished conditioned behavior may explain the inability to change maladaptive eating habits driven by food cues, similar to the mechanisms of drug use relapse. Here, we investigated sex differences in context-induced renewal of responding to food cues, and the role of estradiol in females in a Pavlovian conditioning preparation. We compared adult male and female rats because there is evidence for sex differences in learning and memory and in the control of feeding. Context-induced renewal involves conditioning and extinction in different contexts and the renewal of conditioned behavior is induced by return to the conditioning context ("ABA renewal"; experimental groups). Control groups remain in the same context during conditioning, extinction, and test. In Experiment 1, male and female rats were trained to associate a tone with food pellets during acquisition, and after extinction with tone only presentations, were tested for renewal of responding to the tone. Learning was assessed through the expression of the conditioned response, which included approach and activity directed at food receptacle (food cup behavior). Males and females learned the acquisition and extinction of tone-food associations similarly, but there were sex differences during renewal of the conditioned responses to the food cue. Males showed robust renewal of responding, while renewal in intact females was inconsistent. Males in the experimental group had significantly higher food cup behavior compared to males in the control group, while females in both groups showed similar levels of food cup behavior during the tone. In Experiment 2, we examined a potential role of estradiol in renewal, by comparing intact females with ovariectomized females with, and without, estradiol replacement. Rats in all groups acquired and extinguished tone-food associations similarly. During the test for renewal, the ovariectomized rats with estradiol replacement in the experimental group showed renewal of responding, evidenced by significantly higher food cup behavior compared to the control group. Intact and ovariectomized rats in the experimental groups had similar rates of food cup behavior as their corresponding control groups. These results provide novel evidence for sex differences and relevance of estradiol in renewal of responding to food cues and more broadly in contextual processing and appetitive associative learning, potentially relevant to maladaptive eating habits and eating disorders.

Dorsal hippocampus inactivation impairs spontaneous recovery of Pavlovian magazine approach responding in rats

Destruction or inactivation of the dorsal hippocampus (DH) has been shown to eliminate the renewal of extinguished fear [1-4]. However, it has recently been reported that the contextual control of responding to extinguished appetitive stimuli is not disrupted when the DH is destroyed or inactivated prior to tests for renewal of Pavlovian conditioned magazine approach [5]. In the present study we extend the analysis of DH control of appetitive extinction learning to the spontaneous recovery of Pavlovian conditioned magazine approach responding. Subjects were trained to associate two separate stimuli with the delivery of food and had muscimol or vehicle infused into the DH prior to a single test-session for spontaneous recovery occurring immediately following extinction of one of these stimuli, but one week following extinction of the other. While vehicle treated subjects showed more recovery to the distally extinguished stimulus than the proximal one, muscimol treated subjects failed to show spontaneous recovery to either stimulus. This result suggests that, while the DH is not involved in the control of extinction by physical contexts [5], it may be involved when time is the gating factor controlling recovery of extinguished responding.

Psychological and neural mechanisms of experimental extinction: a selective review

The present review examines key psychological concepts in the study of experimental extinction and implications these have for an understanding of the underlying neurobiology of extinction learning. We suggest that many of the signature characteristics of extinction learning (spontaneous recovery, renewal, reinstatement, rapid reacquisition) can be accommodated by the standard associative learning theory assumption that extinction results in partial erasure of the original learning together with new inhibitory learning. Moreover, we consider recent behavioral and neural evidence that supports the partial erasure view of extinction, but also note shortcomings in our understanding of extinction circuits as these relate to the negative prediction error concept. Recent work suggests that common prediction error and stimulus-specific prediction error terms both may be required to explain neural plasticity both in acquisition and extinction learning. In addition, we suggest that many issues in the content of extinction learning have not been fully addressed in current research, but that neurobiological approaches should be especially helpful in addressing such issues. These include questions about the nature of extinction learning (excitatory CS-No US, inhibitory CS-US learning, occasion setting processes), especially as this relates to studies of the micro-circuitry of extinction, as well as its representational content (sensory, motivational, response). An additional understudied problem in extinction research is the role played by attention processes and their underlying neural networks, although some research and theory converge on the idea that extinction is accompanied by attention decrements (i.e., habituation-like processes).

Dopamine and extinction: a convergence of theory with fear and reward circuitry

Research on dopamine lies at the intersection of sophisticated theoretical and neurobiological approaches to learning and memory. Dopamine has been shown to be critical for many processes that drive learning and memory, including motivation, prediction error, incentive salience, memory consolidation, and response output. Theories of dopamine's function in these processes have, for the most part, been developed from behavioral approaches that examine learning mechanisms in reward-related tasks. A parallel and growing literature indicates that dopamine is involved in fear conditioning and extinction. These studies are consistent with long-standing ideas about appetitive-aversive interactions in learning theory and they speak to the general nature of cellular and molecular processes that underlie behavior. We review the behavioral and neurobiological literature showing a role for dopamine in fear conditioning and extinction. At a cellular level, we review dopamine signaling and receptor pharmacology, cellular and molecular events that follow dopamine receptor activation, and brain systems in which dopamine functions. At a behavioral level, we describe theories of learning and dopamine function that could describe the fundamental rules underlying how dopamine modulates different aspects of learning and memory processes.

Behavioral and neurobiological mechanisms of extinction in Pavlovian and instrumental learning

This article reviews research on the behavioral and neural mechanisms of extinction as it is represented in both Pavlovian and instrumental learning. In Pavlovian extinction, repeated presentation of a signal without its reinforcer weakens behavior evoked by the signal; in instrumental extinction, repeated occurrence of a voluntary action without its reinforcer weakens the strength of the action. In either case, contemporary research at both the behavioral and neural levels of analysis has been guided by a set of extinction principles that were first generated by research conducted at the behavioral level. The review discusses these principles and illustrates how they have informed the study of both Pavlovian and instrumental extinction. It shows that behavioral and neurobiological research efforts have been tightly linked and that their results are readily integrated. Pavlovian and instrumental extinction are also controlled by compatible behavioral and neural processes. Since many behavioral effects observed in extinction can be multiply determined, we suggest that the current close connection between behavioral-level and neural-level analyses will need to continue.