Mechanisms of resurgence II: Response-contingent reinforcers can reinstate a second extinguished behavior

Resurgence of goal-directed actions and habits

This study investigated how goal-directed and habitual behaviors recover after extinction within the context of the resurgence effect, a form of relapse induced by the removal or worsening of alternative reinforcement. Rats were trained to press a target lever with one reinforcer (O1) for either minimal (4) or extended (16) sessions. An extinction test after the completion of O1 devaluation confirmed that minimal and extended training formed goal-directed and habitual behaviors, respectively. Then, pressing an alternative lever was reinforced with a second reinforcer (O2) while the target response was placed on extinction. When O2 was discontinued, the minimally trained target response resurged with goal-directed status as in the extinction test. However, the extinguished habitual behavior in the extensively trained rats did not recover as a habit but instead with goal-directed status, possibly due to the context specificity of habits or the introduction of a new response-reinforcer contingency. The critical finding that reinforcer devaluation consistently led to less resurgence regardless of the amount of acquisition training provides a clinical implication that coupling differential-reinforcement-of-alternative-behavior (DRA) treatments with the devaluation of the associated reinforcer of problematic behavior could effectively diminish its recurrence.

A Comprehensive Systematic Review of Procedures and Analyses Used in Basic and Preclinical Studies of Resurgence, 1970-2020

Resurgence is the return of a previously reinforced response as conditions worsen for an alternative response, such as the introduction of extinction, reductions in reinforcement, or punishment. As a procedure, resurgence has been used to model behavioral treatments and understand behavioral processes contributing both to relapse of problem behavior and flexibility during problem-solving. Identifying existing procedural and analytic methods arranged in basic/preclinical research could be used by basic and preclinical researchers to develop novel approaches to study resurgence, whereas translational and clinical researchers could identify potential approaches to combating relapse during behavioral interventions. Despite the study of resurgence for over half a century, there have been no systematic reviews of the basic/preclinical research on resurgence. To characterize the procedural and analytic methods used in basic/preclinical research on resurgence, we performed a systematic review consistent with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). We identified 120 articles consisting of 200 experiments that presented novel empirical research, examined operant behavior, and included standard elements of a resurgence procedure. We reported prevalence and trends in over 60 categories, including participant characteristics (e.g., species, sample size, disability), designs (e.g., single subject, group), procedural characteristics (e.g., responses, reinforcer types, control conditions), criteria defining resurgence (e.g., single test, multiple tests, relative to control), and analytic strategies (e.g., inferential statistics, quantitative analysis, visual inspection). We make some recommendations for future basic, preclinical, and clinical research based on our findings of this expanding literature.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40614-022-00361-y.

Context, attention, and the switch between habit and goal-direction in behavior

This article reviews recent findings from the author’s laboratory that may provide new insights into how habits are made and broken. Habits are extensively practiced behaviors that are automatically evoked by antecedent cues and performed without their goal (or reinforcer) “in mind.” Goal-directed actions, in contrast, are instrumental behaviors that are performed because their goal is remembered and valued. New results suggest that actions may transition to habit after extended practice when conditions encourage reduced attention to the behavior. Consistent with theories of attention and learning, a behavior may command less attention (and become habitual) as its reinforcer becomes well-predicted by cues in the environment; habit learning is prevented if presentation of the reinforcer is uncertain. Other results suggest that habits are not permanent, and that goal-direction can be restored by several environmental manipulations, including exposure to unexpected reinforcers or context change. Habits are more context-dependent than goal-directed actions are. Habit learning causes retroactive interference in a way that is reminiscent of extinction: It inhibits, but does not erase, goal-direction in a context-dependent way. The findings have implications for the understanding of habitual and goal-directed control of behavior as well as disordered behaviors like addictions.

Examining effects of training duration on humans' resurgence and variability using a novel touchscreen procedure

Resurgence occurs when a previously reinforced and then extinguished target response increases due to reducing/eliminating an alternative source of reinforcement or punishing an alternative response. We evaluated whether duration of reinforcement history for a target response (1) affects the degree to which resurgence is observed in humans and (2) produces different gradients of response generalization around target responding during extinction testing. We arranged a novel touchscreen interface in which university students could swipe a 3D soccer ball to spin any direction. In Phase 1, the first direction swiped became the target and produced points exchangeable for money for 3 or 1 min across 2 groups. The first swipe was recorded but had no programmed consequence in a third group. In Phase 2, swipes 180-degrees from the target resulted in points for 3 min in all groups. Point deliveries ceased for 2 min to test for resurgence in Phase 3. Target responses resurged during testing to a relatively greater extent with longer Phase-1 training but gradients of response generalization did not differ among groups. These findings extend prior research on the role of training duration on resurgence. We discuss methodological and conceptual issues surrounding the assessment of response generalization in resurgence.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Repeated resurgence with and without a context change

Resurgence is the reoccurrence of a target response when reinforcement for a more recently reinforced alternative response is eliminated or reduced. The present study arranged two successive three-phase procedures to assess whether resurgence decreases with repeated assessments. Moreover, we arranged a contextual change from the first to second assessment for some groups. Phase 1 reinforced a target response on a touchscreen computer with typically developing adults as participants according to either variable-ratio or variable-interval schedules of reinforcement. Phase 2 extinguished target responding and reinforced alternative responding. Phase 3 tested for resurgence by extinguishing alternative responding. Resurgence reliably occurred in all tests and decreased from the first to second exposure to the procedures but there were no effects of context change. Therefore, repeated exposures to resurgence tests reduced those effects but contextual changes had no effect.

Oscillotherapeutics - Time-targeted interventions in epilepsy and beyond

Oscillatory brain activities support many physiological functions from motor control to cognition. Disruptions of the normal oscillatory brain activities are commonly observed in neurological and psychiatric disorders including epilepsy, Parkinson's disease, Alzheimer's disease, schizophrenia, anxiety/trauma-related disorders, major depressive disorders, and drug addiction. Therefore, these disorders can be considered as common oscillation defects despite having distinct behavioral manifestations and genetic causes. Recent technical advances of neuronal activity recording and analysis have allowed us to study the pathological oscillations of each disorder as a possible biomarker of symptoms. Furthermore, recent advances in brain stimulation technologies enable time- and space-targeted interventions of the pathological oscillations of both neurological disorders and psychiatric disorders as possible targets for regulating their symptoms.

Relapse of operant behavior after response elimination with an extinction or an omission contingency

The present study compared relapse after responding was eliminated by extinction or omission training in rats. In Experiment 1, lever pressing was reinforced with food pellets in Context A and then eliminated with either extinction or omission training in Context B. The response was then tested in Contexts A and B in either the presence or absence of free food pellets delivered on a random time schedule. All rats showed higher responding when tested in Context A than Context B, and there was little evidence that omission training attenuated this ABA renewal effect. Noncontingent pellets increased responding after extinction but not after omission. However, when responding on the last day of response elimination was compared to responding during the test in the response-elimination context, there was some evidence that omission-trained rats showed a small increase in responding even when tested with free pellets. Results of Experiment 2 suggest this increase was not due to differences in the temporal distribution of pellets during elimination and the test, and that the result might be due to mere removal of the omission contingency, but any such effect is small and difficult to detect statistically. The results provide new information about factors generating relapse after omission training.

Resurgence in humans: Reducing relapse by increasing generalization between treatment and testing

Resurgence is the increase in performance of an extinguished instrumental (operant) response that accompanies the extinction of a response that has been reinforced to replace it. Resurgence may involve processes that are relevant for understanding relapse in applied and clinical settings. While resurgence is known to be a robust phenomenon in human operant extinction, the processes that control it remain unclear. Here we asked whether human resurgence is controlled by processes that are similar to those that have been identified in animals by asking whether two methods that reduce resurgence in animals also reduce it in humans. Participants first learned to make an operant response (R1) for a tangible food reinforcer (O1). In a second phase (Phase 2), R1 was extinguished while a second response (R2) was introduced and reinforced with a virtual monetary reward (USD $0.10 coins; O2). In a test phase, extinction was then introduced for R2 and resurgence of R1 was assessed. In Experiment 1, resurgence that occurred after the treatment just described was attenuated if there had been periodic exposure to R2 extinction during the treatment phase (Phase 2). In Experiment 2, resurgence was prevented when O2, but not O1, was presented noncontingently during the test. The results are among the first to suggest a mechanism underlying resurgence in humans, namely, renewal caused by contextual change. They also provide initial evidence to suggest that resurgence may be the result of common processes in animals and humans. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Prediction and control of operant behavior: What you see is not all there is

Prediction and control of operant behavior are major goals of behavior analysis. We suggest that achieving these goals can benefit from doing more than identifying the three-term contingency between the behavior, its setting stimulus, and its consequences. Basic research now underscores the idea that prediction and control require consideration of the behavior's history. As one example, if an operant is a goal-directed action, it is controlled by the current value of the reinforcer, as illustrated by the so-called reinforcer devaluation effect. In contrast, if the behavior is a habit, it occurs automatically, without regard to the reinforcer's value, as illustrated by its insensitivity to the reinforcer devaluation effect. History variables that distinguish actions and habits include the extent of their prior practice and their schedule of reinforcement. Other operants can appear to have very low or zero strength. However, if the behavior has reached that level through extinction or punishment, it may precipitously increase in strength by changing the context, allowing time to pass, presenting the reinforcer contingently or noncontingently, or extinguishing an alternative behavior. Behaviors that are not suppressed by extinction or punishment are not affected the same way. When predicting the strength of an operant behavior, what you see is not all there is. The behavior's history counts.

Extinction of instrumental (operant) learning: interference, varieties of context, and mechanisms of contextual control

This article reviews recent research on the extinction of instrumental (or operant) conditioning from the perspective that it is an example of a general retroactive interference process. Previous discussions of interference have focused primarily on findings from Pavlovian conditioning. The present review shows that extinction in instrumental learning has much in common with other examples of retroactive interference in instrumental learning (e.g., omission learning, punishment, second-outcome learning, discrimination reversal learning, and differential reinforcement of alternative behavior). In each, the original learning can be largely retained after conflicting information is learned, and behavior is cued or controlled by the current context. The review also suggests that a variety of stimuli can play the role of context, including room and apparatus cues, temporal cues, drug state, deprivation state, stress state, and recent reinforcers, discrete cues, or behaviors. In instrumental learning situations, the context can control behavior through its direct association with the reinforcer or punisher, through its hierarchical relation with response-outcome associations, or its direct association (inhibitory or excitatory) with the response. In simple instrumental extinction and habit learning, the latter mechanism may play an especially important role.

Discriminative properties of the reinforcer can be used to attenuate the renewal of extinguished operant behavior

Previous research on the resurgence effect has suggested that reinforcers that are presented during the extinction of an operant behavior can control inhibition of the response. To further test this hypothesis, in three experiments with rat subjects we examined the effectiveness of using reinforcers that were presented during extinction as a means of attenuating or inhibiting the operant renewal effect. In Experiment 1, lever pressing was reinforced in Context A, extinguished in Context B, and then tested in Context A. Renewal of responding that occurred during the final test was attenuated when a distinct reinforcer that had been presented independent of responding during extinction was also presented during the renewal test. Experiment 2 established that this effect depended on the reinforcer being featured as a part of extinction (and thus associated with response inhibition). Experiment 3 then showed that the reinforcers presented during extinction suppressed performance in both the extinction and renewal contexts; the effects of the physical and reinforcer contexts were additive. Together, the results further suggest that reinforcers associated with response inhibition can serve a discriminative role in suppressing behavior and may be an effective stimulus that can attenuate operant relapse.

Role of the discriminative properties of the reinforcer in resurgence

In three experiments with rat subjects, we examined the effects of the discriminative effects of reinforcers that were presented during or after operant extinction. Experiments 1 and 2 examined resurgence, in which an extinguished operant response (R1) recovers when a second behavior (R2) that has been reinforced to replace it is also placed in extinction. The results of Experiment 1 suggest that the amount of R1's resurgence is a decreasing linear function of the interreinforcement interval used during the reinforcement of R2. In Experiment 2, R1 was reinforced with one outcome (O1), and R2 was then reinforced with a second outcome (O2) while R1 was extinguished. In resurgence tests, response-independent (noncontingent) presentations of O2 prevented resurgence of R1, which otherwise occurred when testing was conducted with either no reinforcers or noncontingent presentations of O1. In Experiment 3, we then examined the effects of noncontingent O1 and O2 presentations after simple extinction in either the presence or the absence of noncontingent presentations of O2. Overall, the results are consistent with a role for the discriminative properties of the reinforcer in controlling operant behavior. In resurgence, the reinforcer used during response elimination provides a distinct context that controls the inhibition of R1. The results are less consistent with an alternative view emphasizing the disrupting effects of alternative reinforcement.

Factors that influence the persistence and relapse of discriminated behavior chains

Behavior chains are composed of sequences of behaviors that minimally include procurement and then consumption. This review surveys recent research from this laboratory that has examined the properties of discriminated heterogeneous behavior chains. In contrast to another review (Thrailkill and Bouton, 2016a), it discusses work examining what makes chained behavior persistent, and what makes it relapse. Results suggest that responses in a discriminated heterogeneous behavior chain may become associated, so that extinction of either one reduces the strength of the other. Evidence also suggests that the goal of the first (procurement) response may be the next (consumption) response (rather than the upcoming discriminative stimulus, a putative conditioned reinforcer, or the primary reinforcer at the end of the chain). Further studies suggest that methods that promote generalization across acquisition and extinction (partial reinforcement and delivery of noncontingent reinforcers during extinction) lead to greater persistence of the procurement response. A third set of studies analyzed the contextual control and relapse of chained behaviors. The context controls both the acquisition and extinction of chained behaviors. In addition, a separately-extinguished consumption response is renewed when returned to the context of the chain. The research expands our general understanding of the learning processes that govern instrumental behavior as well as our understanding of chains.

Resurgence and alternative-reinforcer magnitude

Resurgence is defined as an increase in the frequency of a previously reinforced target response when an alternative source of reinforcement is suspended. Despite an extensive body of research examining factors that affect resurgence, the effects of alternative-reinforcer magnitude have not been examined. Thus, the present experiments aimed to fill this gap in the literature. In Experiment 1, rats pressed levers for single-pellet reinforcers during Phase 1. In Phase 2, target-lever pressing was extinguished, and alternative-lever pressing produced either five-pellet, one-pellet, or no alternative reinforcement. In Phase 3, alternative reinforcement was suspended to test for resurgence. Five-pellet alternative reinforcement produced faster elimination and greater resurgence of target-lever pressing than one-pellet alternative reinforcement. In Experiment 2, effects of decreasing alternative-reinforcer magnitude on resurgence were examined. Rats pressed levers and pulled chains for six-pellet reinforcers during Phases 1 and 2, respectively. In Phase 3, alternative reinforcement was decreased to three pellets for one group, one pellet for a second group, and suspended altogether for a third group. Shifting from six-pellet to one-pellet alternative reinforcement produced as much resurgence as suspending alternative reinforcement altogether, while shifting from six pellets to three pellets did not produce resurgence. These results suggest that alternative-reinforcer magnitude has effects on elimination and resurgence of target behavior that are similar to those of alternative-reinforcer rate. Thus, both suppression of target behavior during alternative reinforcement and resurgence when conditions of alternative reinforcement are altered may be related to variables that affect the value of the alternative-reinforcement source.

Preventing relapse after incentivized choice treatment: A laboratory model

Two experiments with rats examined relapse of an operant behavior that occurred after the behavior was suppressed by reinforcing (incentivizing) an alternative behavior. In the first phase, a target response (R1) was reinforced. In a treatment phase, R1 was still reinforced, but a new response (R2) was introduced and associated with a larger reinforcer. As in human contingency management treatments, incentivizing R2 this way was effective at suppressing R1. However, when R2's reinforcement was discontinued, there was a robust and immediate relapse to R1. Experiment 1 found that the strength of R1 during relapse testing was not different from that seen in a no treatment control. Experiment 2 found that relapse could nevertheless be reduced by presenting reinforcers not contingent on responding during the test. Either the reinforcer for R1 or the reinforcer for R2 (which were qualitatively different types of food pellets) were effective. The experiments introduce a laboratory method for studying relapse and how to prevent it after contingency management treatments, and suggest at least one treatment that discourages relapse. The incentivized choice paradigm differs from other models of relapse of operant behavior (e.g., resurgence, renewal, reinstatement) in that it does not focus on the return of behaviors that are inhibited by extinction.

Increasing the persistence of a heterogeneous behavior chain: Studies of extinction in a rat model of search behavior of working dogs

Dogs trained to search for contraband perform a chain of behavior in which they first search for a target and then make a separate response that indicates to the trainer that they have found one. The dogs often conduct multiple searches without encountering a target and receiving the reinforcer (i.e., no contraband is present). Understanding extinction (i.e., the decline in work rate when reinforcers are no longer encountered) may assist in training dogs to work in conditions where targets are rare. We therefore trained rats on a search-target behavior chain modeled on the search behavior of working dogs. A discriminative stimulus signaled that a search response (e.g., chain pull) led to a second stimulus that set the occasion for a target response (e.g., lever press) that was reinforced by a food pellet. In Experiment 1 training with longer search durations and intermittent (partial) reinforcement of searching (i.e. some trials had no target present) both led to more persistent search responding in extinction. The loss of search behavior in extinction was primarily dependent on the number of non-reinforced searches rather than time searching without reinforcement. In Experiments 2 and 3, delivery of non-contingent reinforcers during extinction increased search persistence provided they had also been presented during training. Thus, results with rats suggest that the persistence of working dog performance (or chained behavior generally) may be improved by training with partial reinforcement of searching and non-contingent reinforcement during both training and work (extinction).

Behavioral momentum theory fails to account for the effects of reinforcement rate on resurgence

The behavioral-momentum model of resurgence predicts reinforcer rates within a resurgence preparation should have three effects on target behavior. First, higher reinforcer rates in baseline (Phase 1) produce more persistent target behavior during extinction plus alternative reinforcement. Second, higher rate alternative reinforcement during Phase 2 generates greater disruption of target responding during extinction. Finally, higher rates of either reinforcement source should produce greater responding when alternative reinforcement is suspended in Phase 3. Recent empirical reports have produced mixed results in terms of these predictions. Thus, the present experiment further examined reinforcer-rate effects on persistence and resurgence. Rats pressed target levers for high-rate or low-rate variable-interval food during Phase 1. In Phase 2, target-lever pressing was extinguished, an alternative nose-poke became available, and nose-poking produced either high-rate variable-interval, low-rate variable-interval, or no (an extinction control) alternative reinforcement. Alternative reinforcement was suspended in Phase 3. For groups that received no alternative reinforcement, target-lever pressing was less persistent following high-rate than low-rate Phase-1 reinforcement. Target behavior was more persistent with low-rate alternative reinforcement than with high-rate alternative reinforcement or extinction alone. Finally, no differences in Phase-3 responding were observed for groups that received either high-rate or low-rate alternative reinforcement, and resurgence occurred only following high-rate alternative reinforcement. These findings are inconsistent with the momentum-based model of resurgence. We conclude this model mischaracterizes the effects of reinforcer rates on persistence and resurgence of operant behavior.

Animal models of drug relapse and craving: From drug priming-induced reinstatement to incubation of craving after voluntary abstinence

High rates of relapse to drug use during abstinence is a defining feature of drug addiction. In abstinent drug users, drug relapse is often precipitated by acute exposure to the self-administered drug, drug-associated cues, stress, as well as by short-term and protracted withdrawal symptoms. In this review, we discuss different animal models that have been used to study behavioral and neuropharmacological mechanisms of these relapse-related phenomena. In the first part, we discuss relapse models in which abstinence is achieved through extinction training, including the established reinstatement model, as well as the reacquisition and resurgence models. In the second part, we discuss recent animal models in which drug relapse is assessed after either forced abstinence (e.g., the incubation of drug craving model) or voluntary (self-imposed) abstinence achieved either by introducing adverse consequences to ongoing drug self-administration (e.g., punishment) or by an alternative nondrug reward using a discrete choice (drug vs. palatable food) procedure. We conclude by briefly discussing the potential implications of the recent developments of animal models of drug relapse after voluntary abstinence to the development of medications for relapse prevention.

Effect of the Novel Positive Allosteric Modulator of Metabotropic Glutamate Receptor 2 AZD8529 on Incubation of Methamphetamine Craving After Prolonged Voluntary Abstinence in a Rat Model

BACKGROUND

Cue-induced methamphetamine craving increases after prolonged forced (experimenter-imposed) abstinence from the drug (incubation of methamphetamine craving). Here, we determined whether this incubation phenomenon would occur under conditions that promote voluntary (self-imposed) abstinence. We also determined the effect of the novel metabotropic glutamate receptor 2 positive allosteric modulator, AZD8529, on incubation of methamphetamine craving after forced or voluntary abstinence.

METHODS

We trained rats to self-administer palatable food (6 sessions) and then to self-administer methamphetamine under two conditions: 12 sessions (9 hours/day) or 50 sessions (3 hours/day). We then assessed cue-induced methamphetamine seeking in extinction tests after 1 or 21 abstinence days. Between tests, the rats underwent either forced abstinence (no access to the food- or drug-paired levers) or voluntary abstinence (achieved via a discrete choice procedure between methamphetamine and palatable food; 20 trials per day) for 19 days. We also determined the effect of subcutaneous injections of AZD8529 (20 and 40 mg/kg) on cue-induced methamphetamine seeking 1 day or 21 days after forced or voluntary abstinence.

RESULTS

Under both training and abstinence conditions, cue-induced methamphetamine seeking in the extinction tests was higher after 21 abstinence days than after 1 day (incubation of methamphetamine craving). AZD8529 decreased cue-induced methamphetamine seeking on day 21 but not day 1 of forced or voluntary abstinence.

CONCLUSIONS

We introduce a novel animal model to study incubation of drug craving and cue-induced drug seeking after prolonged voluntary abstinence, mimicking the human condition of relapse after successful contingency management treatment. Our data suggest that positive allosteric modulators of metabotropic glutamate receptor 2 should be considered for relapse prevention.

Context change explains resurgence after the extinction of operant behavior

Extinguished operant behavior can return or "resurge" when a response that has replaced it is also extinguished. Typically studied in nonhuman animals, the resurgence effect may provide insight into relapse that is seen when reinforcement is discontinued following human contingency management (CM) and functional communication training (FCT) treatments, which both involve reinforcing alternative behaviors to reduce behavioral excess. Although the variables that affect resurgence have been studied for some time, the mechanisms through which they promote relapse are still debated. We discuss three explanations of resurgence (response prevention, an extension of behavioral momentum theory, and an account emphasizing context change) as well as studies that evaluate them. Several new findings from our laboratory concerning the effects of different temporal distributions of the reinforcer during response elimination and the effects of manipulating qualitative features of the reinforcer pose a particular challenge to the momentum-based model. Overall, the results are consistent with a contextual account of resurgence, which emphasizes that reinforcers presented during response elimination have a discriminative role controlling behavioral inhibition. Changing the "reinforcer context" at the start of testing produces relapse if the organism has not learned to suppress its responding under conditions similar to the ones that prevail during testing.

Examination of the role of dopamine D₂ and adrenergic α₂ receptors in resurgence of food seeking

Resurgence refers to the reappearance of an extinguished operant behavior when reinforcement for an alternative behavior is also subsequently discontinued. Resurgence has been noted as a source of relapse to problem behavior following interventions involving alternative reinforcement, and has also been recently used as an animal model of relapse to drug seeking induced by reinforcement loss. Existing information about the neuropharmacology of resurgence is scarce, but suggests overlap between relapse observed in the resurgence model and relapse observed in reinstatement and renewal models. In the present experiment rats earned food pellets for pressing a target lever in Phase I. In Phase II lever pressing no longer produced food, but food was delivered for an alterative nose poke response. Finally in Phase III, neither response produced food deliveries. Prior to these Phase III sessions, separate groups of rats were injected with 0, 50, or 100 μg/kg of the dopamine D2 receptor antagonist raclopride or 0, 20, or 40 μg/kg of α 2 agonist clonidine. Both doses of raclopride were effective in blocking resurgence, but there was evidence that the higher dose did so via motor rather than motivational impairment. Only the higher dose of clonidine blocked resurgence, but did so with no evidence of motor impairment. Raclopride significantly impacted extinction of the alternative poke at both doses tested, whereas clonidine had no effect at either dose. The results of the present study provide additional information about the neuropharmacology of resurgence, as well as additional evidence of overlap between resurgence, reinstatement, and renewal.

Behavioral and neurobiological mechanisms of extinction in Pavlovian and instrumental learning

This article reviews research on the behavioral and neural mechanisms of extinction as it is represented in both Pavlovian and instrumental learning. In Pavlovian extinction, repeated presentation of a signal without its reinforcer weakens behavior evoked by the signal; in instrumental extinction, repeated occurrence of a voluntary action without its reinforcer weakens the strength of the action. In either case, contemporary research at both the behavioral and neural levels of analysis has been guided by a set of extinction principles that were first generated by research conducted at the behavioral level. The review discusses these principles and illustrates how they have informed the study of both Pavlovian and instrumental extinction. It shows that behavioral and neurobiological research efforts have been tightly linked and that their results are readily integrated. Pavlovian and instrumental extinction are also controlled by compatible behavioral and neural processes. Since many behavioral effects observed in extinction can be multiply determined, we suggest that the current close connection between behavioral-level and neural-level analyses will need to continue.

Recent developments in animal models of drug relapse

Drug craving and relapse to drug use during abstinence are defining features of addiction. Evidence indicates that drug craving and relapse in humans are often provoked by acute exposure to the self-administered drug, drug-associated cues, or stress. During the last two decades, this clinical scenario has been primarily studied at the preclinical level using the classical reinstatement model. However, a single preclinical model cannot capture the complicated nature of human drug relapse. Therefore, more recently, we and others have developed several other models to study different facets of human drug relapse. In this review, we introduce and discuss recent findings from these other relapse models, including incubation of drug craving, reacquisition and resurgence models, and punishment-based and conflict-based relapse models.

Some Factors Modulating the Strength of Resurgence After Extinction of an Instrumental Behavior

In resurgence, an operant behavior that has undergone extinction can return ("resurge") when a second operant that has replaced it itself undergoes extinction. The phenomenon may provide insight into relapse that may occur after incentive or contingency management therapies in humans. Three experiments with rats examined the impact of several variables on the strength of the resurgence effect. In each, pressing one lever (L1) was first reinforced and then extinguished while pressing a second, alternative, lever (L2) was now reinforced. When L2 responding was then itself extinguished, L1 responses resurged. Experiment 1 found that resurgence was especially strong after an extensive amount of L1 training (12 as opposed to 4 training sessions) and after L1 was reinforced on a random ratio schedule as opposed to a variable interval schedule that was matched on reinforcement rate. Experiment 2 found that after 12 initial sessions of L1 training, 4, 12, or 36 sessions of Phase 2 each allowed substantial (and apparently equivalent) resurgence. Experiment 3 found no effect of changing the identity of the reinforcer (from grain pellet to sucrose pellet or sucrose to grain) on the amount of resurgence. The results suggest that resurgence can be robust; in the natural world, an operant behavior with an extensive reinforcement history may still resurge after extensive incentive-based therapy. The results are discussed in terms of current explanations of the resurgence effect.

Effects of thinning the rate at which the alternative behavior is reinforced on resurgence of an extinguished instrumental response

Three experiments with rats examined the effects of thinning the rate of reinforcement for the alternative behavior in the resurgence paradigm. In all experiments, pressing one lever (L1) was first reinforced and then extinguished while pressing a second alternative lever (L2) was then reinforced. When L2 responding was then extinguished, L1 responses "resurged." Resurgence was always observed when L2 was reinforced on an unchanging reinforcement schedule during Phase 2. However, other rats received systematic decreases in the rate of L2 reinforcement before extinction of L2 began. Such a "thinning" procedure was predicted to reduce final resurgence by associating L1 extinction with longer and longer periods without a reinforcer. The procedure did reduce the resurgence effect observed when L2 was put on extinction (Experiment 3). However, in each experiment, thinned groups also returned to L1 responding, and continued to make L1 responses, while the reinforcement schedule for L2 was being thinned. Fine-grained analysis of behavior in time suggested that this early resurgence was not due to adventitious reinforcement of L1, occasion setting of L1 by reinforcer presentation, or the entrainment of L1 as a schedule-induced interim behavior. The results are overall consistent with the hypothesis that resurgence is a renewal effect in which extinguished L1 responding recovers when the context provided by the L2 reinforcement schedule is changed. Challenges for this view are also discussed.

Relapse processes after the extinction of instrumental learning: renewal, resurgence, and reacquisition

It is widely recognized that extinction (the procedure in which a Pavlovian conditioned stimulus or an instrumental action is repeatedly presented without its reinforcer) weakens behavior without erasing the original learning. Most of the experiments that support this claim have focused on several "relapse" effects that occur after Pavlovian extinction, which collectively suggest that the original learning is saved through extinction. However, although such effects do occur after instrumental extinction, they have not been explored there in as much detail. This article reviews recent research in our laboratory that has investigated three relapse effects that occur after the extinction of instrumental (operant) learning. In renewal, responding returns after extinction when the behavior is tested in a different context; in resurgence, responding recovers when a second response that has been reinforced during extinction of the first is itself put on extinction; and in rapid reacquisition, extinguished responding returns rapidly when the response is reinforced again. The results provide new insights into extinction and relapse, and are consistent with principles that have been developed to explain extinction and relapse as they occur after Pavlovian conditioning. Extinction of instrumental learning, like Pavlovian learning, involves new learning that is relatively dependent on the context for expression.

Effects of D-cycloserine on the extinction of appetitive operant learning

Four experiments with rat subjects examined whether D-cycloserine (DCS), a partial NMDA agonist, facilitates the extinction of operant lever-pressing reinforced by food. Previous research has demonstrated that DCS facilitates extinction learning with methods that involve Pavlovian extinction. In the current experiments, operant conditioning occurred in Context A, extinction in Context B, and then testing occurred in both the extinction and conditioning contexts. Experiments 1A and 1B tested the effects of three doses of DCS (5, 15, and 30 mg/kg) on the extinction of lever pressing trained as a free operant. Experiment 2 examined their effects when extinction of the free operant was conducted in the presence of nonresponse-contingent deliveries of the reinforcer (that theoretically reduced the role of generalization decrement in suppressing responding). Experiment 3 examined their effects on extinction of a discriminated operant, that is, one that had been reinforced in the presence of a discriminative stimulus, but not in its absence. A strong ABA renewal effect was observed in all four experiments during testing. However, despite the use of DCS doses and a drug administration procedure that facilitates the extinction of Pavlovian learning, there was no evidence in any experiment that DCS facilitated operant extinction learning assessed in either the extinction or the conditioning context. DCS may primarily facilitate learning processes that underlie Pavlovian, rather than purely operant, extinction.