Role of the discriminative properties of the reinforcer in resurgence

Resurgence Following Higher or Lower Quality Alternative Reinforcement

Resurgence is a temporary increase in a previously suppressed target behavior following a worsening in reinforcement conditions. Previous studies have examined how higher rates or magnitudes of alternative reinforcement affect suppression of the target behavior and subsequent resurgence. However, there has been no investigation of the effects of higher versus lower qualities of alternative reinforcement on resurgence. Using a three-phase resurgence preparation with rats, the present experiments examined the effects of an alternative reinforcer that was of higher (Experiment 1) or lower (Experiment 2) quality than the reinforcer that had previously maintained the target behavior. The results of both experiments showed greater reductions in target behavior with a higher quality alternative reinforcer and larger increases in target responding when a higher quality alternative reinforcer was removed. Along with prior findings with higher rates and magnitudes of alternative reinforcement, these findings suggest that variations in reinforcer dimensions that increase the efficacy of alternative reinforcement also tend to increase resurgence when alternative reinforcement is removed. The results are discussed in terms of the resurgence as choice in context model and in terms of potential clinical implications.

Resurgence of goal-directed actions and habits

This study investigated how goal-directed and habitual behaviors recover after extinction within the context of the resurgence effect, a form of relapse induced by the removal or worsening of alternative reinforcement. Rats were trained to press a target lever with one reinforcer (O1) for either minimal (4) or extended (16) sessions. An extinction test after the completion of O1 devaluation confirmed that minimal and extended training formed goal-directed and habitual behaviors, respectively. Then, pressing an alternative lever was reinforced with a second reinforcer (O2) while the target response was placed on extinction. When O2 was discontinued, the minimally trained target response resurged with goal-directed status as in the extinction test. However, the extinguished habitual behavior in the extensively trained rats did not recover as a habit but instead with goal-directed status, possibly due to the context specificity of habits or the introduction of a new response-reinforcer contingency. The critical finding that reinforcer devaluation consistently led to less resurgence regardless of the amount of acquisition training provides a clinical implication that coupling differential-reinforcement-of-alternative-behavior (DRA) treatments with the devaluation of the associated reinforcer of problematic behavior could effectively diminish its recurrence.

The impact of prolonging extinction on the ABC "super renewal" of instrumental responses in rats

Two free operant conditioning experiments with rats examined the impact of conducting a large amount of extinction training on situations that enhance the ABC renewal effect (ABC super renewal). In Experiment 1, ABC renewal was strengthened by conducting acquisition in multiple contexts. All rats were trained to press a lever for food. One group was trained in one context, while the other two groups were trained in three contexts. Then, all rats received extinction in context B. For two groups this phase lasted 4 sessions, whereas it lasted 36 sessions for the other group. In Experiment 2, ABC renewal was strengthened by using a large number of acquisition sessions. Rats were trained to perform an operant response to obtain food in context A. One group received a moderate amount of training, while the rest of the rats received a larger number of acquisition sessions. Responses underwent extinction in context B. Two groups received 4 sessions, while 36 extinction sessions were used for the remaining group. In both experiments, rats were tested in context B (extinction context) and C (renewal context). Greater ABC renewal occurred both when acquisition training was conducted in multiple contexts (Experiment 1) and by increasing the amount of acquisition training (Experiment 2). Nevertheless, we found that conducting a large number of extinction sessions reduced ABC super renewal in Experiment 1 only.

Relapse and its mitigation: Toward behavioral inoculation

Relapse following the successful treatment of problem behavior can increase the likelihood of injury and the need for more intensive care. Current research offers some predictions of how treatment procedures may contribute to relapse, and conversely, how the risk of relapse can be mitigated. This review describes relapse-mitigation procedures with varying levels of support, the quantitative models that have influenced the research on relapse mitigation, different experimental methods for measuring relapse mitigation, and directions for future research. We propose that by viewing the implementation of relapse-mitigation procedures as a means of producing behavioral inoculation, clinicians are placed in the proactive and intentional role of exposing their client's behavior to an array of reinforcement and stimulus conditions during treatment with the goal of decreasing the detrimental impact of future treatment challenges.

Learning to stop responding

Learning to stop responding is an important process that allows behavior to adapt to a changing and variable environment. This article reviews recent research in this laboratory and others that has studied how animals learn to stop responding in operant extinction, punishment, and feature-negative learning. Extinction and punishment are shown to be similar in two fundamental ways. First, the response-suppressing effects of both are highly context-specific. Second, the response-suppressing effects of both can be remarkably response-specific: Inhibition of one response transfers little to other responses. Learning to inhibit the response so specifically may result from the correction of "response error," the difference between the level of responding and what the current reinforcer supports. In contrast, the inhibition of responding that develops in feature-negative learning, where the response is reinforced during one discriminative stimulus (A) but not in a compound of A and stimulus B, is less response-specific: The inhibition of responding by stimulus B transfers and inhibits a second response, especially if the second response has itself been inhibited before. The results thus indicate both response-specific and response-general forms of behavioral inhibition. One possibility is that response-specific inhibition is learned when the circumstances encourage the organism to pay attention to the response-to what it is actually doing-as behavioral suppression is learned.

A Comprehensive Systematic Review of Procedures and Analyses Used in Basic and Preclinical Studies of Resurgence, 1970-2020

Resurgence is the return of a previously reinforced response as conditions worsen for an alternative response, such as the introduction of extinction, reductions in reinforcement, or punishment. As a procedure, resurgence has been used to model behavioral treatments and understand behavioral processes contributing both to relapse of problem behavior and flexibility during problem-solving. Identifying existing procedural and analytic methods arranged in basic/preclinical research could be used by basic and preclinical researchers to develop novel approaches to study resurgence, whereas translational and clinical researchers could identify potential approaches to combating relapse during behavioral interventions. Despite the study of resurgence for over half a century, there have been no systematic reviews of the basic/preclinical research on resurgence. To characterize the procedural and analytic methods used in basic/preclinical research on resurgence, we performed a systematic review consistent with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). We identified 120 articles consisting of 200 experiments that presented novel empirical research, examined operant behavior, and included standard elements of a resurgence procedure. We reported prevalence and trends in over 60 categories, including participant characteristics (e.g., species, sample size, disability), designs (e.g., single subject, group), procedural characteristics (e.g., responses, reinforcer types, control conditions), criteria defining resurgence (e.g., single test, multiple tests, relative to control), and analytic strategies (e.g., inferential statistics, quantitative analysis, visual inspection). We make some recommendations for future basic, preclinical, and clinical research based on our findings of this expanding literature.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40614-022-00361-y.

Motivational state-dependent renewal and reinstatement of operant responding under food and water deprivation states

Most studies of operant relapse phenomena have studied the role of exteroceptive discriminative features of context in the recovery of behavior after extinction. Interoceptive stimuli (i.e., stimuli arising from the bodies of organisms) may also serve as a part of learning contexts and contribute to relapse. Recent studies have demonstrated that events that function as motivating operations (MOs) may contribute to relapse both by a) eliciting interoceptive conditions that serve as a discriminative context for relapse (i.e., discriminative function) and b) altering the amount of operant responding in the presence of discriminative stimuli associated with reinforcers (i.e., motivational function). The current study examined interactions between these functions of food and water deprivation MOs in the relapse of operant behavior. During acquisition of an operant response, one group of mice was food-deprived and another was water-deprived. The groups then received extinction sessions under the opposite condition. Renewal and reinstatement tests were conducted under both conditions, and more renewal and reinstatement were observed in the motivational states in which acquisition occurred for each. These results are discussed in the context of state-dependent learning.

Relapse after incentivized choice treatment in humans: A laboratory model for studying behavior change

Behavior change interventions that incentivize desired behavior are highly effective for improving personal health, but difficult to maintain long term. Relapse is common and examining the mechanisms that contribute to relapse in experimental settings can identify processes relevant to substance abuse treatment. We developed a laboratory task that parallels a recent operant model of relapse after incentivized choice reported in the rodent laboratory. In two experiments, undergraduate participants first learned to make an operant response (keyboard button; R1) to earn a reinforcer consisting of an image of a preferred snack food (O1). In a second phase (Phase 2), R1 was still reinforced, but a new response (R2) was introduced and reinforced with a different reinforcer (a coin; O2). In a test phase, contingent incentives for R2 were removed (extinction) and relapse of R1 was assessed. Experiment 1 found that the O2 contingency suppressed R1 during Phase 2, and R1 relapsed rapidly in the test. Neither effect was consistently related to O2 value. Experiment 2 examined whether noncontingent presentations of O1 or O2 during the test could weaken relapse. Here, we found that noncontingent reinforcers did little to reduce or slow the increase in R1 responding. The present experiments highlight a laboratory approach to studying variables that may influence relapse after incentivized treatment. We identify and discuss areas for development to address differences between the present results and prior observations from animal and clinical studies. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Examination of alternative-response discrimination training and resurgence in rats

Resurgence is an increase in a previously reinforced behavior following a worsening of conditions for a more recently reinforced behavior. Discrimination training is incorporated into treatment for problem behavior to prevent treatment adherence failures that may result in resurgence. There is evidence that resurgence may be reduced when a stimulus that signals alternative-response extinction is present compared with absent; however, the generality of this effect is unknown given the limited testing conditions. The goal of the present experiments was to further examine the effects of such stimuli in a reverse-translational evaluation using rats. Target responding was reinforced in baseline and then placed on extinction in the following discrimination-training phase. An alternative response was differentially reinforced in a two-component multiple schedule where one stimulus (i.e., S^D) signaled alternative-response reinforcement and the other (i.e., S^Δ) signaled extinction. Experiment 1 assessed resurgence in both the S^Δ and S^D when alternative reinforcement was removed. Experiment 2 evaluated resurgence under conditions that better approximated those used in the clinic in which the alternative-response S^Δ was present or absent. The S^Δ failed to suppress target responding during resurgence testing in both experiments. These findings suggest that the conditions under which an alternative-response S^Δ will successfully mitigate resurgence may be limited and require further research.

Recaída de Variabilidade Operante com a Combinação dos Procedimentos de Renovação, Restabelecimento e Ressurgência

Resumo O reaparecimento de variabilidade comportamental previamente extinta (i.e., recaída) foi investigado por meio dos modelos experimentais de renovação, restabelecimento e ressurgência. Na Fase de Treino (contexto A), ao serem expostos ao esquema múltiplo Lag 10 Acoplado, ratos apresentaram níveis similares de variabilidade nos dois componentes. Na Fase de Eliminação (contexto B), o esquema múltiplo Repetição Repetição promoveu a extinção da variabilidade. Na Fase de Teste (contexto A), com a suspensão da contingência de repetição e a liberação de reforços independentes, a variabilidade reapareceu no componente “Lag 10”, anteriormente correlacionado com reforçamento da variação, mas não no componente “Acoplado”. Esse resultado sugere que a variabilidade observada no teste de recaída corresponde à variabilidade operante, e não à variabilidade induzida pela extinção.

Resurgence and downshifts in alternative reinforcement rate

Resurgence refers to an increase in a previously suppressed target behavior with a relative worsening of conditions for a more recently reinforced alternative behavior. This experiment examined the relation between resurgence and the magnitude of a reduction in the rate of reinforcement for the alternative behavior. Groups of both male and female rats initially pressed a target lever for food on a variable-interval (VI) 30-s schedule. In a second phase, responding to the target lever was extinguished for all groups and pressing an alternative lever was reinforced on a VI 10-s schedule. Next, the rate of reinforcement for alternative behavior was reduced differentially across groups by arranging extinction, VI 80-s, VI 40-s, VI 20-s, or continued VI 10-s reinforcement. Target responding increased as an exponential function of the magnitude of the reduction in alternative reinforcement rates. With the exception that males appeared to show higher rates of target responding in baseline and higher rates of alternative responding in other phases, the overall pattern of responding across phases was not meaningfully different between sexes. The pattern of both target and alternative response rates across sessions and phases was well described quantitatively by the Resurgence as Choice in Context model.

Cues Associated with Alternative Reinforcement During Extinction Can Attenuate Resurgence of an Extinguished Instrumental Response

In resurgence, a target behavior (R1) is acquired in an initial phase and extinguished in a second phase while an R2 behavior is reinforced. When R2 is extinguished, R1 behavior can return or resurge. Two experiments tested the effectiveness of a potential retrieval cue associated with extinction in attenuating resurgence. Experiment 1 established that a 2-s cue paired with outcome delivery in Phase 2 can attenuate resurgence when presented during testing. This effect depended on the cue being associated with the outcome, and it occurred if the cue was delivered contingently or noncontingently on responding during testing. Pairing the cue with reinforcement might be necessary to maintain attention to it during Phase 2. Experiment 2 demonstrated that the cue must be experienced in sessions that also include R1 extinction and that it does not reduce resurgence through a conditioned reinforcement mechanism. The results suggest that previously neutral stimuli can attenuate resurgence if they are first paired with alternative reinforcement and presented in sessions in which R1 is extinguished. They build on existing literature that suggests enhancing generalization between extinction and testing reduces resurgence. The results may have implications for reducing relapse following interventions in humans such as contingency management (CM), in which participants can earn vouchers contingent upon drug abstinence. A cue associated with CM might help reduce this relapse.

Relapse of operant behavior after response elimination with an extinction or an omission contingency

The present study compared relapse after responding was eliminated by extinction or omission training in rats. In Experiment 1, lever pressing was reinforced with food pellets in Context A and then eliminated with either extinction or omission training in Context B. The response was then tested in Contexts A and B in either the presence or absence of free food pellets delivered on a random time schedule. All rats showed higher responding when tested in Context A than Context B, and there was little evidence that omission training attenuated this ABA renewal effect. Noncontingent pellets increased responding after extinction but not after omission. However, when responding on the last day of response elimination was compared to responding during the test in the response-elimination context, there was some evidence that omission-trained rats showed a small increase in responding even when tested with free pellets. Results of Experiment 2 suggest this increase was not due to differences in the temporal distribution of pellets during elimination and the test, and that the result might be due to mere removal of the omission contingency, but any such effect is small and difficult to detect statistically. The results provide new information about factors generating relapse after omission training.

Voucher-Based Contingency Management is Efficacious but Underutilized in Treating Addictions

The United States lags far behind other industrialized countries on major markers of population health. Population health experts identify unhealthy behavior patterns (e.g., cigarette smoking, other substance use disorders, physical inactivity and poor food choices, nonadherence with recommended medical regimens) as the largest contributor to the status quo. Because these risk behaviors are overrepresented in socioeconomically disadvantaged and other vulnerable populations, they also increase health disparities. Hence, identifying evidence-based strategies to promote and sustain health-related behavior change is critical to improving U.S. population health. In this report, we review research demonstrating the efficacy of voucher-based contingency management delivered alone or in combination with other interventions for treating substance use disorders and other health-related behavior problems. The efficacy supporting these interventions is robust and discernible at the level of controlled randomized clinical trials and meta-analyses. Unfortunately, these evidence-based interventions are being underutilized in routine clinical care for substance use disorders, although they are used broadly in private-sector wellness programs and international programs to reduce chronic poverty. This report reviews the evidence supporting the efficacy of voucher-based contingency management using projects developed at the University of Vermont as exemplars and discusses dissemination of the model to public and private sector efforts to improve individual and population health.

The reoccurrence of voluntary behavior in humans is reduced by retrieval cues from extinction

Changes in the temporal as well as the physical context produces the reappearance of extinguished behaviors. Furthermore, combining both kinds of contextual stimuli often causes greater levels of recovery. The current experiment explored the impact of extinction reminders on spontaneous recovery, renewal, and a combination of both effects using an instrumental learning task with humans. All participants learned to shoot at enemies in a videogame. Then, throughout extinction, the instrumental response was eliminated. We found a return of the extinguished behavior by introducing a retention interval of 48 h, by changing the physical background and by testing participants in a spatiotemporal context different from the extinction context. However, we also found that the presentation of a stimulus directly associated with extinction attenuates all three forms of operant reoccurrence. These results are consistent with the perspective that emphasizes that context plays a key role in response-recovery phenomena. Moreover, our findings may be promissory for therapeutic strategies involving relapse treatment.

Resurgence in humans: Reducing relapse by increasing generalization between treatment and testing

Resurgence is the increase in performance of an extinguished instrumental (operant) response that accompanies the extinction of a response that has been reinforced to replace it. Resurgence may involve processes that are relevant for understanding relapse in applied and clinical settings. While resurgence is known to be a robust phenomenon in human operant extinction, the processes that control it remain unclear. Here we asked whether human resurgence is controlled by processes that are similar to those that have been identified in animals by asking whether two methods that reduce resurgence in animals also reduce it in humans. Participants first learned to make an operant response (R1) for a tangible food reinforcer (O1). In a second phase (Phase 2), R1 was extinguished while a second response (R2) was introduced and reinforced with a virtual monetary reward (USD $0.10 coins; O2). In a test phase, extinction was then introduced for R2 and resurgence of R1 was assessed. In Experiment 1, resurgence that occurred after the treatment just described was attenuated if there had been periodic exposure to R2 extinction during the treatment phase (Phase 2). In Experiment 2, resurgence was prevented when O2, but not O1, was presented noncontingently during the test. The results are among the first to suggest a mechanism underlying resurgence in humans, namely, renewal caused by contextual change. They also provide initial evidence to suggest that resurgence may be the result of common processes in animals and humans. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Resurgence of a target behavior suppressed by a combination of punishment and alternative reinforcement

Differential-reinforcement-based treatments involving extinction of target problem behavior and reinforcement of an alternative behavior are highly effective. However, extinction of problem behavior is sometimes difficult or contraindicated in clinical settings. In such cases, punishment instead of extinction may be used in combination with alternative reinforcement. Although it is well documented that omitting alternative reinforcement can produce recurrence (i.e., resurgence) of behavior previously suppressed by extinction plus alternative reinforcement, it remains unclear if resurgence similarly occurs for behavior previously suppressed by punishment plus alternative reinforcement. The present experiment examined this question with rats. In Phase 1, a target behavior (lever pressing) was reinforced with food pellets. In Phase 2, the target behavior continued to be reinforced, but it also produced mild foot shock and an alternative behavior (nose poking) also produced food. Finally, all consequences were removed and resurgence of target behavior occurred. Resurgence did not occur for another group that similarly received punishment of target behavior in Phase 2 but not alternative reinforcement. These results indicate that resurgence was a product of the history of exposure to and then removal of alternative reinforcement and that the removal of punishment alone did not produce resurgence of target behavior.

Prediction and control of operant behavior: What you see is not all there is

Prediction and control of operant behavior are major goals of behavior analysis. We suggest that achieving these goals can benefit from doing more than identifying the three-term contingency between the behavior, its setting stimulus, and its consequences. Basic research now underscores the idea that prediction and control require consideration of the behavior's history. As one example, if an operant is a goal-directed action, it is controlled by the current value of the reinforcer, as illustrated by the so-called reinforcer devaluation effect. In contrast, if the behavior is a habit, it occurs automatically, without regard to the reinforcer's value, as illustrated by its insensitivity to the reinforcer devaluation effect. History variables that distinguish actions and habits include the extent of their prior practice and their schedule of reinforcement. Other operants can appear to have very low or zero strength. However, if the behavior has reached that level through extinction or punishment, it may precipitously increase in strength by changing the context, allowing time to pass, presenting the reinforcer contingently or noncontingently, or extinguishing an alternative behavior. Behaviors that are not suppressed by extinction or punishment are not affected the same way. When predicting the strength of an operant behavior, what you see is not all there is. The behavior's history counts.

Extinction of instrumental (operant) learning: interference, varieties of context, and mechanisms of contextual control

This article reviews recent research on the extinction of instrumental (or operant) conditioning from the perspective that it is an example of a general retroactive interference process. Previous discussions of interference have focused primarily on findings from Pavlovian conditioning. The present review shows that extinction in instrumental learning has much in common with other examples of retroactive interference in instrumental learning (e.g., omission learning, punishment, second-outcome learning, discrimination reversal learning, and differential reinforcement of alternative behavior). In each, the original learning can be largely retained after conflicting information is learned, and behavior is cued or controlled by the current context. The review also suggests that a variety of stimuli can play the role of context, including room and apparatus cues, temporal cues, drug state, deprivation state, stress state, and recent reinforcers, discrete cues, or behaviors. In instrumental learning situations, the context can control behavior through its direct association with the reinforcer or punisher, through its hierarchical relation with response-outcome associations, or its direct association (inhibitory or excitatory) with the response. In simple instrumental extinction and habit learning, the latter mechanism may play an especially important role.

Renewal of extinguished operant behavior following changes in social context

Renewal is the reoccurrence of previously extinguished behavior following a change in the context in which extinction was conducted. Despite an extensive body of research examining renewal, little is known about the role of social stimuli in renewal. The present experiments provided a novel examination of renewal of operant behavior by changing social stimuli across phases in an ABA renewal preparation. In both experiments, social stimuli were arranged by placing another rat in a second compartment of a divided operant chamber. In Experiment 1, the presence of another rat defined the extinction context, whereas an empty second compartment defined the baseline and testing contexts. We reversed these contextual manipulations in Experiment 2 such that the presence of another rat defined the baseline and testing contexts and the second compartment was empty during extinction. Renewal of lever pressing occurred when the other rat was removed from the chamber in Experiment 1 and when the other rat was returned to the chamber in Experiment 2. Thus, social stimuli may function as contextual stimuli, and changes in social contexts may produce renewal of previously extinguished behavior.

Factors that encourage generalization from extinction to test reduce resurgence of an extinguished operant response

Two experiments investigated methods that reduce the resurgence of an extinguished behavior (R1) that occurs when reinforcement for an alternative behavior (R2) is discontinued. In Experiment 1, R1 was first trained and then extinguished while R2 was reinforced during a 5- or 25-session treatment phase. For half the rats, sessions in which R2 was reinforced alternated with sessions in which R2 was extinguished. Controls received the same number of treatment sessions, but R2 was never extinguished. When reinforcement for R2 was discontinued, R1 resurged in the controls. However, the alternating groups showed reduced resurgence, and the magnitude of the resurgences observed during their R2 extinction sessions decreased systematically over Phase 2. In Experiment 2, R1 was first reinforced with one outcome (O1). The rats then had two types of double-alternating treatment sessions. In one type, R1 was extinguished and R2 produced O2. In the other, R1 was unavailable and R2 produced O3. R1 resurgence was weakened when O2, but not O3, was delivered freely during testing. Together, the results suggest that methods that encourage generalization between R1 extinction and resurgence testing weaken the resurgence effect. They are not consistent with an account of resurgence proposed by Shahan and Craig (2017).

Discriminative properties of the reinforcer can be used to attenuate the renewal of extinguished operant behavior

Previous research on the resurgence effect has suggested that reinforcers that are presented during the extinction of an operant behavior can control inhibition of the response. To further test this hypothesis, in three experiments with rat subjects we examined the effectiveness of using reinforcers that were presented during extinction as a means of attenuating or inhibiting the operant renewal effect. In Experiment 1, lever pressing was reinforced in Context A, extinguished in Context B, and then tested in Context A. Renewal of responding that occurred during the final test was attenuated when a distinct reinforcer that had been presented independent of responding during extinction was also presented during the renewal test. Experiment 2 established that this effect depended on the reinforcer being featured as a part of extinction (and thus associated with response inhibition). Experiment 3 then showed that the reinforcers presented during extinction suppressed performance in both the extinction and renewal contexts; the effects of the physical and reinforcer contexts were additive. Together, the results further suggest that reinforcers associated with response inhibition can serve a discriminative role in suppressing behavior and may be an effective stimulus that can attenuate operant relapse.

Delivering alternative reinforcement in a distinct context reduces its counter-therapeutic effects on relapse

Delivery of alternative reinforcers in the presence of stimuli previously associated with reinforcement for target behavior increases the susceptibility of target behavior to relapse. To explore contingencies that might mitigate this counter-therapeutic effect, we trained pigeons on a procedure that entailed extinction of previously reinforced target-key pecking, access to a distinct stimulus context contingently on refraining from target behavior (differential-reinforcement-of-other-behavior; DRO), and reinforcement of alternative-key pecks (differential-reinforcement of alternative behavior; DRA) in that context. This DRO-DRA treatment was compared with standard DRA in successive conditions, counterbalanced across pigeons. Target behavior extinguished more rapidly in the Standard-DRA condition. When alternative reinforcement was discontinued, however, there was less resurgence after DRO-DRA than after Standard DRA. In a third condition, the DRO contingency was suspended so that the former DRA stimuli were not presented (DRO-NAC), and resurgence was greater than in the Standard-DRA and DRO-DRA conditions. Reinstatement produced by response-independent reinforcers was small and similar across conditions. Subsequent reacquisition of target-key pecking under baseline reinforcement conditions was faster following DRO-NAC than Standard-DRA or DRO-DRA. These findings suggest that DRO-DRA might serve as a useful method in clinical settings for reducing problem behavior while minimizing the threat of posttreatment relapse.

Factors that influence the persistence and relapse of discriminated behavior chains

Behavior chains are composed of sequences of behaviors that minimally include procurement and then consumption. This review surveys recent research from this laboratory that has examined the properties of discriminated heterogeneous behavior chains. In contrast to another review (Thrailkill and Bouton, 2016a), it discusses work examining what makes chained behavior persistent, and what makes it relapse. Results suggest that responses in a discriminated heterogeneous behavior chain may become associated, so that extinction of either one reduces the strength of the other. Evidence also suggests that the goal of the first (procurement) response may be the next (consumption) response (rather than the upcoming discriminative stimulus, a putative conditioned reinforcer, or the primary reinforcer at the end of the chain). Further studies suggest that methods that promote generalization across acquisition and extinction (partial reinforcement and delivery of noncontingent reinforcers during extinction) lead to greater persistence of the procurement response. A third set of studies analyzed the contextual control and relapse of chained behaviors. The context controls both the acquisition and extinction of chained behaviors. In addition, a separately-extinguished consumption response is renewed when returned to the context of the chain. The research expands our general understanding of the learning processes that govern instrumental behavior as well as our understanding of chains.

Resurgence as Choice

Resurgence is typically defined as an increase in a previously extinguished target behavior when a more recently reinforced alternative behavior is later extinguished. Some treatments of the phenomenon have suggested that it might also extend to circumstances where either the historic or more recently reinforced behavior is reduced by other non-extinction related means (e.g., punishment, decreases in reinforcement rate, satiation, etc.). Here we present a theory of resurgence suggesting that the phenomenon results from the same basic processes governing choice. In its most general form, the theory suggests that resurgence results from changes in the allocation of target behavior driven by changes in the values of the target and alternative options across time. Specifically, resurgence occurs when there is an increase in the relative value of an historically effective target option as a result of a subsequent devaluation of a more recently effective alternative option. We develop a more specific quantitative model of how extinction of the target and alternative responses in a typical resurgence paradigm might produce such changes in relative value across time using a temporal weighting rule. The example model does a good job in accounting for the effects of reinforcement rate and related manipulations on resurgence in simple schedules where Behavioral Momentum Theory has failed. We also discuss how the general theory might be extended to other parameters of reinforcement (e.g., magnitude, quality), other means to suppress target or alternative behavior (e.g., satiation, punishment, differential reinforcement of other behavior), and other factors (e.g., non- contingent versus contingent alternative reinforcement, serial alternative reinforcement, and multiple schedules).

Coal Is Not Black, Snow Is Not White, Food Is Not a Reinforcer: The Roles of Affordances and Dispositions in the Analysis of Behavior

Reinforcers comprise sequences of actions in context. Just as the white of snow and black of coal depend on the interaction of an organism's visual system and the reflectances in its surrounds, reinforcers depend on an organism's motivational state and the affordances-possibilities for perception and action-in its surrounds. Reinforcers are not intrinsic to things but are a relation between what the thing affords, its context, the organism, and his or her history as capitulated in their current state. Reinforcers and other affordances are potentialities rather than intrinsic features. Realizing those potentialities requires motivational operations and stimulus contexts that change the state of the organism-they change its disposition to make the desired response. An expansion of the three-term contingency is suggested in order to help keep us mindful of the importance of behavioral systems, states, emotions, and dispositions in our research programs.

On defining resurgence

A review of different investigators' definitions of resurgence revealed several common features: First, characteristics of the resurgent, or target, response, such as its transience; magnitude; time course within and across sessions; and relativity to a baseline response rate are not mentioned. Second, the target response is described as being established through its reinforcement in the first, or Training, phase of a resurgence procedure. Third, the target response must be eliminated as an alternative response is reinforced in the second, Alternative Reinforcement, phase of a resurgence procedure. Fourth, the alternative response must be extinguished during the Resurgence Test phase. Fifth, none of the definitions allude to any contribution of stimulus variables to resurgence. When reconsidered in light of contemporary research germane to these features, none of the reviewed definitions sufficiently reflect important variables in the generation and assessment of resurgence. The review concludes with a proposed working definition that takes into account contemporary research involving all of the aforementioned factors.

Preventing relapse after incentivized choice treatment: A laboratory model

Two experiments with rats examined relapse of an operant behavior that occurred after the behavior was suppressed by reinforcing (incentivizing) an alternative behavior. In the first phase, a target response (R1) was reinforced. In a treatment phase, R1 was still reinforced, but a new response (R2) was introduced and associated with a larger reinforcer. As in human contingency management treatments, incentivizing R2 this way was effective at suppressing R1. However, when R2's reinforcement was discontinued, there was a robust and immediate relapse to R1. Experiment 1 found that the strength of R1 during relapse testing was not different from that seen in a no treatment control. Experiment 2 found that relapse could nevertheless be reduced by presenting reinforcers not contingent on responding during the test. Either the reinforcer for R1 or the reinforcer for R2 (which were qualitatively different types of food pellets) were effective. The experiments introduce a laboratory method for studying relapse and how to prevent it after contingency management treatments, and suggest at least one treatment that discourages relapse. The incentivized choice paradigm differs from other models of relapse of operant behavior (e.g., resurgence, renewal, reinstatement) in that it does not focus on the return of behaviors that are inhibited by extinction.

Contextual control of chained instrumental behaviors

Recent studies suggest a significant role for context in controlling the acquisition and extinction of simple operant responding. The present experiments examined the contextual control of a heterogeneous behavior chain. Rats first learned a chain in which a discriminative stimulus set the occasion for a procurement response (e.g., pulling a chain), which led to a second discriminative stimulus that occasion-set a consumption response (e.g., pressing a lever) that produced a food-pellet reinforcer. Experiment 1 showed that, after separate extinction of procurement and consumption, each response increased when it was returned to the acquisition context (ABA renewal) or was tested in a new context (AAB renewal). In addition, procurement responding, but not consumption responding, was decreased by changing the context after acquisition. Experiment 2 demonstrated ABA and AAB renewal of procurement and consumption following extinction of the whole chain. This time, the context-switch after acquisition weakened both procurement and consumption. Experiment 3 found that return to the context of the behavior chain renewed a consumption response that had been extinguished separately. Finally, in Experiment 4, rats learned 2 different discriminated heterogeneous chains; consumption extinguished outside its chain was only renewed on return to a chain when it was preceded by its associated procurement response. The results suggest a role for context in the extinction of chained behavior. They also support the view that procurement is influenced by the physical context and that consumption is controlled primarily by the response that precedes it in the chain. (PsycINFO Database Record

Extinction and the associative structure of heterogeneous instrumental chains

Drug abuse, overeating, and smoking are all examples of instrumental behaviors that often involve chains or sequences of behavior. A behavior chain is minimally composed of a procurement response that is required in order for a subsequent consumption response to be reinforced. Despite the translational importance of behavior chains, few studies have attempted to understand what binds them together and takes them apart. This article surveys the development of the heterogeneous instrumental chain method and introduces recent findings that have used extinction to analyze the associative content of (what is learned in) the chain. Chained responses that are occasion-set by their own discriminative stimuli may be directly associated; extinction of the procurement response weakens its associated consumption response, and extinction of the consumption response weakens its associated procurement response. Extinction itself involves learning to inhibit the response. Extinguished chained responses are subject to renewal when they are tested either back in the acquisition context or in a new context. In addition, a consumption response that is extinguished outside its chain is renewed when returned to the context of the preceding response in the chain. Research on heterogeneous behavior chains can provide important insights into an important but often overlooked aspect of instrumental learning.

Learning to inhibit the response during instrumental (operant) extinction

Five experiments tested implications of the idea that instrumental (operant) extinction involves learning to inhibit the learned response. All experiments used a discriminated operant procedure in which rats were reinforced for lever pressing or chain pulling in the presence of a discriminative stimulus (S), but not in its absence. In Experiment 1, extinction of the response (R) in the presence of S weakened responding in S, but equivalent nonreinforced exposure to S (without the opportunity to make R) did not. Experiment 2 replicated that result and found that extinction of R had no effect on a different R that had also been reinforced in the stimulus. In Experiments 3 and 4, rats first learned to perform several different stimulus and response combinations (S1R1, S2R1, S3R2, and S4R2). Extinction of a response in one stimulus (i.e., S1R1) transferred and weakened the same response, but not a different response, when it was tested in another stimulus (i.e., S2R1 but not S3R2). In Experiment 5, extinction still transferred between S1 and S2 when the stimuli set the occasion for R's association with different types of food pellets. The results confirm the importance of response inhibition in instrumental extinction: Nonreinforcement of the response in S causes the most effective suppression of responding, and response suppression is specific to the response but transfers and influences performance of the same response when it is occasioned by other stimuli. Theoretical and practical implications are discussed. (PsycINFO Database Record

Higher rate alternative non-drug reinforcement produces faster suppression of cocaine seeking but more resurgence when removed

Relapse following removal of an alternative source of reinforcement introduced during extinction of a target behavior is called resurgence. This form of relapse may be related to relapse of drug taking following loss of alternative non-drug reinforcement in human populations. Laboratory investigations of factors mediating resurgence with food-maintained behavior suggest higher rates of alternative reinforcement produce faster suppression of target behavior but paradoxically generate more relapse when alternative reinforcement is discontinued. At present, it is unknown if a similar effect occurs when target behavior is maintained by drug reinforcement and the alternative is a non-drug reinforcer. In the present experiment three groups of rats were trained to lever press for infusions of cocaine during baseline. Next, during treatment, cocaine reinforcement was suspended and an alternative response was reinforced with either high-rate, low-rate, or no alternative food reinforcement. Finally, all reinforcement was suspended to test for relapse of cocaine seeking. Higher rate alternative reinforcement produced faster elimination of cocaine seeking than lower rates or extinction alone, but when treatment was suspended resurgence of cocaine seeking occurred following only high-rate alternative reinforcement. Thus, although higher rate alternative reinforcement appears to more effectively suppress drug seeking, should it become unavailable, it can have the unfortunate effect of increasing relapse.