Effects of extinction in multiple contexts on renewal of instrumental responses

Infralimbic cortex plays a similar role in the punishment and extinction of instrumental behavior

Learning to stop responding is a fundamental process in instrumental learning. Animals may learn to stop responding under a variety of conditions that include punishment-where the response earns an aversive stimulus in addition to a reinforcer-and extinction-where a reinforced response now earns nothing at all. Recent research suggests that punishment and extinction may be related manifestations of a common retroactive interference process. In both paradigms, animals learn to stop performing a specific response in a specific context, suggesting direct inhibition of the response by the context. This process may depend on the infralimbic cortex (IL), which has been implicated in a variety of interference-based learning paradigms including extinction and habit learning. Despite the behavioral parallels between extinction and punishment, a corresponding role for IL in punishment has not been identified. Here we report that, in a simple arrangement where either punishment or extinction was conducted in a context that differed from the context in which the behavior was first acquired, IL inactivation reduced response suppression in the inhibitory context, but not responding when it "renewed" in the original context. In a more complex arrangement in which two responses were first trained in different contexts and then extinguished or punished in the opposite one, IL inactivation had no effect. The results advance our understanding of the effects of IL in retroactive interference and the behavioral mechanisms that can produce suppression of a response.

Extinction in multiple contexts reduces the return of extinguished responses: A multilevel meta-analysis

Extinguished responses have been shown to reappear under several circumstances, and this reappearance is considered to model behaviors such as relapse after exposure therapy. Conducting extinction in multiple contexts has been explored as a technique to decrease the recovery of extinguished responses. The present meta-analysis aimed to examine whether extinction in multiple contexts can consistently reduce the recovery of extinguished responses. After searching in several databases, experiments were included in the analysis if they presented extinction in multiple contexts, an experimental design, and an adequate statistical report. Cohen's d was obtained for each critical comparison and weighted to obtain the sample's average weighted effect size. Analyses were then performed using a multilevel meta-analytic approach. Twenty-five studies were included, with a total sample of 37 experiments or critical comparisons. The analyses showed a large effect size for the sample, moderated by the length of conditioned stimulus exposure, type of experimental subject, and type of recovery. The robust effect of extinction in multiple contexts on relapse should encourage clinicians to consider extinction in multiple contexts as a useful technique in therapy and research.

A comparison of renewal, spontaneous recovery, and reacquisition after punishment and extinction

Punishment and extinction are both effective methods of reducing instrumental responding and may involve similar learning mechanisms. To characterize the similarities and differences between them, we examined three well-established recovery or "relapse" effects -renewal, spontaneous recovery, and reacquisition - following either punishment or extinction of an instrumental response. In Experiment 1a, both punished and extinguished responses renewed to similar degrees following a context change at test (ABA renewal). In Experiment 1b, responding spontaneously recovered to similar degrees following punishment or extinction. In Experiment 2, responding was rapidly reacquired when the response was reinforced again following extinction but not following punishment, as predicted by the idea that the reinforcer delivered in reacquisition is part of the context of punishment, but not extinction. The results collectively suggest that both punishment and extinction produce similar context-dependent retroactive interference effects. More broadly, they also suggest that punished and extinguished responses may be equally likely to return following a change of context despite the intuition that punishment might provide a more extreme and effective means of suppressing behavior. To our knowledge, this is the first direct behavioral comparison of response recovery after punishment and extinction within individual experiments.

ABA, AAB and ABC renewal with Pavlovian Conditioning of Tentacle Lowering procedure in the snail Cornu aspersum

This study assesses the recovery of the conditioned response (CR) due to a contextual change (renewal effect) in the Cornu aspersum, using the appetitive Pavlovian Conditioning of Tentacle Lowering procedure. Snails experienced an odorous conditioned stimulus (CS) paired with food (conditioning), followed by the exposition to the CS without any consequence (extinction). Then, they were exposed to the CS in a different context from the extinction one (renewal test). The contexts were three types of illumination. In Experiment 1a, the conditioning was performed in context A, the extinction was conducted in context B and the renewal test was performed in context A. For Experiment 1b, the conditioning and extinction were conducted in context A and renewal was performed in context B. In Experiment 1c, three dissimilar contexts were used for each experimental phase: context A for the conditioning, context B for the extinction and context C for the renewal. In Experiment 2, the renewal magnitude was compared among the three paradigms (ABA, AAB and ABC). Experiments 1a, 1b and 1c showed a recovery of the CR when subjects experienced a contextual change and Experiment 2 showed equivalent levels of renewal in the three paradigms. Learning processes and theories involved are discussed.

Relapse and its mitigation: Toward behavioral inoculation

Relapse following the successful treatment of problem behavior can increase the likelihood of injury and the need for more intensive care. Current research offers some predictions of how treatment procedures may contribute to relapse, and conversely, how the risk of relapse can be mitigated. This review describes relapse-mitigation procedures with varying levels of support, the quantitative models that have influenced the research on relapse mitigation, different experimental methods for measuring relapse mitigation, and directions for future research. We propose that by viewing the implementation of relapse-mitigation procedures as a means of producing behavioral inoculation, clinicians are placed in the proactive and intentional role of exposing their client's behavior to an array of reinforcement and stimulus conditions during treatment with the goal of decreasing the detrimental impact of future treatment challenges.

Primacy and recency effects in hierarchical renewal in rats

Previous studies on hierarchical resurgence produced mixed results regarding the order and magnitude of recurrence of responses trained initially (primacy effect) or more recently (recency effect). Although changes in contextual stimuli could explain such differences, in resurgence procedures contextual stimuli are not commonly presented, thus their effects on multiple operants trained sequentially remain unclear. Renewal procedures, in contrast, have been useful to determine the effects of exteroceptive contextual stimuli on response recurrence. Thus, primacy and recency effects were studied using a renewal procedure in which three contexts were presented sequentially. Lever presses by rats were reinforced on a different lever under each training context and were then exposed to extinction in a different context. Presses on a fourth lever were never reinforced. During renewal testing, the three training contexts were presented in the same or inverse order relative to training. A strong primacy effect was found in rats exposed to the original training order. Both primacy and recency effects were found when the rats were exposed to contexts in inverse order. These results suggest that the magnitude of renewal of hierarchically trained responses is affected by training order and order of presentation of contextual stimuli during testing.

Evaluating effects of context changes on resurgence in humans

Laboratory models of relapse provide methods for evaluating challenges to behavioral treatments with differential reinforcement of an alternative response (DRA). Resurgence occurs with the worsening of conditions of reinforcement for appropriate behavior and renewal occurs when transitioning out of a treatment context. Across five experiments, participants recruited via online crowdsourcing pressed onscreen buttons to earn points exchangeable for money and contexts sometimes changed through changes in the background image. Returning to the training context (ABA, Experiment 1) and transitioning to a novel context (ABC, Experiment 2) produced greater resurgence when removing alternative reinforcement in comparison with remaining in the treatment context (ABB). In contrast, we observed little difference in resurgence among AAA, ABB, and AAC context manipulations (Experiment 3) and ABA, ABC, and AAC context manipulations (Experiment 4). In Experiment 5, we evaluated relative contributions of the presence versus absence of context changes (ABA vs. ABB) in combination with or without the removal of alternative reinforcement. Both changing context and removing alternative reinforcement increased responding in isolation and the combination produced greater-than-additive effects. Overall, the present findings demonstrate a consistent effect of removing alternative reinforcement on relapse that, under certain conditions, can be enhanced by context change.

Recovery-from-extinction effects in an anuran amphibian: renewal effect, but no reinstatement

Recovery-from-extinction effects in which a conditioned response returns after extinction have been shown in mammals, birds and fish. Thus, these effects appear to be conserved among vertebrates; however, they have yet to be investigated in amphibians. Using prey catching conditioning in the fire-bellied toad (Bombina orientalis), we tested if renewal and reinstatement occurred after extinction when subjects were respectively re-exposed to the context or reinforcer used during conditioning. For renewal, a different context was used during extinction and thus renewal tests assessed if external contextual cues associated during conditioning stimulated prey catching performance. For reinstatement, the reinforcer withheld during extinction was simply delivered again prior to a test assessing if internal cues associated with recent prey consumption stimulated prey catching performance. Conditioning followed a fixed ratio 5 schedule of reinforcement, where five attempts to capture a cricket stimulus displayed on a computer screen were reinforced by delivery of a single live cricket. Performance was measured as the time to reach five prey catching attempts. A significant improvement in prey catching performance during conditioning followed by deterioration with extinction was seen in the experiments. Upon return to the context used for conditioning after extinction, toads showed a renewal effect whereby they displayed faster performance during testing compared to the end of extinction. Conversely, toads showed no reinstatement effect because pre-feeding of a cricket did not influence performance during the test that followed extinction. Reinstatement could have been lost in amphibian phylogeny due to secondary simplification of the nervous system.

Effects of delay of reinforcement on ABA renewal

ABA renewal was assessed by decreasing response rates by means of extinction or a 12-s delay of reinforcement in one experiment with rats. In a first phase, rats were trained to lever press for water in Context A. Rats were exposed to a second phase in Context B, where lever-pressing decreased for all rats; for one group of rats, lever-pressing decreased by means of extinction, while lever-pressing decreased by means of a 12-s delay for the other group. During a third phase, both groups were once again exposed to Context A with no reinforcer deliveries for lever-pressing. Once rats were exposed to the latter phase, renewal was observed for both groups. Results suggest that delay of reinforcement can be used as an alternative procedure to help reduce behavior and mitigate relapse.

Emergence of complex dynamics of choice due to repeated exposures to extinction learning

Extinction learning, the process of ceasing an acquired behavior in response to altered reinforcement contingencies, is not only essential for survival in a changing environment, but also plays a fundamental role in the treatment of pathological behaviors. During therapy and other forms of training involving extinction, subjects are typically exposed to several sessions with a similar structure. The effects of this repeated exposure are not well understood. Here, we studied the behavior of pigeons across several sessions of a discrimination-learning task in context A, extinction in context B, and a return to context A to test the context-dependent return of the learned responses (ABA renewal). By focusing on individual learning curves across animals, we uncovered a session-dependent variability of behavior: (1) during extinction, pigeons preferred the unrewarded alternative choice in one-third of the sessions, predominantly during the first one. (2) In later sessions, abrupt transitions of behavior at the onset of context B emerged, and (3) the renewal effect decayed as sessions progressed. We show that the observed results can be parsimoniously accounted for by a computational model based only on associative learning between stimuli and actions. Our work thus demonstrates the critical importance of studying the trial-by-trial dynamics of learning in individual sessions, and the power of “simple” associative learning processes.

Positive affective states can play the role of context to renew extinguished instrumental behavior in rats

In one experiment with rats, we examined whether positive affective states can serve as contexts in a between-subjects ABA renewal design using appetitive instrumental conditioning. Two groups of rats received training to press a lever for food where each acquisition session was preceded by administration of a tickling procedure (Context A) known to induce positive affective states. Then, lever pressing underwent extinction where rats received a pure handling treatment (Context B) before each session. During a final test session, we found stronger responding when the session was preceded by tickling (Group ABA) compared to handling (Group ABB), indicating an ABA renewal effect. Furthermore, test performance in Group ABB was not different from that in a third group where handling preceded acquisition sessions, and tickling extinction and test sessions (Group BAA), showing that tickling did not elevate instrumental responding during the test if it had been unrelated to initial acquisition. We discuss implications of our results for understanding the role of positive affective states in relapse of problem behavior.

An evaluation of a renewal-mitigation procedure for inappropriate mealtime behavior

Renewal, the increase in behavior during extinction following context changes, may be particularly concerning during intervention for feeding disorders because context changes are often necessary for intervention generality and maintenance (Podlesnik et al., 2017). In the current study, we tested for renewal and evaluated a renewal-mitigation procedure when we transferred intervention from a therapist to a caregiver, from clinic to the home, and changed the foods the feeder presented. We used an ABA arrangement to evaluate the generality of the renewal effect with 7 participants who engaged in inappropriate mealtime behavior. Context A was functional reinforcement. Context B was function-based extinction during the control and mitigation conditions and our renewal-mitigation procedure in the mitigation condition. The renewal test was function-based extinction in Context A. We observed renewal of inappropriate mealtime behavior in 4 of 7 participants, and our renewal-mitigation procedure was effective for 4 of 4 participants.

Further evaluation of a nonsequential approach to studying operant renewal

Basic-laboratory assessments of renewal may inform clinical efforts to maintain reduction of severe destructive behavior when clients transition between contexts. The contextual changes arranged during standard renewal procedures, however, do not necessarily align with those that clients experience during outpatient therapy. More specifically, clients transition between clinical (associated with extinction for target behavior) and home/community (associated with reinforcement for target behavior) contexts during outpatient treatment. Standard renewal assessments do not incorporate these contextual alternations during treatment. The present experiment aimed to directly compare renewal of rats' lever pressing following a standard ("sequential") ABA renewal procedure (i.e., baseline in Context A, extinction in Context B, renewal test in Context A) and a "nonsequential" renewal assessment wherein treatment consisted of daily alternation between Context A (associated with reinforcement for lever pressing) and Context B (associated with extinction). Lever pressing renewed to a greater extent for rats in the Nonsequential group than for rats in the Sequential group, suggesting the contextual changes that clients experience during outpatient treatment for severe destructive behavior may be a variable that is important to consider in translational research on renewal. Potential implications of these findings for basic and clinical research on renewal are discussed.

Conducting extinction in multiple contexts attenuates relapse of operant behavior in humans

In one experiment with human participants, we investigated the effects of using multiple contexts during extinction on the renewal of operant responses. Undergraduate students played a videogame in which they learned to shoot at enemies in Context A. Then, all participants experienced an extinction training. For half of the participants, extinction trials were conducted in a single context, whereas the other half received extinction in three different contexts. Finally, all participants were tested in Context A. We observed that conducting extinction in multiple contexts attenuated ABA renewal. The present results suggest that conducting extinction in multiple contexts can be used as a behavioral technique to reduce operant renewal.

The impacts of acquisition and extinction cues on ABC renewal of voluntary behaviors

In two instrumental conditioning experiments with rats, we examined the impacts of acquisition and extinction cues on ABC renewal of instrumental behavior. Animals were reinforced with food for lever pressing in one context, followed by extinction of the response in a second one. Presentations of a brief tone accompanied extinction in Experiment 1 (extinction cue), and acquisition in Experiment 2 (acquisition cue). A final test in a third context revealed that instrumental responding was decreased in the presence of the extinction cue, whereas it was increased in the presence of the acquisition cue. We discuss theoretical and clinical implications of our results.

Investigations of operant ABA renewal during differential reinforcement

Operant renewal is a form of relapse in which a previously extinguished response recurs due to a change in context. We designed two experiments to examine the impact of differential reinforcement of alternative behavior on ABA renewal in a translational model of relapse with 12 children. We compared levels of renewal in two 3-phase arrangements. In one arrangement, we reinforced target responding in Context A, extinguished responding in Context B, and returned to Context A while continuing to implement extinction. In a second arrangement, an alternative response produced reinforcement in Context B and during the return to Context A. Results across the 2 experiments indicated 3 general findings. First, extinction plus differential reinforcement disrupted target behavior more consistently in Context B relative to extinction alone. Second, renewal tended to be greater and more persistent during extinction alone relative to extinction plus differential reinforcement. Third, the renewal effect appeared to depend on whether the alternative response had a history of extinction in Context A. We discuss methodological implications for the treatment of severe destructive behavior.

Increased Maternal Care Rescues Altered Reinstatement Responding Following Moderate Prenatal Alcohol Exposure

BACKGROUND

Fetal alcohol spectrum disorders (FASD) commonly include deficits in learning, memory, and executive control that can have a severe negative impact on quality of life across the life span. It is still unclear how prenatal alcohol exposure (PAE) affects executive control processes, such as control over reward seeking, that lead to inappropriate behavior later in life. Learning and reinstatement of a previously learned response after extinction is a simple, well-validated measure of both acquisition of a rewarded instrumental response and sensitivity to reward and reward-associated cues. We investigated the effects of PAE on learning, extinction, and reinstatement of a simple instrumental response for food reward. Next, we assessed the effectiveness of an early intervention, communal nest (CN) housing, on increased reinstatement of an extinguished response seen after PAE.

METHODS

To assess the effects of PAE on control over reward seeking, we tested male and female PAE and saccharine (SAC) controls raised in a standard nest (SN) on the acquisition, extinction, and food reward-induced reinstatement of an instrumental response utilizing a touch screen-based paradigm. Next, in order to examine the effects of an early-life intervention on these behaviors, we tested PAE and SAC mice raised in a CN early-life environment on these behaviors.

RESULTS

PAE mice readily acquired and extinguished a simple touch response to a white square stimulus. However, PAE mice showed significantly increased and persistent reinstatement compared to controls. Increased maternal care via rearing in CN slowed acquisition and sped extinction learning and rescued the significantly increased reinstatement responding in PAE mice.

CONCLUSIONS

Together these results demonstrate that even moderate PAE is sufficient to alter control over reward seeking as measured by reinstatement. Importantly, an early-life intervention previously shown to improve cognitive outcomes in PAE mice was sufficient to ameliorate this effect.