Effects of the amount of acquisition and contextual generalization on the renewal of instrumental behavior after extinction

Increased Generalization, Stronger Acquisition, or Reduced Extinction? Investigation of the Mechanisms Underlying the Acquisition-in-Multiple-Contexts Effect

Prior research has demonstrated that conducting acquisition in multiple contexts results in more responding to the point that it can even nullify the benefit of subsequent extinction in multiple contexts on reducing renewal of excitatory responding. The underlying mechanism to explain why this happens has not been systematically examined. Using self-reported expectancy of the outcome, the current study investigates three mechanisms that potentially explain why acquisition in multiple contexts results in more responding-greater generalization, stronger acquisition learning, or slower extinction learning. Participants (N = 180) received discriminative training with a conditioned stimulus (CS+) and outcome pairing and a CS- → noOutcome pairing in either one or three contexts. This was followed by either extinction treatment in a novel context or no extinction. Finally, testing occurred in the acquisition context, the extinction context, or a novel context. Stronger renewal of extinguished conditioned expectation was observed for participants who received CS+ → Outcome pairings in three contexts relative to one context. There was no effect of the number of contexts on the strength of the excitatory CS+ → Outcome association or degree of inhibitory learning that occurred during extinction. This suggests that generalization is the mechanism responsible for the adverse impact to extinction learning when acquisition is conducted in multiple contexts.

Perceived stress and renewal: The effects of long-term stress on the renewal effect

Two online experiments evaluated the relationship between long-term stress, as measured with the Perceived Stress Scale-10, and the Renewal Effect. In the first experiment renewal was assessed with a behavioral suppression task in a science-fiction based video game. Participants learned to suppress mouse clicking during a signal for an upcoming attack to avoid losing points. The signal was first paired with an attack in Context A and extinguished in Context B and tested back in Context A. The contexts were different space galaxies where the gameplay took place. Experiment 2 used a food/illness predictive-learning paradigm. Two food items were paired with stomachache in one restaurant (A) and extinguished in Context B prior to testing in both contexts without feedback. Positive correlations were obtained between renewal and stress in each experiment. Unlike acute stress (Drexler et al., 2017), long term stress was associated with greater renewal. The effects of stress, both chronic and punctual, on renewal are discussed.

Habit and persistence

Voluntary behaviors (operants) can come in two varieties: Goal-directed actions, which are emitted based on the remembered value of the reinforcer, and habits, which are evoked by antecedent cues and performed without the reinforcer's value in active memory. The two are perhaps most clearly distinguished with the reinforcer-devaluation test: Goal-directed actions are suppressed when the reinforcer is separately devalued and responding is tested in extinction, and habitual behaviors are not. But what is the function of habit learning? Habits are often thought to be strong and unusually persistent. The present selective review examines this idea by asking whether habits identified by the reinforcer-devaluation test are more resistant to extinction, resistant to the effects of other contingency change, vulnerable to relapse, resistant to the weakening effects of context change, or permanently in place once they are learned. Surprisingly little evidence supports the idea that habits are permanent or more persistent. Habits are more context-specific than goal-directed actions are. Methods that make behavior persistent do not necessarily work by encouraging habit. The function of habit learning may not be to make a behavior strong or more persistent but to make it automatic and efficient in a particular context.

Estrous cycle state-dependent renewal of appetitive behavior recruits unique patterns of Arc mRNA in female rats

Introduction

Renewal is a behavioral phenomenon wherein extinction learning fails to generalize between different contextual environments, thereby representing a significant challenge to extinction-based rehabilitative therapies. Previously, we have shown that renewal of extinguished appetitive behavior differs across the estrous cycle of the female rat. In this experiment that effect is replicated and extended upon to understand how the estrous cycle may modulate contextual representation at the neuronal population level to drive renewal.

Methods

Estrous cycle stage [i.e., proestrus (P, high hormone) or metestrus/diestrus (M/D, low hormone)] was considered during two important learning and behavioral expression windows: at extinction training and during long-term memory (LTM)/renewal testing. Cellular compartment analysis of temporal activity using fluorescence in situ hybridization (catFISH) for Arc mRNA was conducted after the distinct context-stimulus exposures.

Results

Rats in P during context-dependent extinction training but in a different stage of the estrous cycle during LTM and renewal testing (P-different) were shown to exhibit more renewal of conditioned foodcup (but not conditioned orienting) behavior compared to rats in other estrous cycle groups. Importantly, we discovered this depends on the order of tests. P-different rats showed differential Arc mRNA expression in regions of the prefrontal cortex (PFC), amygdala, and hippocampus (HPC). For each case P-different rats had more co-expression (i.e., expression of both nuclear and cytoplasmic) of Arc mRNA compared to other groups; specific to the dorsal HPC, P-different rats also had a more robust Arc mRNA response to the extinction context exposure.

Conclusion

These data suggest female rats show estrous cycle state-dependent renewal of appetitive behavior, and differences in context and conditioned stimulus representation at the neuronal level may drive this effect.

The impact of prolonging extinction on the ABC "super renewal" of instrumental responses in rats

Two free operant conditioning experiments with rats examined the impact of conducting a large amount of extinction training on situations that enhance the ABC renewal effect (ABC super renewal). In Experiment 1, ABC renewal was strengthened by conducting acquisition in multiple contexts. All rats were trained to press a lever for food. One group was trained in one context, while the other two groups were trained in three contexts. Then, all rats received extinction in context B. For two groups this phase lasted 4 sessions, whereas it lasted 36 sessions for the other group. In Experiment 2, ABC renewal was strengthened by using a large number of acquisition sessions. Rats were trained to perform an operant response to obtain food in context A. One group received a moderate amount of training, while the rest of the rats received a larger number of acquisition sessions. Responses underwent extinction in context B. Two groups received 4 sessions, while 36 extinction sessions were used for the remaining group. In both experiments, rats were tested in context B (extinction context) and C (renewal context). Greater ABC renewal occurred both when acquisition training was conducted in multiple contexts (Experiment 1) and by increasing the amount of acquisition training (Experiment 2). Nevertheless, we found that conducting a large number of extinction sessions reduced ABC super renewal in Experiment 1 only.

ABA, AAB and ABC renewal with Pavlovian Conditioning of Tentacle Lowering procedure in the snail Cornu aspersum

This study assesses the recovery of the conditioned response (CR) due to a contextual change (renewal effect) in the Cornu aspersum, using the appetitive Pavlovian Conditioning of Tentacle Lowering procedure. Snails experienced an odorous conditioned stimulus (CS) paired with food (conditioning), followed by the exposition to the CS without any consequence (extinction). Then, they were exposed to the CS in a different context from the extinction one (renewal test). The contexts were three types of illumination. In Experiment 1a, the conditioning was performed in context A, the extinction was conducted in context B and the renewal test was performed in context A. For Experiment 1b, the conditioning and extinction were conducted in context A and renewal was performed in context B. In Experiment 1c, three dissimilar contexts were used for each experimental phase: context A for the conditioning, context B for the extinction and context C for the renewal. In Experiment 2, the renewal magnitude was compared among the three paradigms (ABA, AAB and ABC). Experiments 1a, 1b and 1c showed a recovery of the CR when subjects experienced a contextual change and Experiment 2 showed equivalent levels of renewal in the three paradigms. Learning processes and theories involved are discussed.

Transcranial direct current stimulation impairs updating of avoidance-based associative learning

Introduction

Exposure-based psychotherapies for the treatment of anxiety- and fear-based disorders rely on "corrective" associative learning. Namely the repeated confrontation with feared stimuli in the absence of negative outcomes allows the formation of new, corrected associations of safety, indicating that such stimuli no longer need to be avoided. Unfortunately, exposure-facilitated corrective learning tends to be bound by context and often poorly generalizes. One brain structure, the prefrontal cortex, is implicated in context-guided behavior and may be a relevant target for improving generalization of safety learning. Here, we tested whether inhibition of the left prefrontal cortex causally impaired updating of context-bound associations specifically or, alternatively, impaired updating of learned associations irrespective of contextual changes. Additionally, we tested whether prefrontal inhibition during corrective learning influenced subsequent generalization of associations to a novel context.

Methods

In two separate experiments, participants received either 10 min of 2 mA cathodal transcranial direct current stimulation (tDCS) over EEG coordinate F3 (Experiment 1 n = 9, Experiment 2 n = 22) or sham stimulation (Experiment 1 n = 10, Experiment 2 n = 22) while previously learned associations were reversed in the same or a different context from initial learning. Next, to assess generalization of learning, participants were asked to indicate which of the previously seen images they preferred in a novel, never seen before context.

Results

Results indicate that tDCS significantly impaired reversal irrespective of context in Experiment 2 only. When taking learning rate across trials into account, both experiments suggest that participants who received sham had the greatest learning rate when reversal occurred in a different context, as expected, whereas participants who received active tDCS in this condition had the lowest learning rate. However, active tDCS was associated with preferring the originally disadvantageous, but then neural stimulus after stimulus after reversal occurred in a different context in Experiment 1 only.

Discussion

These results support a causal role for the left prefrontal cortex in the updating of avoidance-based associations and encourage further inquiry investigating the use of non-invasive brain stimulation on flexible updating of learned associations.

Relapse and its mitigation: Toward behavioral inoculation

Relapse following the successful treatment of problem behavior can increase the likelihood of injury and the need for more intensive care. Current research offers some predictions of how treatment procedures may contribute to relapse, and conversely, how the risk of relapse can be mitigated. This review describes relapse-mitigation procedures with varying levels of support, the quantitative models that have influenced the research on relapse mitigation, different experimental methods for measuring relapse mitigation, and directions for future research. We propose that by viewing the implementation of relapse-mitigation procedures as a means of producing behavioral inoculation, clinicians are placed in the proactive and intentional role of exposing their client's behavior to an array of reinforcement and stimulus conditions during treatment with the goal of decreasing the detrimental impact of future treatment challenges.

Operant ABA renewal during dense and lean schedules of differential reinforcement

Renewal is a type of relapse that occurs due to a change in context. Previous research has demonstrated that renewal of target responding may occur despite the availability of differential reinforcement for an alternative response (DRA). Nevertheless, the current literature on renewal presents mixed findings regarding the effects of dense and lean schedules of DRA on the magnitude of renewal. We used a translational approach with undergraduate college students and a task on a touchscreen tablet device to study the effects of dense and lean schedules of DRA during repeated renewal tests. All participants experienced two, three-phase ABA renewal arrangements. In the dense and lean renewal arrangements, we differentially reinforced alternative behavior in Context B and the renewal test in Context A on a VI 3-s or a VI 12-s schedule, respectively. Overall, we observed renewal in 31/36 (86%) renewal tests regardless of the density of reinforcement for the alternative response. Furthermore, the results showed that although renewal occurred in both arrangements, we found slightly higher magnitudes of renewal during DRA with lean schedules of reinforcement relative to dense schedules. We discuss the implications of these findings as they relate to the treatment of problem behavior.

Optogenetic stimulation of infralimbic cortex projections to the paraventricular thalamus attenuates context-induced renewal

Contexts associated with prior reinforcement can renew extinguished conditioned responding. The prelimbic (PL) and infralimbic (IL) cortices are thought to mediate the expression and suppression of conditioned responding, respectively. Evidence suggests that PL inputs to the paraventricular nucleus of the thalamus (PVT) drive the expression of cue-induced reinstatement of drug seeking and that IL inputs to the PVT mediate fear extinction retrieval. However, the role of these projections in renewal of appetitive Pavlovian conditioned responding is unknown. We trained male and female Long-Evans rats to associate a conditioned stimulus (CS; 10 s white noise) with delivery of a 10% sucrose unconditioned stimulus (US; .2 ml/CS) to a fluid port in a distinct context (Context A). We then extinguished responding by presenting the CS without the US in a different context (Context B). At test, rats were returned to Context A, and optogenetic stimulation was delivered to either the IL-to-PVT or PL-to-PVT pathway during CS presentations. Optically stimulating the IL-to-PVT, but not the PL-to-PVT pathway, attenuated ABA renewal of CS port entries, and this effect was similar in males and females. Further, rats self-administered optical stimulation of the IL-to-PVT but not the PL-to-PVT pathway suggesting that activation of the IL-to-PVT pathway is reinforcing. The effectiveness of optical stimulation parameters to activate neurons in the IL, PL and PVT was confirmed using Fos immunohistochemistry. These findings provide evidence for novel neural mechanisms in renewal of responding to a sucrose-predictive CS, as well as more generally in contextual processing and appetitive associative learning.

Estradiol and progesterone in female reward-learning, addiction, and therapeutic interventions

Sex steroid hormones like estradiol (E2) and progesterone (P4) guide the sexual organization and activation of the developing brain and control female reproductive behavior throughout the lifecycle; importantly, these hormones modulate functional activity of not just the endocrine system, but most of the nervous system including the brain reward system. The effects of E2 and P4 can be seen in the processing of and memory for rewarding stimuli and in the development of compulsive reward-seeking behaviors like those seen in substance use disorders. Women are at increased risk of developing substance use disorders; however, the origins of this sex difference are not well understood and therapeutic interventions targeting ovarian hormones have produced conflicting results. This article reviews the contribution of the E2 and P4 in females to functional modulation of the brain reward system, their possible roles in origins of addiction vulnerability, and the development and treatment of compulsive reward-seeking behaviors.

Primacy and recency effects in hierarchical renewal in rats

Previous studies on hierarchical resurgence produced mixed results regarding the order and magnitude of recurrence of responses trained initially (primacy effect) or more recently (recency effect). Although changes in contextual stimuli could explain such differences, in resurgence procedures contextual stimuli are not commonly presented, thus their effects on multiple operants trained sequentially remain unclear. Renewal procedures, in contrast, have been useful to determine the effects of exteroceptive contextual stimuli on response recurrence. Thus, primacy and recency effects were studied using a renewal procedure in which three contexts were presented sequentially. Lever presses by rats were reinforced on a different lever under each training context and were then exposed to extinction in a different context. Presses on a fourth lever were never reinforced. During renewal testing, the three training contexts were presented in the same or inverse order relative to training. A strong primacy effect was found in rats exposed to the original training order. Both primacy and recency effects were found when the rats were exposed to contexts in inverse order. These results suggest that the magnitude of renewal of hierarchically trained responses is affected by training order and order of presentation of contextual stimuli during testing.

Choosing a future from a murky past: A generalization-based model of behavior

Remembering the past appears critical in allowing organisms to detect order in an environment, and hence to behave in accordance with likely future events. Yet the shortcomings of remembering and perceiving typically mean that the remembered past differs from the actual past, and hence that behavior does not perfectly track the structure of the environment. Here, we outline how the process of generalization might be used to understand differences between what an organism does, and the structure of the past and potential structure of the environment. We explore how different sources of generalization - both from within the same stimulus situation, and from different stimulus situations - might be modeled quantitatively, and how predictions made by this modeling approach are supported by research. Finally, we discuss how generalization from multiple stimulus situations, longer-term experience, and from stimulus situations in the past that are not identical to the stimulus situation in the present, might contribute to our understanding of how an organism's experience translates into behavior.

Evaluating effects of context changes on resurgence in humans

Laboratory models of relapse provide methods for evaluating challenges to behavioral treatments with differential reinforcement of an alternative response (DRA). Resurgence occurs with the worsening of conditions of reinforcement for appropriate behavior and renewal occurs when transitioning out of a treatment context. Across five experiments, participants recruited via online crowdsourcing pressed onscreen buttons to earn points exchangeable for money and contexts sometimes changed through changes in the background image. Returning to the training context (ABA, Experiment 1) and transitioning to a novel context (ABC, Experiment 2) produced greater resurgence when removing alternative reinforcement in comparison with remaining in the treatment context (ABB). In contrast, we observed little difference in resurgence among AAA, ABB, and AAC context manipulations (Experiment 3) and ABA, ABC, and AAC context manipulations (Experiment 4). In Experiment 5, we evaluated relative contributions of the presence versus absence of context changes (ABA vs. ABB) in combination with or without the removal of alternative reinforcement. Both changing context and removing alternative reinforcement increased responding in isolation and the combination produced greater-than-additive effects. Overall, the present findings demonstrate a consistent effect of removing alternative reinforcement on relapse that, under certain conditions, can be enhanced by context change.

Prevalence of relapse of automatically maintained behavior resulting from context changes

Increases in behavior due to context changes are common and are known as instances of renewal. Clinically relevant examples from the literature highlighting renewal often include socially mediated problem behaviors. This report retrospectively analyzed data during context changes for individuals who engaged in problem behavior maintained by automatic reinforcement, to evaluate the prevalence of relapse. Problem behavior reemerged during changes both in the person implementing treatment (e.g., introducing a caregiver; 36%) and in the setting (e.g., introducing treatment in the home; 26%). Most prevalence studies report greatest relapse immediately following context changes but the highest level of relapse was observed after 5 sessions following person changes and no systematic pattern with setting changes. These patterns of relapse likely reflect differences in the function of settings and people relative to automatically reinforced behavior in the present study. Implications of relapse for treatments of problem behavior maintained by automatic reinforcement are discussed.

New functions of the rodent prelimbic and infralimbic cortex in instrumental behavior

The prelimbic and infralimbic cortices of the rodent medial prefrontal cortex mediate the effects of context and goals on instrumental behavior. Recent work from our laboratory has expanded this understanding. Results have shown that the prelimbic cortex is important for the modulation of instrumental behavior by the context in which the behavior is learned (but not other contexts), with context potentially being broadly defined (to include at least previous behaviors). We have also shown that the infralimbic cortex is important in the expression of extensively-trained instrumental behavior, regardless of whether that behavior is expressed as a stimulus-response habit or a goal-directed action. Some of the most recent data suggest that infralimbic cortex may control the currently active behavioral state (e.g., habit vs. action or acquisition vs. extinction) when two states have been learned. We have also begun to examine prelimbic and infralimbic cortex function as key nodes of discrete circuits and have shown that prelimbic cortex projections to an anterior region of the dorsomedial striatum are important for expression of minimally-trained instrumental behavior. Overall, the use of an associative learning perspective on instrumental learning has allowed the research to provide new perspectives on how these two "cognitive" brain regions contribute to instrumental behavior.

Pre-treatment hippocampal functioning impacts context renewal for cholinergic modulated exposure therapy

Our recent trial demonstrated individuals suffering from social anxiety with performance-related concerns who received virtual reality exposure augmented with scopolamine, a cholinergic antagonist, experienced significantly less post-treatment context renewal (CX) than placebo (Craske et al., 2019). The purpose of the present investigation was to determine who specifically benefits from scopolamine by examining hippocampal (HPC) functioning as a moderator of treatment response (Placebo n = 15, SCOP 0.5 mg n = 15, SCOP 0.6 mg n = 15). Skin conductance response to conditional stimulus (SCR-to-CS) termination suggested a dose-response relationship for enhanced HPC functioning individuals, wherein individuals receiving scopolamine demonstrated less fear at CX. In addition, SCR-to-CS onset indicated reduced fear at CX for impaired HPC individuals receiving SCOP 0.5 mg and SCOP 0.6 mg relative to Placebo. Our findings, however, lacked consistency across measures. Scopolamine remains a promising agent and additional research required to further understand its effects.

Evaluating Extinction, Renewal, and Resurgence of Operant Behavior in Humans with Amazon Mechanical Turk

Amazon Mechanical Turk (MTurk) is a crowdsourcing marketplace providing researchers with the opportunity to collect behavioral data from remote participants at a low cost. Recent research demonstrated reliable extinction effects, as well as renewal and resurgence of button pressing with MTurk participants. To further examine the generality of these findings, we replicated and extended these methods across six experiments arranging reinforcement and extinction of a target button press. In contrast to previous findings, we did not observe as reliable of decreases in button pressing during extinction (1) after training with VR or VI schedules of reinforcement, (2) in the presence or absence of context changes, or (3) with an added response cost for button pressing. However, we found that that a 1-point response cost for all button presses facilitated extinction to a greater extent than the absence of response cost. Nevertheless, we observed ABA renewal of button pressing when changing background contexts across phases and resurgence when extinguishing presses on an alternative button. Our findings suggest that MTurk could be a viable platform from which to ask and address questions about extinction and relapse processes, but further procedural refinements will be necessary to improve the replicability of control by experimental contingencies.

Positive affective states can play the role of context to renew extinguished instrumental behavior in rats

In one experiment with rats, we examined whether positive affective states can serve as contexts in a between-subjects ABA renewal design using appetitive instrumental conditioning. Two groups of rats received training to press a lever for food where each acquisition session was preceded by administration of a tickling procedure (Context A) known to induce positive affective states. Then, lever pressing underwent extinction where rats received a pure handling treatment (Context B) before each session. During a final test session, we found stronger responding when the session was preceded by tickling (Group ABA) compared to handling (Group ABB), indicating an ABA renewal effect. Furthermore, test performance in Group ABB was not different from that in a third group where handling preceded acquisition sessions, and tickling extinction and test sessions (Group BAA), showing that tickling did not elevate instrumental responding during the test if it had been unrelated to initial acquisition. We discuss implications of our results for understanding the role of positive affective states in relapse of problem behavior.

Associative processes in addiction relapse models: A review of their Pavlovian and instrumental mechanisms, history, and terminology

Animal models of relapse to drug-seeking have borrowed heavily from associative learning approaches. In studies of relapse-like behaviour, animals learn to self-administer drugs then receive a period of extinction during which they learn to inhibit the operant response. Several triggers can produce a recovery of responding which form the basis of a variety of models. These include the passage of time (spontaneous recovery), drug availability (rapid reacquisition), extinction of an alternative response (resurgence), context change (renewal), drug priming, stress, and cues (reinstatement). In most cases, the behavioural processes driving extinction and recovery in operant drug self-administration studies are similar to those in the Pavlovian and behavioural literature, such as context effects. However, reinstatement in addiction studies have several differences with Pavlovian reinstatement, which have emerged over several decades, in experimental procedures, associative mechanisms, and terminology. Interestingly, in cue-induced reinstatement, drug-paired cues that are present during acquisition are omitted during lever extinction. The unextinguished drug-paired cue may limit the model’s translational relevance to cue exposure therapy and renders its underlying associative mechanisms ambiguous. We review major behavioural theories that explain recovery phenomena, with a particular focus on cue-induced reinstatement because it is a widely used model in addiction. We argue that cue-induced reinstatement may be explained by a combination of behavioural processes, including reacquisition of conditioned reinforcement and Pavlovian to Instrumental Transfer. While there are important differences between addiction studies and the behavioural literature in terminology and procedures, it is clear that understanding associative learning processes is essential for studying relapse.

An evaluation of a renewal-mitigation procedure for inappropriate mealtime behavior

Renewal, the increase in behavior during extinction following context changes, may be particularly concerning during intervention for feeding disorders because context changes are often necessary for intervention generality and maintenance (Podlesnik et al., 2017). In the current study, we tested for renewal and evaluated a renewal-mitigation procedure when we transferred intervention from a therapist to a caregiver, from clinic to the home, and changed the foods the feeder presented. We used an ABA arrangement to evaluate the generality of the renewal effect with 7 participants who engaged in inappropriate mealtime behavior. Context A was functional reinforcement. Context B was function-based extinction during the control and mitigation conditions and our renewal-mitigation procedure in the mitigation condition. The renewal test was function-based extinction in Context A. We observed renewal of inappropriate mealtime behavior in 4 of 7 participants, and our renewal-mitigation procedure was effective for 4 of 4 participants.

Renewal of goal direction with a context change after habit learning

An instrumental action can be goal-directed after a moderate amount of practice and then convert to habit after more extensive practice. Recent evidence suggests, however, that habits can return to action status after different environmental manipulations. The present experiments therefore asked whether habit learning interferes with goal direction in a context-dependent manner like other types of retroactive interference (e.g., extinction, punishment, counterconditioning). In Experiment 1, rats were given a moderate amount of instrumental training to form an action in one context (Context A) and then more extended training of the same response to form a habit in another context (Context B). We then performed reinforcer devaluation with taste aversion conditioning in both contexts, and tested the response in both contexts. The response remained habitual in Context B, but was goal-directed in Context A, indicating renewal of goal direction after habit learning. Experiment 2 expanded on Experiment 1 by testing the response in a third context (Context C). It found that the habitual response also renewed as action in this context. Together, the results establish a parallel between habit and extinction learning: Conversion to habit does not destroy action knowledge, but interferes with it in a context-specific way. They are also consistent with other results suggesting that habit is specific to the context in which it is learned, whereas goal-direction can transfer between contexts. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Recaída de Variabilidade Operante com a Combinação dos Procedimentos de Renovação, Restabelecimento e Ressurgência

Resumo O reaparecimento de variabilidade comportamental previamente extinta (i.e., recaída) foi investigado por meio dos modelos experimentais de renovação, restabelecimento e ressurgência. Na Fase de Treino (contexto A), ao serem expostos ao esquema múltiplo Lag 10 Acoplado, ratos apresentaram níveis similares de variabilidade nos dois componentes. Na Fase de Eliminação (contexto B), o esquema múltiplo Repetição Repetição promoveu a extinção da variabilidade. Na Fase de Teste (contexto A), com a suspensão da contingência de repetição e a liberação de reforços independentes, a variabilidade reapareceu no componente “Lag 10”, anteriormente correlacionado com reforçamento da variação, mas não no componente “Acoplado”. Esse resultado sugere que a variabilidade observada no teste de recaída corresponde à variabilidade operante, e não à variabilidade induzida pela extinção.

Further evaluation of a nonsequential approach to studying operant renewal

Basic-laboratory assessments of renewal may inform clinical efforts to maintain reduction of severe destructive behavior when clients transition between contexts. The contextual changes arranged during standard renewal procedures, however, do not necessarily align with those that clients experience during outpatient therapy. More specifically, clients transition between clinical (associated with extinction for target behavior) and home/community (associated with reinforcement for target behavior) contexts during outpatient treatment. Standard renewal assessments do not incorporate these contextual alternations during treatment. The present experiment aimed to directly compare renewal of rats' lever pressing following a standard ("sequential") ABA renewal procedure (i.e., baseline in Context A, extinction in Context B, renewal test in Context A) and a "nonsequential" renewal assessment wherein treatment consisted of daily alternation between Context A (associated with reinforcement for lever pressing) and Context B (associated with extinction). Lever pressing renewed to a greater extent for rats in the Nonsequential group than for rats in the Sequential group, suggesting the contextual changes that clients experience during outpatient treatment for severe destructive behavior may be a variable that is important to consider in translational research on renewal. Potential implications of these findings for basic and clinical research on renewal are discussed.

Context and renewal of habits and goal-directed actions after extinction

Instrumental behaviors that are goal-directed actions after moderate amounts of training can become habits after more extended training. Little research has asked how actions and habits are affected by retroactive interference treatments like extinction. The present experiments begin to fill this gap in the literature. In Experiments 1a and 1b, lever pressing in rats was minimally trained (1a) or extensively trained (1b) in one context (Context A), extinguished in a second context (Context B), and then tested in the acquisition context (Context A). Exposure to both contexts was equated and controlled throughout, and the status of the behavior as action or habit was determined by reinforcer devaluation methods (taste aversion conditioning). Results confirmed that action (1a) and habit (1b) renewed with action or habit status, respectively, when they were returned to Context A. Experiments 2a and 2b then similarly tested action and habit after extinction in an ABC renewal paradigm. Here, lever pressing that was trained in Context A and extinguished in Context B renewed as action in Context C regardless of whether it had been an action or habit before extinction. The apparent conversion of habit to action during renewal testing in Context C was consistent with other results suggesting that habits converted to action when the context was changed at the start of extinction. Together, the results suggest that extinction in a second context inhibits instrumental behaviors trained as either actions or habits in a context-specific manner. They also expand on prior findings suggesting that actions transfer across contexts, and that habits do not. A change of context may be sufficient to convert a habit to goal-directed action. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Comparing ABA, AAB, and ABC Renewal of Appetitive Pavlovian Conditioned Responding in Alcohol- and Sucrose-Trained Male Rats

Conditioned responding can be renewed by re-exposure to the conditioning context following extinction in a different context (ABA renewal) or by removal from the extinction context (AAB or ABC renewal). ABA renewal is robust in Pavlovian and operant conditioning paradigms. However, fewer studies have investigated AAB and ABC renewal of appetitive conditioning, and those that did predominantly used operant conditioning tasks. Renewal has theoretical relevance for extinction and for exposure-based treatments for substance use disorders that aim to extinguish reactivity to drug-predictive cues. We therefore investigated ABA, AAB, and ABC renewal of Pavlovian conditioned responding to cues that predicted either alcohol or sucrose. Male, Long-Evans rats (Charles River) were exposed to either 15% ethanol (Study 1: “alcohol”) or 10% sucrose (Study 2: “sucrose”) in their home cages. Next, they were trained to discriminate between two auditory stimuli (white noise and clicker; 10 s) in conditioning chambers equipped with distinct olfactory, visual, and tactile contextual stimuli (context A). One conditioned stimulus (CS+) was paired with fluid delivery (0.2 ml/CS+; 3.2 ml/session; alcohol or sucrose in separate experiments), and the second CS (CS−) was not. In all sessions (conditioning, extinction, and test), each CS was presented 16 times/session on a variable-time 67-s schedule, and entries into the fluid port were recorded. CS+ port entries were then extinguished by withholding fluid delivery either in context A or in a second, different context (context B). Next, we assessed ABA, AAB, and ABC renewal in the absence of fluid delivery. During extinction, CS+ port entries were initially elevated in context A relative to context B. ABA renewal of CS+ port entries occurred in both alcohol- and sucrose-trained rats. ABC renewal approached statistical significance when data from both experiments were combined. No AAB renewal was observed, and, in fact, alcohol-trained rats showed AAB suppression. These results corroborate the reliability of ABA renewal and suggest that ABC renewal is a modest effect that may require greater statistical power to detect. From a treatment perspective, the lack of AAB renewal suggests that exposure-based treatments for substance use disorders might benefit from implementation in real-world, drug-use contexts.

Investigations of operant ABA renewal during differential reinforcement

Operant renewal is a form of relapse in which a previously extinguished response recurs due to a change in context. We designed two experiments to examine the impact of differential reinforcement of alternative behavior on ABA renewal in a translational model of relapse with 12 children. We compared levels of renewal in two 3-phase arrangements. In one arrangement, we reinforced target responding in Context A, extinguished responding in Context B, and returned to Context A while continuing to implement extinction. In a second arrangement, an alternative response produced reinforcement in Context B and during the return to Context A. Results across the 2 experiments indicated 3 general findings. First, extinction plus differential reinforcement disrupted target behavior more consistently in Context B relative to extinction alone. Second, renewal tended to be greater and more persistent during extinction alone relative to extinction plus differential reinforcement. Third, the renewal effect appeared to depend on whether the alternative response had a history of extinction in Context A. We discuss methodological implications for the treatment of severe destructive behavior.

Operant renewal of desirable behavior in a simulated workplace: A translational model

Organizational settings are replete with changing stimulus contexts and contingencies, which makes relapse a particularly relevant framework for understanding the ways in which controlling stimuli influence employee responding. The purpose of the current study was to develop a translational model to assess renewal of desirable behavior in a simulated workplace with neurotypical adults. Experiment 1 assessed renewal of desirable behavior using a computerized check processing task. Experiment 2 extended the findings and the translational utility of the experimental arrangement to implementation of a behavior-analytic teaching procedure. Results across both experiments demonstrated renewal of desirable behavior. Overall, the current methodology and findings extend the human operant literature on renewal and demonstrate a translational model that brings together operant renewal and organizational behavior management.

Re-exposure to reinforcement in context a during treatment in context b reduces ABC renewal

Previous work from our laboratory showed that intermittently re-exposing rats to reinforcement for lever pressing in a training (A) context, while eliminating lever pressing in a second (B) context, increased ABA renewal of lever pressing relative to rats that experienced only Context B during response elimination. In the current study, we replicated these procedures while assessing renewal in the presence of a novel context (i.e., ABC renewal). Unlike the findings described above, renewal was reduced in the group that experienced re-exposure to Context A during lever-press elimination relative to rats that experienced only Context B. These findings suggest that alternating between contexts associated with reinforcement and extinction during treatment reduces the probability that organisms will respond in novel contexts. These outcomes may be the result of discrimination and/or generalization processes. Moreover, this training procedure may offer a potential mitigation strategy for ABC renewal.

Vagus nerve stimulation during extinction learning reduces conditioned place preference and context-induced reinstatement of cocaine seeking

BACKGROUND

Drug use causes the formation of strong cue/reward associations which persist long after cessation of drug-taking and contribute to the long-term risk of relapse. Extinguishing these associations may reduce cue-induced craving and relapse. Previously, we found that pairing vagus nerve stimulation (VNS) with extinction of cocaine self-administration reduces cue-induced reinstatement; however, it remains unclear whether this was primarily caused by extinguishing the context, the instrumental response, or both.

OBJECTIVE

Hypothesis: We hypothesized that VNS can facilitate the extinction of both contextual cues and instrumental responding.

METHODS

Extinction of context was first tested using Pavlovian conditioned place preference (CPP). Next, the impact of VNS on the extinction of instrumental responding was assessed under ABA and AAA context conditions. In each extinction context separate groups of rats were either provided the opportunity to perform the instrumental response, or the levers were retracted for the duration of extinction training. Reinstatement was induced by reintroduction of the conditioned stimuli and/or the drug-paired context. Data were analyzed using one-way or two-way repeated measures ANOVAs.

RESULTS

VNS during extinction reduced reinstatement of CPP. VNS also reduced cue- and context-induced reinstatement of the instrumental response under both AAA and ABA conditions. The subjects' ability to engage with the lever during extinction was crucial for this effect. P values < 0.05 were considered significant.

CONCLUSIONS

Craving occurs in response to a range of conditioned stimuli and contexts; VNS may improve outcomes of behavioral therapy by facilitating extinction of both an instrumental response and/or contextual cues.

Removing but not adding elements of a context affects generalization of instrumental responses

Three experiments with rats investigated whether adding or removing elements of a context affects generalization of instrumental behavior. Each of the experiments used a free operant procedure. In Experiments 1 and 2, rats were trained to press a lever for food in a distinctive context. Then, transfer of lever pressing was tested in a context created either by adding an element to the context of initial acquisition or by removing one of the acquisition context's elements. In Experiment 3, a similar generalization test was conducted after rats received acquisition and extinction within the same context. For Experiments 1 and 2, we observed that removing elements from the acquisition context disrupted acquisition performance, whereas the addition of elements to the context did not. Experiment 3 revealed that removing elements from but not adding elements to the original context improved extinction performance. Our results are consistent with an elemental view of context representation.

Retrieval practice after multiple context changes, but not long retention intervals, reduces the impact of a final context change on instrumental behavior

Recent evidence from this laboratory suggests that a context switch after operant learning consistently results in a decrement in responding. One way to reduce this decrement is to train the response in multiple contexts. One interpretation of this result, rooted in stimulus sampling theory, is that conditioning of a greater number of common stimulus elements arising from more contexts causes better generalization to new contexts. An alternative explanation is that each change of context causes more effortful retrieval, and practice involving effortful retrieval results in learning that is better able to transfer to new situations. The current experiments were designed to differentiate between these two explanations for the first time in an animal learning and memory task. Experiment 1 demonstrated that the detrimental impact of a context change on an instrumental nose-poking response can be reduced by training the response in multiple contexts. Experiment 2 then found that a training procedure which inserted extended retention intervals between successive training sessions did not reduce the detrimental impact of a final context change. This occurred even though the inserted retention intervals had a detrimental impact on responding (and, thus, presumably retrieval) similar to the effect that context switches had in Experiment 1. Together, the results suggest that effortful retrieval practice may not be sufficient to reduce the negative impact of a context change on instrumental behavior. A common elements explanation which supposes that physical and temporal contextual cues do not overlap may account for the findings more readily.

Extinction of instrumental (operant) learning: interference, varieties of context, and mechanisms of contextual control

This article reviews recent research on the extinction of instrumental (or operant) conditioning from the perspective that it is an example of a general retroactive interference process. Previous discussions of interference have focused primarily on findings from Pavlovian conditioning. The present review shows that extinction in instrumental learning has much in common with other examples of retroactive interference in instrumental learning (e.g., omission learning, punishment, second-outcome learning, discrimination reversal learning, and differential reinforcement of alternative behavior). In each, the original learning can be largely retained after conflicting information is learned, and behavior is cued or controlled by the current context. The review also suggests that a variety of stimuli can play the role of context, including room and apparatus cues, temporal cues, drug state, deprivation state, stress state, and recent reinforcers, discrete cues, or behaviors. In instrumental learning situations, the context can control behavior through its direct association with the reinforcer or punisher, through its hierarchical relation with response-outcome associations, or its direct association (inhibitory or excitatory) with the response. In simple instrumental extinction and habit learning, the latter mechanism may play an especially important role.

A laboratory model for evaluating relapse of undesirable caregiver behavior

The success of behavioral treatments like functional communication training depends on their continued implementation outside of the clinical context, where failures in caregiver treatment adherence can lead to the relapse of destructive behavior. In the present study, we developed a laboratory model for evaluating the relapse of undesirable caregiver behavior that simulates two common sources of disruption (i.e., changes in context and in treatment efficacy) believed to affect caregiver treatment adherence using simulated confederate destructive behavior. In Phase 1, the caregiver's delivery of reinforcers for destructive behavior terminated confederate destructive behavior in a home-like context. In Phase 2, the caregiver implemented functional communication training in a clinical context in which providing reinforcers for destructive or alternative behavior terminated confederate destructive behavior. In Phase 3, the caregiver returned to the home-like context, and caregiver behavior produced no effect on confederate destructive or alternative behavior, simulating an inconsolable child. Undesirable caregiver behavior relapsed in three of four treatment-adherence challenges.

Quantitative analysis of local-level resurgence

Resurgence is the recurrence of a previously reinforced and then extinguished behavior induced by the extinction of another more recently reinforced behavior. Resurgence provides insight into behavioral processes relevant to treatment relapse of a range of problem behaviors. Resurgence is typically studied across three phases: (1) reinforcement of a target response, (2) extinction of the target and concurrent reinforcement of an alternative response, and (3) extinction of the alternative response, resulting in the recurrence of target responding. Because each phase typically occurs successively and spans multiple sessions, extended time frames separate the training and resurgence of target responding. This study assessed resurgence more dynamically and throughout ongoing training in 6 pigeons. Baseline entailed 50-s trials of a free-operant psychophysical procedure, resembling Phases 1 and 2 of typical resurgence procedures. During the first 25 s, we reinforced target (left-key) responding but not alternative (right-key) responding; contingencies reversed during the second 25 s. Target and alternative responding followed the baseline reinforcement contingencies, with alternative responding replacing target responding across the 50 s. We observed resurgence of target responding during signaled and unsignaled probes that extended trial durations an additional 100 s in extinction. Furthermore, resurgence was greater and/or sooner when probes were signaled, suggesting an important role of discriminating transitions to extinction in resurgence. The data were well described by an extension of a stimulus-control model of discrimination that assumes resurgence is the result of generalization of obtained reinforcers across space and time. Therefore, the present findings introduce novel methods and quantitative analyses for assessing behavioral processes underlying resurgence.