Priming and trial spacing in extinction: effects on extinction performance, spontaneous recovery, and reinstatement in appetitive conditioning

Exposure Therapy and Its Mechanisms

Exposure therapy is the gold-standard treatment approach for pathological anxiety. This therapeutic approach builds on principles of extinction training from traditional fear conditioning and extinction protocols. In this chapter, we discuss principles of exposure therapy in the clinic and the laboratory experimental results that guide our decisions in the therapy. We discuss emotional processing theory and inhibitory learning principles, with a focus on expectation violation. We conclude with future research directions needed to improve exposure therapy outcomes among patients with anxiety-related disorders.

EXPRESS: Determinants of Extinction in a Streamed Trial Procedure

The strength of an association between a cue and its outcome is influenced by both the probability of the outcome given the cue and the probability of the outcome in the absence of the cue. Once an association has been formed, extinction is the procedure for reducing responding indicative of the association by repeated presentation of the cue without the outcome. The present experiments tested whether cumulative frequency and/or cumulative duration of these events affects associative extinction in a streamed trial extinction procedure with human participants. Experiment 1 assessed the effects of parametric manipulations of the frequency and duration of either the cue by itself or cue-outcome coabsence. In Experiment 1, participants proved relatively insensitive to manipulation of the event's duration. In contrast, judgments of the association by participants decreased when the frequency of cue-alone events was increased, even when the durations of those events were decreased so that cumulative exposure to the cue was equated. No effect of either the duration or the frequency of cue-outcome coabsence was observed. Experiment 2 demonstrated that the effect of cue-alone (i.e., extinction trial) frequency generalizes across a wide range of parameters for initial acquisition achieved by cue-outcome pairings. Experiment 3 tested for an interaction between event duration during initial learning and event duration during extinction. Collectively, these results indicate that the cumulative frequency, and not the cumulative duration, of extinction trials as well as the duration of the cue-outcome coabsences between extinction trials controls the effectiveness of an extinction procedure.

Gambling on Smartphones: A Study of a Potentially Addictive Behaviour in a Naturalistic Setting

Smartphone users engage extensively with their devices, on an intermittent basis for short periods of time. These patterns of behaviour have the potential to make mobile gambling especially perseverative. This paper reports the first empirical study of mobile gambling in which a simulated gambling app was used to measure gambling behaviour in phases of acquisition and extinction. We found that participants showed considerable perseverance in the face of continued losses that were linearly related to their prior engagement with the app. Latencies between gambles were associated with the magnitude of reinforcement; more positive outcomes were associated with longer breaks between play and a greater propensity to end a gambling session. Greater latencies were associated with measurements of problem gambling, and perseverance with gambling-related cognitions and sensation-seeking behaviour.

Understanding the psychology of mobile gambling: A behavioural synthesis

This manuscript reviews the extant literature on key issues related to mobile gambling and considers whether the potential risks of harm emerging from this platform are driven by pre‐existing comorbidities or by psychological processes unique to mobile gambling. We propose an account based on associative learning that suggests this form of gambling is likely to show distinctive features compared with other gambling technologies. Smartphones are a rapidly growing platform on which individuals can gamble using specifically designed applications, adapted websites or text messaging. This review considers how mobile phone use interacts with psychological processes relevant to gambling, the games users are likely to play on smartphones, and the interactions afforded by smartphones. Our interpretation of the evidence is that the schedules of reinforcement found in gambling interact with the ways in which people tend to use smartphones that may expedite the acquisition of maladaptive learned behaviours such as problem gambling. This account is consistent with existing theories and frameworks of problem gambling and has relevance to other forms of mobile phone use.

Why are Some Games More Addictive than Others: The Effects of Timing and Payoff on Perseverance in a Slot Machine Game

Manipulating different behavioral characteristics of gambling games can potentially affect the extent to which individuals persevere at gambling, and their transition to problematic behaviors. This has potential impact for mobile gambling technologies and responsible gambling interventions. Two laboratory models pertinent to this are the partial reinforcement extinction effect (PREE) and the trial spacing effect. Both of these might speed up or delay the acquisition and extinction of conditioned behavior. We report an experiment that manipulated the rate of reinforcement and inter trial interval (ITI) on a simulated slot machine where participants were given the choice between gambling and skipping on each trial, before perseverative gambling was measured in extinction, followed by measurements of the illusion of control, depression and impulsivity. We hypothesized that longer ITI’s in conjunction with the low rates of reinforcement observed in gambling would lead to greater perseverance. We further hypothesized, given that timing is known to be important in displaying illusory control and potentially in persevering in gambling, that prior exposure to longer intervals might affect illusions of control. An interaction between ITI and rate of reinforcement was observed, as low reinforced gamblers with a long ITI gambled for longer. Respondents also displayed extinction and a PREE. Gamblers exposed to a higher rate of reinforcement gambled for longer in acquisition. Impulsivity was associated with extended perseverance in extinction, and more depressed gamblers in the high reinforcement short ITI group persevered for longer. Performance in the contingency judgment failed to support the second hypothesis: the only significant contrast observed was that participants became better calibrated as the task progressed.

Implications of learning theory for developing programs to decrease overeating

Childhood obesity is associated with medical and psychological comorbidities, and interventions targeting overeating could be pragmatic and have a significant impact on weight. Calorically dense foods are easily available, variable, and tasty which allows for effective opportunities to learn to associate behaviors and cues in the environment with food through fundamental conditioning processes, resulting in measurable psychological and physiological food cue reactivity in vulnerable children. Basic research suggests that initial learning is difficult to erase, and that it is vulnerable to a number of phenomena that will allow the original learning to re-emerge after it is suppressed or replaced. These processes may help explain why it may be difficult to change food cue reactivity and overeating over the long term. Extinction theory may be used to develop effective cue-exposure treatments to decrease food cue reactivity through inhibitory learning, although these processes are complex and require an integral understanding of the theory and individual differences. Additionally, learning theory can be used to develop other interventions that may prove to be useful. Through an integration of learning theory, basic and translational research, it may be possible to develop interventions that can decrease the urges to overeat, and improve the weight status of children.

Design and implementation of a study evaluating extinction processes to food cues in obese children: the Intervention for Regulations of Cues Trial (iROC)

Obesity and its health sequelae affect a significant portion of children in the United States. Yet, the current gold-standard family-based behavioral weight-loss treatments are only effective for one-third of children long-term. Therefore, we developed iROC (Intervention for Regulation of Cues) to specifically target a method to decrease overeating in overweight children, based on learning theory, to inform and enhance interventions targeting diet and obesity in youth. This study will rigorously test extinction processes as a method of decreasing physiological and psychological responses to food cues in overweight and obese children. Through exposing children to their highly craved foods, and 'training the brain and body' to decrease overeating, we are hoping to produce longer-lasting weight loss or weight-gain prevention over time.

Interoceptive conditioning with nicotine using extinction and re-extinction to assess stimulus similarity with bupropion

Bupropion is an atypical antidepressant that increases long-term quit rates of tobacco smokers. A better understanding of the relation between nicotine and this first-line medication may provide insight into improving treatment. For all experiments, rats first had nicotine (0.4 mg base/kg) and saline session intermixed; intermittent access to sucrose only occurred on nicotine session. Nicotine in this protocol comes to differentially control "anticipatory" dipper entries. To more closely examine the overlap in the interoceptive stimulus effects of nicotine and bupropion, we assessed whether subsequent prolonged and repeated non-reinforced (extinction) sessions with the bupropion stimulus could weaken responding to nicotine (i.e., transfer of extinction). We also examined whether retraining the discrimination after initial extinction and then conducting extinction again (i.e., re-extinction) with bupropion would affect responding. We found that bupropion (20 and 30 mg/kg) fully substituted for the nicotine stimulus in repeated 20-min extinction sessions. The extent of substitution in extinction did not necessarily predict performance in the transfer test (e.g., nicotine responding unchanged after extinction with 20 mg/kg bupropion). Generalization of extinction back to nicotine was not seen with 20 mg/kg bupropion even after increasing the number of extinction session from 6 to 24. Finally, there was evidence that learning in the initial extinction phase was retained in the re-extinction phase for nicotine and bupropion. These findings indicate that learning involving the nicotine stimuli are complex and that assessment approach for stimulus similarity changes conclusions regarding substitution by bupropion. Further research will be needed to identify whether such differences may be related to different facets of nicotine dependence and/or its treatment.

Can fear extinction be enhanced? A review of pharmacological and behavioral findings

There is considerable interest, from both a basic and clinical standpoint, in gaining a greater understanding of how pharmaceutical or behavioral manipulations alter fear extinction in animals. Not only does fear extinction in rodents model exposure therapy in humans, where the latter is a cornerstone of behavioral intervention for anxiety disorders such as post-traumatic stress disorder and specific phobias, but also understanding more about extinction provides basic information into learning and memory processes and their underlying circuitry. In this paper, we briefly review three principal approaches that have been used to modulate extinction processes in animals and humans: a purely pharmacological approach, the more widespread approach of combining pharmacology with behavior, and a purely behavioral approach. The pharmacological studies comprise modulation by: brain derived neurotrophic factor (BDNF), d-cycloserine, serotonergic and noradrenergic drugs, neuropeptides, endocannabinoids, glucocorticoids, histone deacetylase (HDAC) inhibitors, and others. These studies strongly suggest that extinction can be modulated by drugs, behavioral interventions, or their combination, although not always in a lasting manner. We suggest that pharmacotherapeutic manipulations provide considerable promise for promoting effective and lasting fear reduction in individuals with anxiety disorders. This article is part of a Special Issue entitled 'Memory enhancement'.

Separation of time-based and trial-based accounts of the partial reinforcement extinction effect

Two appetitive conditioning experiments with rats examined time-based and trial-based accounts of the partial reinforcement extinction effect (PREE). In the PREE, the loss of responding that occurs in extinction is slower when the conditioned stimulus (CS) has been paired with a reinforcer on some of its presentations (partially reinforced) instead of every presentation (continuously reinforced). According to a time-based or "time-accumulation" view (e.g., Gallistel and Gibbon, 2000), the PREE occurs because the organism has learned in partial reinforcement to expect the reinforcer after a larger amount of time has accumulated in the CS over trials. In contrast, according to a trial-based view (e.g., Capaldi, 1967), the PREE occurs because the organism has learned in partial reinforcement to expect the reinforcer after a larger number of CS presentations. Experiment 1 used a procedure that equated partially and continuously reinforced groups on their expected times to reinforcement during conditioning. A PREE was still observed. Experiment 2 then used an extinction procedure that allowed time in the CS and the number of trials to accumulate differentially through extinction. The PREE was still evident when responding was examined as a function of expected time units to the reinforcer, but was eliminated when responding was examined as a function of expected trial units to the reinforcer. There was no evidence that the animal responded according to the ratio of time accumulated during the CS in extinction over the time in the CS expected before the reinforcer. The results thus favor a trial-based account over a time-based account of extinction and the PREE. This article is part of a Special Issue entitled: Associative and Temporal Learning.

Effects of the amount of acquisition and contextual generalization on the renewal of instrumental behavior after extinction

Four experiments with rat subjects examined the role of context during the extinction of instrumental (free-operant) behavior. In all experiments, leverpressing was first reinforced on a variable-interval 30-s schedule and then extinguished before being tested in the extinction and renewal contexts. The results identified three important variables affecting the renewal effect after instrumental extinction. First, ABA and ABC forms of renewal were strengthened by increasing the amount of acquisition training. This suggests that the strength of the association learned during acquisition, or the final level of performance, influences the degree of renewal after extinction. The effect of the amount of training was modulated by the second factor, the degrees of generalization from the acquisition and extinction contexts to the test context. The third variable was acquisition training in multiple contexts, which was shown to strengthen ABC renewal. Methodological, theoretical, and practical implications are discussed.

Asymmetrical generalization of conditioning and extinction from compound to element and element to compound

Four appetitive conditioning experiments studied generalization between compound conditional stimuli (AB) and their elements (e.g., A or B). In Experiments 1 and 2, rats received conditioning with A and AB, and then extinction with either A or AB. During subsequent testing, there was more generalization of extinction (nonresponding) from the compound (AB) to the element (A) than from the element (A) to the compound (AB). This asymmetry was consistent with earlier results involving temporal discrimination learning in which short and long temporal intervals played the roles of A and AB. In Experiment 3, rats received conditioning with either A or AB, and then testing with A and AB. Consistent with elemental models of conditioning, there was more generalization of conditioned responding from A to AB than from AB to A. Experiment 4 found that these asymmetries in the generalization of extinction (Experiments 1 and 2) and conditioning (Experiment 3) both contribute to the feature-positive effect. Overall, the parallel between the current findings and previous results with temporal discrimination learning supports an associative analysis of interval timing. Implications for elemental and configural theories of conditioning and generalization are also discussed.

Spontaneous recovery but not reinstatement of the extinguished conditioned eyeblink response in the rat

Reinstatement--the return of an extinguished conditioned response (CR) after reexposure to the unconditioned stimulus (US)--and spontaneous recovery--the return of an extinguished CR with the passage of time--are 2 of 4 well-established phenomena that demonstrate that extinction does not erase the conditioned stimulus (CS)-US association. However, reinstatement of extinguished eyeblink CRs has never been demonstrated, and spontaneous recovery of extinguished eyeblink CRs has not been systematically demonstrated in rodent eyeblink conditioning. In Experiment 1, US reexposure was administered 24 hr prior to a reinstatement test. In Experiment 2, US reexposure was administered 5 min prior to a reinstatement test. In Experiment 3, a long, discrete cue (a houselight), present in all phases of training and testing, served as a context within which each trial occurred to maximize context processing, which in other preparations has been shown to be required for reinstatement. In Experiment 4, an additional group was included that received footshock exposure, rather than US reexposure, between extinction and test, and contextual freezing was measured prior to test. Spontaneous recovery was robust in Experiments 3 and 4. In Experiment 4, context freezing was strong in a group given footshock exposure but not in a group given eye shock US reexposure. There was no reinstatement observed in any experiment. With stimulus conditions that produce eyeblink conditioning and research designs that produce reinstatement in other forms of classical conditioning, we observed spontaneous recovery but not reinstatement of extinguished eyeblink CRs. This suggests that reinstatement, but not spontaneous recovery, is a preparation- or substrate-dependent phenomenon.

Renewal after the extinction of free operant behavior

Four experiments were performed to explore the role of context in operant extinction. In all experiments, leverpressing in rats was first reinforced with food pellets on a variable interval 30-s schedule, then extinguished, and finally tested in the same and a different physical context. The experiments demonstrated a clear ABA renewal effect, a recovery of extinguished responding when conditioning, extinction, and testing occurred in contexts A, B, and A, respectively. They also demonstrated ABC renewal (where conditioning extinction and testing occurred in contexts A, B, and C) and, for the first time in operant conditioning, AAB renewal (where conditioning, extinction, and testing occurred in contexts A, A, and B). The latter two phenomena indicate that tests outside the extinction context are sufficient to cause a recovery of extinguished operant behavior and, thus, that operant extinction, like Pavlovian extinction, is relatively specific to the context in which it is learned. AAB renewal was not weakened by tripling the amount of extinction training. ABA renewal was stronger than AAB, but not merely because of context A's direct association with the reinforcer.

Spacing extinction trials alleviates renewal and spontaneous recovery

Studies of extinction in classical conditioning situations can reveal techniques that maximize the effectiveness of exposure-based behavior therapies. In three experiments, we investigated the effect of varying the intertrial interval during an extinction treatment in a fear-conditioning preparation with rats as subjects. In Experiment 1, we found less fear at test (i.e., more effective extinction) when extinction trials were widely spaced, relative to intermediate or massed extinction trials. In Experiment 2, we used an ABA renewal procedure and observed that spaced trials attenuated renewal of conditioned fear relative to massed trials. In Experiment 3, we used a similar design, but instead of changing the physical context at the time of testing, we interposed a retention interval after the extinction treatment to produce a change in the temporal context. The results showed less spontaneous recovery of fear after spaced than after massed extinction trials. These results suggest that extinction is more enduring when the extinction trials are spaced rather than massed. Although the benefits of spacing trials are small when there is no contextual change from extinction to testing, a change in either physical or temporal context following massed extinction trials leads to a recovery from extinction, which is reduced when the trials are spaced.

The facilitative effects of D-cycloserine on extinction of a cocaine-induced conditioned place preference can be long lasting and resistant to reinstatement

RATIONALE AND OBJECTIVE

The N-methyl-D-aspartate receptor agonist, D-cycloserine (DCS), accelerates extinction of a cocaine-induced conditioned place preference (CPP) when given after daily extinction tests. Here, we studied the effects of DCS in rats given spaced-extinction sessions at 3- or 7-day intervals using two different extinction procedures.

MATERIALS AND METHODS

Rats were trained on a CPP (four cocaine, 10 mg/kg, i.p., and four saline pairings with one of two compartments). Immediately following the CPP test and all extinction tests (days 4, 7, 10, and 24, experiment 1), DCS (15 mg/kg, i.p.) or saline was administered. In experiment 2, extinction was conducted by exposing rats to the drug-paired cues for 2 or 20 min, three times, at 7-day intervals followed immediately by DCS or saline. After extinction, tests for retention and cocaine-induced reinstatement were given.

RESULTS

In experiment 1, rats given DCS lost the cocaine CPP after one extinction trial, an effect that persisted for 2 weeks after the last DCS injection and that was resistant to cocaine-induced reinstatement. In experiment 2, extinction was facilitated by DCS compared to saline when rats received 2-min exposures to the conditioned stimulus. Longer 20-min exposures minus/plus repeated testing led to retention of extinction in both groups regardless of DCS treatment.

CONCLUSIONS

Extinction of appetitive conditioning is facilitated by DCS after 1-3 post-spaced trial injections, and retention is lasting and resistant to reinstatement. The facilitative effects appear early in extinction, but when extinction procedures are intensive, DCS appears to have no additional benefit.

Facilitation of extinction of operant behaviour in mice by D-cycloserine

RATIONALE AND OBJECTIVE

The N-methyl-D-aspartate receptor agonist D-cycloserine (DCS) facilitates extinction following Pavlovian fear conditioning or conditioned place preference in rats, but its effects on extinction following operant conditioning are not previously established. We studied the effects of DCS on operant extinction with mice, previously shown to be facilitated by GABAergic potentiators including chlordiazepoxide.

MATERIALS AND METHODS

Following training of lever pressing by C57Bl/6 male mice on a discrete-trial fixed-ratio food reinforcement schedule with six reinforcers per session, 48-trial extinction sessions were conducted at 3- (Experiment 1) or 4-day intervals (Experiment 2). Effects of DCS (15 or 30 mg/kg, i.p.) administered immediately after 48-trial extinction sessions were compared with those of saline injections.

RESULTS

With 3-day intervals between extinction sessions, post-session administration of DCS facilitated extinction, and this effect was stronger with 4-day intervals between extinction sessions. Facilitation of extinction by post-session drug administration persisted over a number of extinction sessions.

CONCLUSIONS

Operant extinction in mice can be facilitated by DCS, a glutamatergic agonist, as well as by GABAergic potentiators. The relationship between glutamatergic and GABAergic processes in operant extinction is yet to be established. These findings strengthen the basis for clinical uses of DCS.

Immediate extinction causes a less durable loss of performance than delayed extinction following either fear or appetitive conditioning

Five experiments with rat subjects compared the effects of immediate and delayed extinction on the durability of extinction learning. Three experiments examined extinction of fear conditioning (using the conditioned emotional response method), and two experiments examined extinction of appetitive conditioning (using the food-cup entry method). In all experiments, conditioning and extinction were accomplished in single sessions, and retention testing took place 24 h after extinction. In both fear and appetitive conditioning, immediate extinction (beginning 10 min after conditioning) caused a faster loss of responding than delayed extinction (beginning 24 h after conditioning). However, immediate extinction was less durable than delayed extinction: There was stronger spontaneous recovery during the final retention test. There was also substantial renewal of responding when the physical context was changed between immediate extinction and testing (Experiment 1). The results suggest that, in these two widely used conditioning preparations, immediate extinction does not erase or depotentiate the original learning, and instead creates a less permanent reduction in conditioned responding. Results did not support the possibility that the strong recovery after immediate extinction was due to a mismatch in the recent "context" provided by the presence or absence of a recent conditioning experience. Several other accounts are considered.

Responding in Configural Discrimination Problems Depends on Density of Reinforcement in Time

Abstract. Previous research has shown that conditioned responding in differential skin conductance conditioning increased for reinforced stimuli (CSs+) but remained constant for nonreinforced stimuli (CSs-) due to decreasing reinforcement density. The present two experiments (Experiment 1: Negative patterning; Experiment 2: Positive patterning) were designed to disentangle a possible confound of reinforcement density with stimulus frequency. In order to achieve this, we varied the intertrial interval (18 s, 24 s, or 48 s) and held constant the numbers of CSs+ and of CSs- in each of both discrimination problems. With increasing intertrial intervals, we found higher responding both to CSs+ and to CSs- as well as increased response differentiation. We discuss these results with respect to two mechanisms offered by Wagner's SOP model and conclude that the observed effects are due to variations in density of reinforcement in time.

Contextual and temporal modulation of extinction: behavioral and biological mechanisms

Extinction depends, at least partly, on new learning that is specific to the context in which it is learned. Several behavioral phenomena (renewal, reinstatement, spontaneous recovery, and rapid reacquisition) suggest the importance of context in extinction. The present article reviews research on the behavioral and neurobiological mechanisms of contextual influences on extinction learning and retrieval. Contexts appear to select or retrieve the current relationship of the conditional stimulus (CS) with the unconditional stimulus (US), and they are provided by physical background cues, interoceptive drug cues, emotions, recent trials, and the passage of time. The current article pays particular attention to the effects of recent trials and trial spacing. Control of fear extinction by physical context involves interactions between the dorsal hippocampus and the lateral nucleus of the amygdala. This interaction may be mediated by gamma-aminobutyric acid (GABA)-ergic and adrenergic mechanisms.

Intertrial interval as a contextual stimulus

Four experiments with rats investigated whether the time between appetitive conditioning trials can serve as a discriminative cue for responding during the next conditional stimulus (CS). In Experiment 1, rats that received extinction trials with a 4-min intertrial interval (ITI) showed spontaneous recovery after a retention interval of 16 min, whereas rats that received extinction with a 16-min ITI did not. Experiments 2 and 3 investigated more explicit discriminations between the 4- and 16-min ITIs. When a 16-min ITI signaled that the CS would be reinforced and a 4-min ITI signaled that it would not, the ITIs modulated responding to the CS. But when the 4-min ITI signaled reinforcement and the 16-min ITI did not, there was less evidence of modulation by the ITIs. This asymmetry was due at least partly to a difficulty in performance rather than learning. Experiment 4 investigated similar discriminations with 1- and 4-min ITIs. Here the results took a different form: time in the reinforced ITI elicited responding directly, but did not modulate responding to the CS. ITI can function as a contextual cue, and the results suggest new similarities between the processes behind interval timing and associative learning.