Intertrial interval as a contextual stimulus

Sex divergent behavioral responses in platform-mediated avoidance and glucocorticoid receptor blockade

Women are more likely than men to develop anxiety or stress-related disorders. A core behavioral symptom of all anxiety disorders is avoidance of fear or anxiety eliciting cues. Recent rodent models of avoidance show reliable reproduction of this behavioral phenomenon in response to learned aversive associations. Here, a modified version of platform-mediated avoidance that lacked an appetitive task was utilized to investigate the learning and extinction of avoidance in male and female C57BL6/J mice. Here, we found a robust sex difference in the acquisition and extinction of platform-mediated avoidance. Across three experiments, 63.7% of female mice acquired avoidance according to our criterion, whereas 83.8% of males acquired it successfully. Of those females that acquired avoidance, they displayed persistent avoidance after extinction compared to males. Given their role in regulating stress responses and habitual behaviors, we investigated if glucocorticoid receptors (GR) mediated avoidance learning in males and females. We found that a subcutaneous injection (25 mg/kg) of the GR antagonist, RU486 (Mifepristone), significantly reduced persistent avoidance in females but did not further reduce avoidance in males after extinction. These data suggest that GR activation during avoidance learning may contribute to persistent avoidance in females that is resistant to extinction.

Sex divergent behavioral responses in platform-mediated avoidance and glucocorticoid receptor blockade

Women are more likely than men to develop anxiety or stress-related disorders. A core behavioral symptom of all anxiety disorders is avoidance of fear or anxiety eliciting cues. Recent rodent models of avoidance show reliable reproduction of this behavioral phenomenon in response to learned aversive associations. Here, a modified version of platform-mediated avoidance that lacked an appetitive task was utilized to investigate the learning and extinction of avoidance in male and female C57BL6/J mice. Here, we found a robust sex difference in the acquisition and extinction of platform-mediated avoidance. Across three experiments, 63.7% of female mice acquired avoidance according to our criterion, whereas 83.8% of males acquired it successfully. Of those females that acquired avoidance, they displayed persistent avoidance after extinction compared to males. Given their role in regulating stress responses and habitual behaviors, we investigated if glucocorticoid receptors (GR) mediated avoidance learning in males and females. Here we found that a subcutaneous injection (25mg/kg) of the GR antagonist, RU486 (mifepristone), significantly reduced persistent avoidance in females but did not further reduce avoidance in males after extinction. These data suggest that GR activation during avoidance learning may contribute to persistent avoidance in females that is resistant to extinction.

Prelimbic cortex inactivation prevents ABA renewal based on satiety state

We have previously shown that the rat prelimbic cortex (PL) is necessary for contexts to promote the performance of instrumental behaviors that have been learned in them, whether the context is physical (operant chamber) or behavioral (recent performance of a behavior that has historically preceded the target in a behavior chain). In the present experiment, we investigated the role of the PL in satiety level as an interoceptive acquisition context. Rats were trained to lever-press for sweet/fat pellets while sated (22 hrs continuous food access) followed by the extinction of the response while hungry (22 hrs food deprived). Pharmacological inactivation of the PL (with baclofen/muscimol infusion) attenuated renewal of the response that occurred upon a return to the sated context. In contrast, animals that received a vehicle (saline) infusion showed renewal of the previously extinguished response. These results support the hypothesis that the PL monitors the relevant contextual elements (physical, behavioral, or satiety state) associated with reinforcement of a response and promotes the subsequent performance of that response in their presence.

Learning About Reward Identities and Time

We discuss three empirical findings that we think any theory attempting to integrate interval timing with associative learning concepts will need to address. These empirical phenomena all come from studies that combine peak timing procedures with reinforcer devaluation or conditional discrimination tasks commonly employed, respectively, in interval timing or associative learning research traditions. The three phenomena we discuss include: (1) the observation that disruptions in reward identity encoding have little to no impact on the encoding of reward time in the a peak procedure (Delamater., 1998), (2) the findings that organisms tend to average their time estimates when presented with a stimulus compound consisting of separately learned stimuli indicating short or long reward times but that such temporal averaging, itself, is sensitive to post-conditioning selective reward devaluation, and (3) that rats can learn a temporal patterning task in which two stimuli presented independently indicate one time to reward availability while their compound indicates another. We review our prior results and present new findings illustrating these three phenomena and we discuss the special challenges they pose for cascade theories of timing, for multiple-oscillator models, and for any approach that attempts to integrate interval timing and associative models. We close by illustrating some ways in which multi-layer connectionist network models might begin to address some of our key findings. We believe this will require an approach that includes separate mechanisms that code for reward identity and time, but that does so in a way that permits for integration between the two systems.

EXPRESS: Determinants of Extinction in a Streamed Trial Procedure

The strength of an association between a cue and its outcome is influenced by both the probability of the outcome given the cue and the probability of the outcome in the absence of the cue. Once an association has been formed, extinction is the procedure for reducing responding indicative of the association by repeated presentation of the cue without the outcome. The present experiments tested whether cumulative frequency and/or cumulative duration of these events affects associative extinction in a streamed trial extinction procedure with human participants. Experiment 1 assessed the effects of parametric manipulations of the frequency and duration of either the cue by itself or cue-outcome coabsence. In Experiment 1, participants proved relatively insensitive to manipulation of the event's duration. In contrast, judgments of the association by participants decreased when the frequency of cue-alone events was increased, even when the durations of those events were decreased so that cumulative exposure to the cue was equated. No effect of either the duration or the frequency of cue-outcome coabsence was observed. Experiment 2 demonstrated that the effect of cue-alone (i.e., extinction trial) frequency generalizes across a wide range of parameters for initial acquisition achieved by cue-outcome pairings. Experiment 3 tested for an interaction between event duration during initial learning and event duration during extinction. Collectively, these results indicate that the cumulative frequency, and not the cumulative duration, of extinction trials as well as the duration of the cue-outcome coabsences between extinction trials controls the effectiveness of an extinction procedure.

The effects of presenting additional stimuli resembling the CS+ during extinction on extinction retention and generalisation to novel stimuli

Recent studies have shown that extinction training including the conditional stimulus (CS+) and stimuli similar to the CS + enhances extinction retention and generalisation to novel stimuli. The aim of the present study was to confirm that these effects are specific to presenting stimuli similar to the CS+ during extinction and not merely an effect of additional trials or additional stimuli unrelated to the CS+. In an experiment conducted in a single session on the same day, participants (134 adults; 70.7% female; 17-40 years of age, M = 20.04, SD = 4.36) completed a habituation phase followed by an acquisition phase using dog images presented with (CS+) and without (CS-) a dog growl paired with a scream unconditional stimulus (US). Participants were randomly allocated to four extinction conditions: Multiple exemplar extinction comprising the CSs and two novel dog images similar to the CS+; Standard extinction control matched for the number of CS+ and CS- presentations; Extended extinction control matched for the total number extinction trials, and Other stimuli extinction control comprising the CSs and two novel stimuli unrelated to the CS+. All participants completed an extinction test with the original CSs followed by a generalisation test with two novel dog images. Multiple, Standard and Other stimuli extinction groups exhibited larger skin conductance responses (SCRs) during extinction to the CSs compared to the Extended extinction group. SCRs to the additional dog images in the Multiple group were larger than SCRs to the additional CSs in the Extended group and the novel images in the Other stimuli group. There were no group differences in responses to the CSs during extinction test. Unlike the other groups, SCRs to the first presentation of the novel generalisation dogs did not differ from those to the last CS trials in extinction in the Multiple group. However, this group difference did not persist beyond the initial generalisation trial. Finally, the Multiple, Extended, and Other stimuli groups exhibited more negative CS evaluations after extinction, extinction test, and generalisation test than the Standard extinction group. The results suggest that extinction with the original CSs and additional stimuli resembling the CS + elevated physiological responses during extinction and reduced physiological responses to novel stimuli similar to the CSs. Further studies are needed including clinical samples and trial-by-trial evaluations of the stimuli presented.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

An evaluation of a renewal-mitigation procedure for inappropriate mealtime behavior

Renewal, the increase in behavior during extinction following context changes, may be particularly concerning during intervention for feeding disorders because context changes are often necessary for intervention generality and maintenance (Podlesnik et al., 2017). In the current study, we tested for renewal and evaluated a renewal-mitigation procedure when we transferred intervention from a therapist to a caregiver, from clinic to the home, and changed the foods the feeder presented. We used an ABA arrangement to evaluate the generality of the renewal effect with 7 participants who engaged in inappropriate mealtime behavior. Context A was functional reinforcement. Context B was function-based extinction during the control and mitigation conditions and our renewal-mitigation procedure in the mitigation condition. The renewal test was function-based extinction in Context A. We observed renewal of inappropriate mealtime behavior in 4 of 7 participants, and our renewal-mitigation procedure was effective for 4 of 4 participants.

Generalization of extinction of a generalization stimulus in fear learning

Two experiments examined whether extinction of a generalization stimulus (GS) after single cue fear conditioning would in turn generalize to other stimuli, relative to a control group that received regular extinction of CS + itself. We found only a weak effect of such "generalization of GS extinction" either back to CS + or to a different GS, on either US expectancy or skin conductance measures. In other words, despite extinction trials with a stimulus highly similar to CS+, participants showed a return of fear when tested with CS + or a novel GS. However this responding declined rapidly over non-reinforced test trials. Trait anxious participants showed higher overall US expectancy ratings in the extinction and test phases, and slower extinction of expectancy, relative to low anxious participants. These results may help explain why exposure therapy, which is unlikely to reproduce the exact stimuli present at acquisition, sometimes fails to transfer to other fear-eliciting stimuli subsequently encountered by anxious clients. The generalization of GS extinction paradigm might provide a useful testbed for evaluation of interventions designed to enhance transfer, such as exposure to multiple diverse exemplars.

Extinction of instrumental (operant) learning: interference, varieties of context, and mechanisms of contextual control

This article reviews recent research on the extinction of instrumental (or operant) conditioning from the perspective that it is an example of a general retroactive interference process. Previous discussions of interference have focused primarily on findings from Pavlovian conditioning. The present review shows that extinction in instrumental learning has much in common with other examples of retroactive interference in instrumental learning (e.g., omission learning, punishment, second-outcome learning, discrimination reversal learning, and differential reinforcement of alternative behavior). In each, the original learning can be largely retained after conflicting information is learned, and behavior is cued or controlled by the current context. The review also suggests that a variety of stimuli can play the role of context, including room and apparatus cues, temporal cues, drug state, deprivation state, stress state, and recent reinforcers, discrete cues, or behaviors. In instrumental learning situations, the context can control behavior through its direct association with the reinforcer or punisher, through its hierarchical relation with response-outcome associations, or its direct association (inhibitory or excitatory) with the response. In simple instrumental extinction and habit learning, the latter mechanism may play an especially important role.

Superior ambiguous occasion setting with visual than temporal feature stimuli

Three experiments with rats compared the relative ease with which different sets of visual or temporal cues could participate in Pavlovian learning. In Experiment 1, 1 group was trained to discriminate between visual cues (Light vs. Dark), whereas the other group learned to discriminate between temporal cues (early [10 s] vs. late [90 s]). Both groups learned to distinguish food-paired from nonpaired periods equally well. In Experiment 2, 2 groups were trained on an ambiguous occasion setting task. For Group Visual, a 2-min Light period signaled that 1 10-s auditory conditioned stimulus, CS1, was reinforced with 1 unconditioned stimulus, US1, but that CS2 was not reinforced; whereas a 2-min dark period signaled that CS1 was not reinforced, but CS2 was reinforced with US2 (i.e., Light: CS1-US1, CS2-; Dark: CS1-, CS2-US2). For Group Temporal, early (10-s) or late (90-s) temporal cues within each of these Light and Dark periods were diagnostic of these contingencies (i.e., Early: CS1-US1, CS2-; Late: CS1-, CS2-US2). Group Visual learned the task, but Group Temporal did not. In Experiment 3 we demonstrated that animals could not solve a related temporal ambiguous occasion setting task in which 1 visual stimulus signaled that both CSs were reinforced early whereas the other visual stimulus signaled that the CSs were reinforced only late. Contrary to a currently popular information theory approach to timing in Pavlovian learning, these results suggest that overt nontemporal visual stimuli are better incorporated into conditional discrimination learning than are temporal stimuli. (PsycINFO Database Record

The discrimination of magnitude: A review and theoretical analysis

In a discrimination based on magnitude, the same stimulus is presented at two different magnitudes and an outcome, such as food, is signalled by one magnitude but not the other. The review presented in the first part of the article shows that, in general, such a discrimination is acquired more readily when the outcome is signalled by the larger rather than the smaller of the two magnitudes. This asymmetry is observed with magnitudes based on sound, odour, temporal duration, quantity, and physical length. The second part of the article, explores the implications of this pattern of results for the theory of discrimination learning presented by Pearce (1994). The asymmetry found with discriminations based on magnitude contradicts predictions derived from the original version of the theory, but it can be explained by a modified version. The asymmetry also has important implications for understanding how animals represent magnitudes.

Temporal generalization gradients following an interdimensional discrimination protocol

We investigated the effects of interdimensional discrimination training in the temporal generalization gradient. In a matching-to-sample task, pigeons learned to choose key S after a T-s houselight sample and key NS in the absence of the houselight sample. For one group of pigeons, T = 20 s; for another, T = 10 s. Subsequently, houselight duration was varied to obtain temporal generalization gradients. Results showed that (a) proportion S increased as houselight duration ranged from 0 s to T s and then remained high for houselight durations longer than T; (b) the gradients were well described by negative-exponential functions; (c) these non-flat gradients were present from the beginning of testing, and; (d) the average gradients obtained with T = 20 s and T = 10 s overlapped when plotted in relative time. We conclude that temporal control does not require explicit discrimination training along the temporal dimension, and that temporal generalization gradients obtained with an interdimensional protocol show the scalar property of timing. We discuss how these findings challenge current models of timing.

Asymmetry in the discrimination of quantity: The role of stimulus generalization

In order to evaluate 1 account for the asymmetry that has been found with discriminations based on stimulus magnitude, in 5 autoshaping experiments, 2 groups of pigeons received a discrimination between 5 and 20 squares presented on a TV screen. One group received a 20+/5- discrimination, with food signaled by 20 squares but not 5 squares; the other group received the opposite discrimination, 5+/20-. The 20+/5- discrimination was acquired more readily than 5+/20- in Experiments 1, 3a, 3b, and 4. For Experiment 1, the screen was white for the intertrial interval (ITI) and the stimuli were black squares on a white background; for Experiment 3a, the screen was black for the ITI and the stimuli were black squares on a white background; and for Experiments 3b and 4, the screen was white for the ITI and the stimuli were white squares on a black background. In Experiment 2, the stimuli were black squares on a white background, but they were separated by an ITI in which 288 black squares were presented against a white background. The 20+/5- discrimination was now acquired more slowly than the 5+/20- discrimination. The asymmetry in the acquisition of the magnitude discriminations in each experiment is attributed to inhibition being associated with the stimuli present during the ITI. The generalization of this inhibition, along a dimension related to the number of squares on the screen, is then assumed to disrupt the acquisition of 1 discrimination to a greater extent than the other.

Asymmetry in the discrimination of quantity by rats: The role of the intertrial interval

In three experiments, rats were trained to discriminate between 20 and five (Exps. 1 and 2), or between 40 and five (Exp. 3), black squares. The squares were randomly distributed in the center of a white background and displayed on a computer screen. For one group, the patterns containing the higher quantity of squares signaled the delivery of sucrose (+), whilst patterns with the lower quantity of squares did not (-). For the second group, sucrose was signaled by the lower, but not by the higher, quantity of squares. In Experiment 1, the intertrial interval (ITI) was a white screen, and the 20+/5- discrimination was acquired more readily than the 5+/20- discrimination. For Experiment 2, the ITI was made up of 80 black squares on a white background. In this instance, the 5+/20- discrimination was acquired more successfully than the 20+/5- discrimination. In Experiment 3, two groups were trained with a 40+/5- discrimination, and two with a 5+/40- discrimination. For one group from each of these pairs, the training trials were separated by a white ITI, and the 40+/5- discrimination was acquired more readily than the 5+/40- discrimination. For the remaining two groups, the training trials were not separated by an ITI, and the two groups acquired the task at approximately the same rate. The results indicate that the cues present during the ITI play a role in the asymmetrical acquisition of magnitude discriminations based on quantity.

Contribution of the retrosplenial cortex to temporal discrimination learning

The retrosplenial cortex (RSC) has an important role in contextual learning and memory. While the majority of experiments have focused on the physical context, the present study asked whether the RSC is involved in processing the temporal context. Rats were trained in a temporal discrimination procedure where the duration of the intertrial interval (ITI) signaled whether or not the next tone conditioned stimulus would be paired with food pellet reinforcement. When the tone was presented after a 16-min ITI it was reinforced, but when it was presented after a 4-min ITI it was not. Rats demonstrated successful discrimination in this procedure by responding more to the tone on reinforced trials than on non-reinforced trials. Pre-training electrolytic lesions of the RSC attenuated acquisition of the temporal discrimination. The results are the first to demonstrate a role for the RSC in processing temporal information and in turn extend the role of the RSC beyond the physical context to now include the temporal context.

Associative Accounts of Recovery-from-Extinction Effects

Recovery-from-extinction effects (e.g., spontaneous recovery, renewal, reinstatement, and facilitated reacquisition) have become the focus of much research in recent years. However, despite a great deal of empirical data, there are few theoretical explanations for these effects. This paucity poses a severe limitation on our understanding of these behavioral effects, impedes advances in uncovering neural mechanisms of response recovery, and reduces our potential to prevent relapse after exposure therapy. Towards correcting this oversight, this review takes prominent models of associative learning that have been used in the past and continue to be used today to explain Pavlovian conditioning and extinction, and assesses how each model can be applied to account for recovery-from-extinction effects. The models include the Rescorla-Wagner (1972) model, Mackintosh's (1975) attentional model, Pearce and Hall's (1980) attentional model, Wagner's (1981) SOP model, Pearce's (1987) configural model, McLaren and Mackintosh's (2002) elemental model, and Stout and Miller's (2007) SOCR (comparator hypothesis) model. Each model is assessed for how well it explains or does not explain the various recovery-from-extinction phenomena. We offer some suggestions for how the models might be modified to account for these effects in those instances in which they initially fail.

Asymmetry in the discrimination of length during spatial learning

The ability of rats to solve a discrimination between two objects that differ in length was investigated in five experiments. Using a rectangular swimming pool, Experiment 1 revealed it is easier to locate a submerged platform when it is near the center of a long rather than a short wall. For Experiments 2-4, the objects were black or white panels pasted onto the gray walls of a square pool, with two long panels pasted to two opposing walls and two short panels pasted to the remaining walls. The platform was easier to locate when it was placed near the middle of a long rather than a short panel. This effect was found when the long panels were twice (Experiments 2-4) or four times the length of the short panels (Experiment 4). Experiment 5 demonstrated that rats can solve a discrimination between panels of length 15 and 45 cm more readily than when they are 70 and 100 cm. The results are consistent with the claim that generalization gradients based on stimulus magnitude are steeper for stimuli that are weaker rather than stronger than the stimulus used for the original training.

Separation of time-based and trial-based accounts of the partial reinforcement extinction effect

Two appetitive conditioning experiments with rats examined time-based and trial-based accounts of the partial reinforcement extinction effect (PREE). In the PREE, the loss of responding that occurs in extinction is slower when the conditioned stimulus (CS) has been paired with a reinforcer on some of its presentations (partially reinforced) instead of every presentation (continuously reinforced). According to a time-based or "time-accumulation" view (e.g., Gallistel and Gibbon, 2000), the PREE occurs because the organism has learned in partial reinforcement to expect the reinforcer after a larger amount of time has accumulated in the CS over trials. In contrast, according to a trial-based view (e.g., Capaldi, 1967), the PREE occurs because the organism has learned in partial reinforcement to expect the reinforcer after a larger number of CS presentations. Experiment 1 used a procedure that equated partially and continuously reinforced groups on their expected times to reinforcement during conditioning. A PREE was still observed. Experiment 2 then used an extinction procedure that allowed time in the CS and the number of trials to accumulate differentially through extinction. The PREE was still evident when responding was examined as a function of expected time units to the reinforcer, but was eliminated when responding was examined as a function of expected trial units to the reinforcer. There was no evidence that the animal responded according to the ratio of time accumulated during the CS in extinction over the time in the CS expected before the reinforcer. The results thus favor a trial-based account over a time-based account of extinction and the PREE. This article is part of a Special Issue entitled: Associative and Temporal Learning.

Asymmetrical generalization of conditioning and extinction from compound to element and element to compound

Four appetitive conditioning experiments studied generalization between compound conditional stimuli (AB) and their elements (e.g., A or B). In Experiments 1 and 2, rats received conditioning with A and AB, and then extinction with either A or AB. During subsequent testing, there was more generalization of extinction (nonresponding) from the compound (AB) to the element (A) than from the element (A) to the compound (AB). This asymmetry was consistent with earlier results involving temporal discrimination learning in which short and long temporal intervals played the roles of A and AB. In Experiment 3, rats received conditioning with either A or AB, and then testing with A and AB. Consistent with elemental models of conditioning, there was more generalization of conditioned responding from A to AB than from AB to A. Experiment 4 found that these asymmetries in the generalization of extinction (Experiments 1 and 2) and conditioning (Experiment 3) both contribute to the feature-positive effect. Overall, the parallel between the current findings and previous results with temporal discrimination learning supports an associative analysis of interval timing. Implications for elemental and configural theories of conditioning and generalization are also discussed.

The neuroscience of learning: beyond the Hebbian synapse

From the traditional perspective of associative learning theory, the hypothesis linking modifications of synaptic transmission to learning and memory is plausible. It is less so from an information-processing perspective, in which learning is mediated by computations that make implicit commitments to physical and mathematical principles governing the domains where domain-specific cognitive mechanisms operate. We compare the properties of associative learning and memory to the properties of long-term potentiation, concluding that the properties of the latter do not explain the fundamental properties of the former. We briefly review the neuroscience of reinforcement learning, emphasizing the representational implications of the neuroscientific findings. We then review more extensively findings that confirm the existence of complex computations in three information-processing domains: probabilistic inference, the representation of uncertainty, and the representation of space. We argue for a change in the conceptual framework within which neuroscientists approach the study of learning mechanisms in the brain.

Relapse processes after the extinction of instrumental learning: renewal, resurgence, and reacquisition

It is widely recognized that extinction (the procedure in which a Pavlovian conditioned stimulus or an instrumental action is repeatedly presented without its reinforcer) weakens behavior without erasing the original learning. Most of the experiments that support this claim have focused on several "relapse" effects that occur after Pavlovian extinction, which collectively suggest that the original learning is saved through extinction. However, although such effects do occur after instrumental extinction, they have not been explored there in as much detail. This article reviews recent research in our laboratory that has investigated three relapse effects that occur after the extinction of instrumental (operant) learning. In renewal, responding returns after extinction when the behavior is tested in a different context; in resurgence, responding recovers when a second response that has been reinforced during extinction of the first is itself put on extinction; and in rapid reacquisition, extinguished responding returns rapidly when the response is reinforced again. The results provide new insights into extinction and relapse, and are consistent with principles that have been developed to explain extinction and relapse as they occur after Pavlovian conditioning. Extinction of instrumental learning, like Pavlovian learning, involves new learning that is relatively dependent on the context for expression.

Facets of Pavlovian and operant extinction

Research on extinction is of fundamental importance in both Pavlovian and operant approaches to the experimental analysis of learning. Although these approaches are often motivated by different empirical and theoretical questions, extinction has emerged as a research area in which common themes unite the two approaches. In this review, we focus on some common considerations in the analysis of Pavlovian and operant extinction. These include methodological challenges and interpretational issues in analyzing behavior during and after extinction. We consider the different roles that theory has played in the development of research on extinction in these preparations and conclude with some attention to applications of extinction.

Modulation of fear memory by retrieval and extinction: a clue for memory deconsolidation

Memories are fragile and easily forgotten at first, but after a consolidation period of hours to weeks, are inscribed in our brains as stable traces, no longer vulnerable to conventional amnesic treatments. Retrieval of a memory renders it labile, akin to the early stages of consolidation. This phenomenon has been explored as memory reactivation, in the sense that the memory is temporarily 'deconsolidated', allowing a short time window for amnesic intervention. This window closes again after reconsolidation, which restores the stability of the memory. In contrast to this 'transient deconsolidation' and the short-spanned amnesic effects of consolidation blockers, some specific treatments can disrupt even consolidated memory, leading to apparent amnesia. We propose the term 'amnesic deconsolidation' to describe such processes that lead to disruption of consolidated memory and/or consolidated memory traces. We review studies of these 'amnesic deconsolidation' treatments that enhance memory extinction, alleviate relapse, and reverse learning-induced plasticity. The transient deconsolidation that memory retrieval induces and the amnesic deconsolidation that these regimes induce both seem to dislodge a component that stabilizes consolidated memory. Characterizing this component, at both molecular and network levels, will provide a key to developing clinical treatments for memory-related disorders and to defining the consolidated memory trace.

Temporal factors control hippocampal contributions to fear renewal after extinction

Fear can be extinguished by repeated exposure to a cue that signals threat. However, extinction does not erase fear, as an extinguished cue presented in a context distinct from that of extinction results in renewed fear of that cue. The hippocampus, which is involved in the formation of contextual representations, is a natural candidate structure for investigations into the neural circuitry underlying fear renewal. Thus far, studies examining the necessity of the hippocampus for fear renewal have produced mixed results. We isolated the conditions under which the hippocampus may be required for renewal. Rats received lesions of the dorsal hippocampus either prior to tone fear conditioning or following extinction. Fear renewal was measured using discrete tone presentations or a long, continuous tone. The topography of fear responding at test was assessed by comparing "early" and "sustained" renewal, where early fear was determined by freezing to the first discrete tone or the equivalent initial segment of a continuous tone and sustained fear was determined by freezing averaged across all discrete tones or the entire continuous tone. We found that following pretraining damage of the hippocampus, early renewal remained intact regardless of lesion condition. However, sustained renewal only persisted in discrete, but not continuous, tone-tested animals. A more extensive analysis of the topography of fear responding revealed that the disruption of renewal was generated when the tone duration at test began to violate that used during extinction, suggesting that the hippocampus is sensitive to mismatches in CS-duration. Postextinction lesions resulted in an overall reduction of fear renewal. This pattern of results is consistent with those observed for contextual fear conditioning, wherein animals display a resistance to anterograde amnesia despite the presence of a strong retrograde amnesia for the same contextual information. Furthermore, the data support a role for the hippocampus in sustaining renewal when the CS duration at test does not match that used during extinction.

Intertrial interval as a contextual stimulus: further analysis of a novel asymmetry in temporal discrimination learning

Four experiments investigated discrimination learning when the duration of the intertrial interval (ITI) signaled whether or not the next conditional stimulus (CS) would be paired with food pellets. Rats received presentations of a 10-s CS separated half the time by long ITIs and half the time by short ITIs. When the long ITI signaled that the CS would be reinforced and the short interval signaled that it would not be (Long+/Short-), rats learned the discrimination readily. However, when the short ITI signaled that the CS would be reinforced and the long interval signaled that it would not (Short+/Long-), discrimination learning was much slower. Experiment 1 compared Long+/Short- and Short+/Long- discrimination learning with 16-min/4-min or 4-min/1-min ITI combinations. Experiment 2 found no evidence that Short+/Long- learning is inferior because the temporal cue corresponding to the short interval is ambiguous. Experiment 3 found no evidence that Short+/Long- learning is poor because the end of a long ITI signals a substantial reduction in delay to the next reinforcer. Long+/Short- learning may be faster than Short+/Long-because elapsing time involves exposure to a sequence of hypothetical stimulus elements (e.g., A then B), and feature-positive discriminations (AB+/A-) are learned quicker than feature-negative discriminations (A+/AB-). Consistent with this view, Experiment 4 found a robust feature-positive effect when sequentially presented CSs played the role of elements A and B.

Learning and the persistence of appetite: extinction and the motivation to eat and overeat

The modern world is saturated with highly palatable and highly available food, providing many opportunities to associate food with environmental cues and actions (through Pavlovian and operant or instrumental learning, respectively). Basic learning processes can often increase the tendency to approach and consume food, whereas extinction, in which Pavlovian and operant behaviors decline when the reinforcer is withheld, weakens but does not erase those tendencies. Contemporary research suggests that extinction involves an inhibitory form of new learning that appears fragile because it is highly dependent on the context for expression. These ideas are supported by the phenomena of renewal, spontaneous recovery, resurgence, reinstatement, and rapid reacquisition in appetitive learning, which together may help explain why overeating may be difficult to suppress permanently, and why appetitive behavior may seem so persistent.

Interstimulus interval as a discriminative stimulus: evidence of the generality of a novel asymmetry in temporal discrimination learning

Three appetitive conditioning experiments with rats examined temporal discrimination learning within Pavlovian conditioning trials. In all experiments, the duration of a feature white noise stimulus signaled whether or not a subsequent 10-s target tone would be reinforced. In Experiment 1, the feature durations were 4 and 1min. For one group of rats (Group 4+/1-), 4min of noise signaled that the tone would be reinforced and 1min of noise signaled that the tone would not be reinforced. A second group (Group 1+/4-) was trained with the reverse contingency. The results showed a clear asymmetry in temporal discrimination learning: rats trained with 4+/1- (Long+/Short-) learned the discrimination readily (responding more in the tone on reinforced than on nonreinforced trials), whereas rats trained with 1+/4- (Short+/Long) did not. In Experiment 2, the feature durations were shortened to 60 and 15s. Due to strong excitatory conditioning of the 15-s feature, the reverse asymmetry was observed, with the Short+/Long- discrimination learned more readily than the Long+/Short- discrimination. However, Experiment 3 demonstrated that the original Long+/Short- advantage could be recovered while using 60- and 15-s feature durations if the excitatory conditioning of the feature was reduced by including nonreinforced feature trials. The results support previous research involving the timing of intertrial intervals and are consistent with the temporal elements hypothesis which holds that the passage of time is encoded as a series of hypothetical stimulus elements.

Early extinction after fear conditioning yields a context-independent and short-term suppression of conditional freezing in rats

Extinction of Pavlovian fear conditioning in rats is a useful model for therapeutic interventions in humans with anxiety disorders. Recently, we found that delivering extinction trials soon (15 min) after fear conditioning yields a short-term suppression of fear, but little long-term extinction. Here, we explored the possible mechanisms underlying this deficit by assessing the suppression of fear to a CS immediately after extinction training (Experiment 1) and the context specificity of fear after both immediate and delayed extinction training (Experiment 2). We also examined the time course of the immediate extinction deficit (Experiment 3). Our results indicate that immediate extinction produces a short-lived and context-independent suppression of conditional freezing. Deficits in long-term extinction were apparent even when the extinction trials were given up to 6 h after conditioning. Moreover, this deficit was not due to different retention intervals that might have influenced the degree of spontaneous recovery after immediate and delayed extinction (Experiment 4). These results suggest that fear suppression under immediate extinction may be due to a short-term, context-independent habituation process, rather than extinction per se. Long-term extinction memory only develops when extinction training occurs at least six hours after conditioning.

Immediate extinction causes a less durable loss of performance than delayed extinction following either fear or appetitive conditioning

Five experiments with rat subjects compared the effects of immediate and delayed extinction on the durability of extinction learning. Three experiments examined extinction of fear conditioning (using the conditioned emotional response method), and two experiments examined extinction of appetitive conditioning (using the food-cup entry method). In all experiments, conditioning and extinction were accomplished in single sessions, and retention testing took place 24 h after extinction. In both fear and appetitive conditioning, immediate extinction (beginning 10 min after conditioning) caused a faster loss of responding than delayed extinction (beginning 24 h after conditioning). However, immediate extinction was less durable than delayed extinction: There was stronger spontaneous recovery during the final retention test. There was also substantial renewal of responding when the physical context was changed between immediate extinction and testing (Experiment 1). The results suggest that, in these two widely used conditioning preparations, immediate extinction does not erase or depotentiate the original learning, and instead creates a less permanent reduction in conditioned responding. Results did not support the possibility that the strong recovery after immediate extinction was due to a mismatch in the recent "context" provided by the presence or absence of a recent conditioning experience. Several other accounts are considered.

Learning and memory in conditioned fear extinction: effects of D-cycloserine

This review addresses the effects of the cognitive enhancer D-cycloserine (DCS) on the memory processes that occur in conditioned fear extinction, which is the experimental model for exposure techniques to reduce clinical anxiety. All reported rat studies show an enhanced fear extinction effect when DCS is administered acutely before or shortly after extinction training. DCS also promotes the generalization of this fear extinction effect. In addition, DCS reduces some forms of relapse (reduced reinstatement, reduced spontaneous recovery), but not others (contextual renewal, rapid reacquisition). It is argued that this pattern of results is best explained by assuming that DCS promotes extinction learning to the background context, resulting in enhanced contextual inhibition. Four human studies have produced mixed results, but some methodological issues complicate the reported failures. It is concluded that DCS is a promising tool as an adjunct to extinction techniques in exposure treatment, but that more pre-clinical and clinical research is needed to fully characterize its behavioral consequences.

Hippocampal involvement in contextual modulation of fear extinction

Extinction of fear conditioning in animals is an excellent model for the study of fear inhibition in humans. Substantial evidence has shown that extinction is a new learning process that is highly context-dependent. Several recovery effects (renewal, spontaneous recovery, and reinstatement) after extinction suggest that the contextual modulation of extinction is a critical behavioral mechanism underlying fear extinction. In addition, recent studies demonstrate a critical role for hippocampus in the context control of extinction. A growing body of evidence suggests that the hippocampus not only plays a role in contextual encoding and retrieval of fear extinction memories, but also interacts with other brain structures to regulate context-specificity of fear extinction. In this article, the authors will first discuss the fundamental behavioral features of the context effects of extinction and its underlying behavioral mechanisms. In the second part, the review will focus on the brain mechanisms for the contextual control of extinction.

The histone deacetylase inhibitor valproic acid enhances acquisition, extinction, and reconsolidation of conditioned fear

Histone modifications contribute to the epigenetic regulation of gene expression, a process now recognized to be important for the consolidation of long-term memory. Valproic acid (VPA), used for many years as an anticonvulsant and a mood stabilizer, has effects on learning and memory and enhances the extinction of conditioned fear through its function as a histone deacetylase inhibitor (HDAC). Here we report that VPA enhances long-term memory for both acquisition and extinction of cued-fear. Interestingly, VPA enhances extinction, but also enhances renewal of the original conditioned fear when tested in a within-subjects design. This effect appears to be related to a reconsolidation-like process since a single CS reminder in the presence of VPA can enhance long-term memory for the original fear in the context in which fear conditioning takes place. We also show that by modifying the intertrial interval during extinction training, VPA can strengthen reconsolidation of the original fear memory or enhance long-term memory for extinction such that it becomes independent of context. These findings have important implications for the use of HDAC inhibitors as adjuncts to behavior therapy in the treatment of phobia and related anxiety disorders.

Effects of ethanol on encoding, consolidation, and expression of extinction following contextual fear conditioning

Studies of contextual fear conditioning have found that ethanol administered prior to a conditioning session impairs the conditioned freezing response during a test session the next day. The present experiments examined the effects of ethanol on extinction, the loss of conditioned responding that occurs as the animal learns that a previously conditioned context no longer signals shock. Ethanol (1.5 g/kg) administered prior to single (Experiment 1) or multiple (Experiment 2) extinction sessions impaired extinction. Ethanol administered prior to a test session disrupted the expression of freezing after extinction (Experiments 3-5). There was some evidence that ethanol served as an internal stimulus signaling the operation of conditioning or extinction contingencies (Experiments 4-5). In Experiment 6, postsession injections of 1.5 g/kg ethanol had no effect on extinction with brief (3 min) or long (24 min) exposures to the context, but injections of 3 g/kg after long exposures impaired extinction. Together, these results indicate that ethanol affects extinction by acting on multiple learning and performance processes, including attention, memory encoding, and memory expression.

The effects of hippocampal system lesions on a novel temporal discrimination task for rats

A novel, appetitive, Pavlovian conditioning task was used to assess interval timing. Experiment 1 showed that normal rats could discriminate between tones of 1.5s and 0.5s duration, or between tones of 12.0 s and 3.0 s duration. Learning was demonstrated by a greater duration of magazine responding in the period before the delivery of a food reward and after cessation of the CS+ compared to the same time period after cessation of the CS-. Learning was, however, asymmetric as it was much quicker when the CS+ was the longer of the two durations (1.5s and 12.0 s, respectively). Experiment 2 assessed the impact of fornix lesions on the acquisition of one version of this task (CS+ 1.5s, CS- 0.5s). No evidence was found of a change in discrimination learning following surgery. Experiment 3 examined whether rats with either fornix or hippocampal lesions affected discriminations between 12.0 s and 3.0 s stimuli. Again, there was no evidence of a lesion-induced deficit. T-maze alternation training confirmed the effectiveness of these lesions. The results not only reveal that neither the fornix nor the hippocampus is necessary for distinguishing temporal intervals within the ranges tested but also showed how under some circumstances these lesions can leave trace conditioning intact.

Contextual and temporal modulation of extinction: behavioral and biological mechanisms

Extinction depends, at least partly, on new learning that is specific to the context in which it is learned. Several behavioral phenomena (renewal, reinstatement, spontaneous recovery, and rapid reacquisition) suggest the importance of context in extinction. The present article reviews research on the behavioral and neurobiological mechanisms of contextual influences on extinction learning and retrieval. Contexts appear to select or retrieve the current relationship of the conditional stimulus (CS) with the unconditional stimulus (US), and they are provided by physical background cues, interoceptive drug cues, emotions, recent trials, and the passage of time. The current article pays particular attention to the effects of recent trials and trial spacing. Control of fear extinction by physical context involves interactions between the dorsal hippocampus and the lateral nucleus of the amygdala. This interaction may be mediated by gamma-aminobutyric acid (GABA)-ergic and adrenergic mechanisms.