Experimental extinction in Pavlovian conditioning: Behavioural and neuroscience perspectives

Effects of dorsal hippocampal orexin-2 receptor antagonism on the acquisition, expression, and extinction of morphine-induced place preference in rats

RATIONALE

Orexinergic system is involved in reward processing and drug addiction.

OBJECTIVES

Here, we investigated the effect of intrahippocampal CA1 administration of orexin-2 receptor (OX2r) antagonist on the acquisition, expression, and extinction of morphine-induced place preference in rats.

METHODS

Conditioned place preference (CPP) was induced by subcutaneous injection of morphine (5 mg/kg) during a 3-day conditioning phase. Three experimental plots were designed; TCS OX2 29 as a selective antagonist of orexin-2 receptors (OX2rs) was dissolved in DMSO, prepared in solutions with different concentrations (1, 3, 10, and 30 nM), and was bilaterally microinjected into the CA1 and some neighboring regions (0.5 μl/side). Conditioning scores and locomotor activities were recorded during the test.

RESULTS

Results demonstrate that intra-CA1 administration of the OX2r antagonist attenuates the induction of morphine CPP during the acquisition and expression phases. Effect of TCS OX2 29 on reduction of morphine CPP was dose-dependent and was more pronounced during the acquisition than the expression. Furthermore, higher concentrations of TCS OX2 29 facilitated the extinction of morphine-induced CPP and reduced extinction latency period. Nevertheless, administration of TCS OX2 29 solutions did not have any influence on locomotor activity of all phases.

CONCLUSIONS

Our findings suggest that OX2rs in the CA1 region of hippocampus are involved in the development of the acquisition and expression of morphine CPP. Moreover, blockade of OX2rs could facilitate extinction and may abrogate or extinguish the ability of drug-related cues, implying that the antagonist might be considered as a propitious therapeutic agent in suppressing drug-seeking behavior.

Impaired fear extinction in adolescent rodents: Behavioural and neural analyses

Despite adolescence being a developmental window of vulnerability, up until very recently there were surprisingly few studies on fear extinction during this period. Here we summarise the recent work in this area, focusing on the unique behavioural and neural characteristics of fear extinction in adolescent rodents, and humans where relevant. A prominent hypothesis posits that anxiety disorders peak during late childhood/adolescence due to the non-linear maturation of the fear inhibition neural circuitry. We discuss evidence that impaired extinction retention in adolescence is due to subregions of the medial prefrontal cortex and amygdala mediating fear inhibition being underactive while other subregions that mediate fear expression are overactive. We also review work on various interventions and surprising circumstances which enhance fear extinction in adolescence. This latter work revealed that the neural correlates of extinction in adolescence are different to that in younger and older animals even when extinction retention is not impaired. This growing body of work highlights that adolescence is a unique period of development for fear inhibition.

Counterconditioned fear responses exhibit greater renewal than extinguished fear responses

This series of experiments used rats to compare counterconditioning and extinction of conditioned fear responses (freezing) with respect to the effects of a context shift. In each experiment, a stimulus was paired with shock in context A, extinguished or counterconditioned through pairings with sucrose in context B, and then tested for renewal outside of context B. Counterconditioned fear responses exhibited greater ABA renewal than extinguished fear responses. This result was observed using a between-subjects design (Experiment 1) and a within-subject design in which counterconditioned and extinguished stimuli were equated in all respects other than their signaling of sucrose (Experiment 2). Counterconditioned fear responses also exhibited greater ABC renewal than extinguished fear responses (Experiment 3). This result was observed using a within-subject design in which context C was identical to context B in terms of its associative history, and when counterconditioned and extinguished CSs were tested in compounds matched for their association with both shock and sucrose (Experiment 4). These results are consistent with models which hold that context regulates expression of associations formed in counterconditioning and extinction, and allow the level of regulation to be greater following counterconditioning than extinction, as noted in previous studies.

The effect of a dopamine antagonist on conditioning of sexual arousal in women

RATIONALE

Dopamine (DA) plays a key role in reward-seeking behaviours. Accumulating evidence from animal and human studies suggests that human sexual reward learning may also depend on DA transmission. However, research on the role of DA in human sexual reward learning is completely lacking.

OBJECTIVES

To investigate whether DA antagonism attenuates classical conditioning of sexual response in humans.

METHODS

Healthy women were randomly allocated to one of two treatment conditions: haloperidol (n = 29) or placebo (n = 29). A differential conditioning paradigm was applied with genital vibrostimulation as unconditional stimulus (US) and neutral pictures as conditional stimuli (CSs). Genital arousal was assessed, and ratings of affective value and subjective sexual arousal were obtained.

RESULTS

Haloperidol administration affected unconditional genital responding. However, no significant effects of medication were found for conditioned responding.

CONCLUSIONS

No firm conclusions can be drawn about whether female sexual reward learning implicates DA transmission since the results do not lend themselves to unambiguous interpretation.

The Memory System Engaged During Acquisition Determines the Effectiveness of Different Extinction Protocols

Previous research indicates that extinction of rodent maze behavior may occur without explicit performance of the previously acquired response. In latent extinction, confining an animal to a previously rewarded goal location without reinforcement is typically sufficient to produce extinction of maze learning. However, previous studies have not determined whether latent extinction may be successfully employed to extinguish all types of memory acquired in the maze, or whether only specific types of memory may be vulnerable to latent extinction. The present study examined whether latent extinction may be effective across two plus-maze tasks that depend on anatomically distinct neural systems. Adult male Long-Evans rats were trained in a hippocampus-dependent place learning task (Experiment 1), in which animals were trained to approach a consistent spatial location for food reward. A separate group of rats were trained in a dorsolateral striatum-dependent response learning task (Experiment 2), in which animals were trained to make a consistent egocentric body-turn response for food reward. Following training, animals received response extinction or latent extinction. For response extinction, animals were given the opportunity to execute the original running approach response toward the empty food cup. For latent extinction, animals were confined to the original goal locations with the empty food cup, thus preventing them from making the original running approach response. Results indicate that, relative to no extinction, latent extinction was effective at extinguishing memory in the place learning task, but remained ineffective in the response learning task. In contrast, typical response extinction remained very effective at extinguishing memory in both place and response learning tasks. The present findings confirm that extinction of maze learning may occur with or without overt performance of the previously acquired response, but that the effectiveness of latent extinction may depend on the type of memory being extinguished. The findings suggest that behavioral treatments modeled after response extinction protocols may be especially useful in alleviating human psychopathologies involving striatum-dependent memory processes (e.g., drug addiction and relapse).

PARP-1 activity is required for the reconsolidation and extinction of contextual fear memory

BACKGROUND

Memory consolidation, reconsolidation, and extinction have been shown to require new gene expression. Poly ADP-ribosylation mediated by poly (ADP-ribose) polymerase-1 (PARP-1) is known to regulate transcription through histone modification. Recent studies have suggested that PARP-1 positively regulates the formation of long-term memory (LTM); however, the roles of PARP-1 in memory processes, especially processes after retrieval, remain unknown.

RESULTS

Here, we show critical roles for PARP-1 in the consolidation, reconsolidation, and extinction of contextual fear memory in mice. We examined the effects of pharmacological inhibition of PARP-1 activity in the hippocampus or medial prefrontal cortex (mPFC) on these memory processes. Similarly with previous findings, a micro-infusion of the PARP-1 inhibitor 3-aminobenzamide or PJ34 into the dorsal hippocampus, but not mPFC, impaired LTM formation without affecting short-term memory (STM). Importantly, this pharmacological blockade of PARP-1 in the dorsal hippocampus, but not mPFC, also disrupted post-reactivation LTM without affecting post-reactivation STM. Conversely, micro-infusion of the PARP-1 inhibitors into the mPFC, but not dorsal hippocampus, blocked long-term extinction. Additionally, systemic administration of the PARP-1 inhibitor Tiq-A blocked c-fos induction in the hippocampus, which is observed when memory is consolidated or reconsolidated, and also blocked c-fos induction in the mPFC, which is observed when memory is extinguished.

CONCLUSIONS

Our observations showed that PARP-1 activation is required for the consolidation, reconsolidation, and extinction of contextual fear memory and suggested that PARP-1 contributes to the new gene expression necessary for these memory processes.

Out with the old and in with the new: Synaptic mechanisms of extinction in the amygdala

Considerable research indicates that long-term synaptic plasticity in the amygdala underlies the acquisition of emotional memories, including those learned during Pavlovian fear conditioning. Much less is known about the synaptic mechanisms involved in other forms of associative learning, including extinction, that update fear memories. Extinction learning might reverse conditioning-related changes (e.g., depotentiation) or induce plasticity at inhibitory synapses (e.g., long-term potentiation) to suppress conditioned fear responses. Either mechanism must account for fear recovery phenomena after extinction, as well as savings of extinction after fear recovery. This article is part of a Special Issue entitled SI: Brain and Memory.

Prefrontal and Auditory Input to Intercalated Neurons of the Amygdala

The basolateral amygdala (BLA) and prefrontal cortex (PFC) are partners in fear learning and extinction. Intercalated (ITC) cells are inhibitory neurons that surround the BLA. Lateral ITC (lITC) neurons provide feed-forward inhibition to BLA principal neurons, whereas medial ITC (mITC) neurons form an inhibitory interface between the BLA and central amygdala (CeA). Notably, infralimbic prefrontal (IL) input to mITC neurons is thought to play a key role in fear extinction. Here, using targeted optogenetic stimulation, we show that lITC neurons receive auditory input from cortical and thalamic regions. IL inputs innervate principal neurons in the BLA but not mITC neurons. These results suggest that (1) these neurons may play a more central role in fear learning as both lITCs and mITCs receive auditory input and that (2) mITC neurons cannot be driven directly by the IL, and their role in fear extinction is likely mediated via the BLA.

Positive Arousal Enhances the Consolidation of Item Memory

Emotional arousal induced after learning has been shown to modulate memory consolidation. However, it is unclear whether the effect of postlearning arousal can extend to different aspects of memory. This study examined the effect of postlearning positive arousal on both item memory and source memory. Participants learned a list of neutral words and took an immediate memory test. Then they watched a positive or a neutral videoclip and took delayed memory tests after either 25 minutes or 1 week had elapsed after the learning phase. In both delay conditions, positive arousal enhanced consolidation of item memory as measured by overall recognition. Furthermore, positive arousal enhanced consolidation of familiarity but not recollection. However, positive arousal appeared to have no effect on consolidation of source memory. These findings have implications for building theoretical models of the effect of emotional arousal on consolidation of episodic memory and for applying postlearning emotional arousal as a technique of memory intervention.

Identifying profiles of recovery from reward devaluation in rats

In humans and other mammals, the unexpected loss of a resource can lead to emotional conflict. Consummatory successive negative contrast (cSNC) is a laboratory model of reward devaluation meant to capture that conflict. In this paradigm, animals are exposed to a sharp reduction in the sucrose concentration of a solution after several days of access. This downshift in sucrose content leads to behavioral responses such as the suppression of consumption and physiologic responses including elevation of corticosterone levels. However, response heterogeneity in cSNC has yet to be explored and may be relevant for increasing the validity of this model, as humans demonstrate clinically meaningful heterogeneity in response to resource loss. The current analysis applied latent growth mixture modeling to test for and characterize heterogeneity in recovery from cSNC among rats (N=262). Although most animals exhibited recovery of consummatory behavior after a sharp drop in consumption in the first postshift trial (Recovery class; 83%), two additional classes were identified including animals that did not change their consumption levels after downshift (No Contrast class; 6%), and animals that exhibited an initial response similar to that of the Recovery class but did not recover to preshift consumption levels (No Recovery class; 11%). These results indicate heterogeneity in recovery from reward loss among rats, which may increase the translatability of this animal model to understand diverse responses to loss among humans.

Behavioral neuroscience of psychological pain

Pain is a common word used to refer to a wide range of physical and mental states sharing hedonic aversive value. Three types of pain are distinguished in this article: Physical pain, an aversive state related to actual or potential injury and disease; social pain, an aversive emotion associated to social exclusion; and psychological pain, a negative emotion induced by incentive loss. This review centers on psychological pain as studied in nonhuman animals. After covering issues of terminology, the article briefly discusses the daily-life significance of psychological pain and then centers on a discussion of the results originating from two procedures involving incentive loss: successive negative contrast-the unexpected devaluation of a reward-and appetitive extinction-the unexpected omission of a reward. The evidence reviewed points to substantial commonalities, but also some differences and interactions between physical and psychological pains. This evidence is discussed in relation to behavioral, pharmacological, neurobiological, and genetic factors that contribute to the multidimensional experience of psychological pain.

Toward a translational approach to targeting the endocannabinoid system in posttraumatic stress disorder: a critical review of preclinical research

Despite the lack of clinical research, marijuana and synthetic cannabinoids have been approved to treat posttraumatic stress disorder (PTSD) in several states in the United States. This review critically examines preclinical research on the endocannabinoid system (ECS) in order to evaluate three key questions that are relevant to PTSD: (1) Does ECS dysfunction impact fear extinction? (2) Can stress-related symptoms be prevented by ECS modulation? (3) Is the ECS a potential target for enhancing PTSD treatment? Disruption of the ECS impaired fear extinction in rodents, and ECS abnormalities have been observed in PTSD. Targeting fear memories via the ECS had mixed results in rodents, whereas augmented cannabinoid receptor activation typically facilitated extinction. However, the translational value of these findings is limited by the paucity and inconsistency of human research. Further investigation is necessary to determine whether incorporating cannabinoids in treatment would benefit individuals with PTSD, with cautious attention to risks.

Extinction and renewal of conditioned sexual responses

Introduction

Extinction involves an inhibitory form of new learning that is highly dependent on the context for expression. This is supported by phenomena such as renewal and spontaneous recovery, which may help explain the persistence of appetitive behavior, and related problems such as addictions. Research on these phenomena in the sexual domain is lacking, where it may help to explain the persistence of learned sexual responses.

Method

Men (n = 40) and women (n = 62) participated in a differential conditioning paradigm, with genital vibrotactile stimulation as US and neutral pictures as conditional stimuli (CSs). Dependent variables were genital and subjective sexual arousal, affect, US expectancy, and approach and avoid tendencies towards the CSs. Extinction and renewal of conditioned sexual responses were studied by context manipulation (AAA vs. ABA condition).

Results

No renewal effect of genital conditioned responding could be detected, but an obvious recovery of US expectancy following a context change after extinction (ABA) was demonstrated. Additionally, women demonstrated recovery of subjective affect and subjective sexual arousal. Participants in the ABA demonstrated more approach biases towards stimuli.

Conclusions

The findings support the context dependency of extinction and renewal of conditioned sexual responses in humans. This knowledge may have implications for the treatment of disturbances in sexual appetitive responses such as hypo- and hypersexuality.

Behavioral and neural bases of extinction learning in Hermissenda

Extinction of classical conditioning is thought to produce new learning that masks or interferes with the original memory. However, research in the nudibranch Hermissenda crassicornis (H.c.) has challenged this view, and instead suggested that extinction erased the original associative memory. We have re-examined extinction in H.c. to test whether extinguished associative memories can be detected on the behavioral and cellular levels, and to characterize the temporal variables involved. Associative conditioning using pairings of light (CS) and rotation (US) produced characteristic suppression of H.c. phototactic behavior. A single session of extinction training (repeated light-alone presentations) reversed suppressed behavior back to pre-training levels when administered 15 min after associative conditioning. This effect was abolished if extinction was delayed by 23 h, and yet was recovered using extended extinction training (three consecutive daily extinction sessions). Extinguished phototactic suppression did not spontaneously recover at any retention interval (RI) tested (2-, 24-, 48-, 72-h), or after additional US presentations (no observed reinstatement). Extinction training (single session, 15 min interval) also reversed the pairing-produced increases in light-evoked spike frequencies of Type B photoreceptors, an identified site of associative memory storage that is causally related to phototactic suppression. These results suggest that the behavioral effects of extinction training are not due to temporary suppression of associative memories, but instead represent a reversal of the underlying cellular changes necessary for the expression of learning. In the companion article, we further elucidate mechanisms responsible for extinction-produced reversal of memory-related neural plasticity in Type B photoreceptors.

Blockade of CB1 receptors prevents retention of extinction but does not increase low preincubated conditioned fear in the fear incubation procedure

We recently developed a procedure to study fear incubation, in which rats given 100 tone-shock pairings over 10 days show low fear 2 days after conditioned fear training and high fear after 30 days. Notably, fear 2 days after 10 sessions of fear conditioning is lower than fear seen 2 days after a single session of fear conditioning, suggesting that fear is suppressed. Here, we investigate the potential role of CB1 receptor activation by endocannabinoids in this fear suppression. We subjected rats to 10 days of fear conditioning and then administered systemic injections of the CB1 receptor antagonist SR141716 before a conditioned fear test was conducted 2 days later under extinction conditions. A second test was conducted without any injections on the following day (3 days after training) to examine retention of fear extinction. SR141716 injections did not increase fear expression 2 days after extended fear conditioning or affect within-session extinction; however, it impaired retention of between-session fear extinction in the day 3 test. These data suggest that CB1 receptor activation does not suppress fear soon after extended fear conditioning in the fear incubation task. The data also add to the existing literature on the role of CB1 receptors in extinction of conditioned fear.

Prefrontal oscillations during recall of conditioned and extinguished fear in humans

Human neuroimaging studies indicate that the anterior midcingulate cortex (AMC) and the ventromedial prefrontal cortex (vmPFC) play important roles in the expression and extinction of fear, respectively. Electrophysiological rodent studies further indicate that oscillatory neuronal activity in homolog regions (i.e., prelimbic and infralimbic cortices) changes during fear expression and fear extinction recall. Whether similar processes occur in humans remains largely unexplored. By assessing scalp surface EEG in conjunction with LORETA source estimation of CS-related theta and gamma activity, we tested whether a priori defined ROIs in the human AMC and vmPFC similarly modulate their oscillatory activity during fear expression and extinction recall, respectively. To this end, 42 healthy individuals underwent a differential conditioning/differential extinction protocol with a Recall Test on the next day. In the Recall Test, nonextinguished versus extinguished stimuli evoked an increased differential (CS(+) vs CS(-)) response with regard to skin conductance and AMC-localized theta power. Conversely, extinguished versus nonextinguished stimuli evoked an increased differential response with regard to vmPFC-localized gamma power. Finally, individuals who failed to show a suppressed skin conductance response to the extinguished versus nonextinguished CS(+) also failed to show the otherwise observed alterations in vmPFC gamma power to extinguished CS(+). These results indicate that fear expression is associated with AMC theta activity, whereas successful fear extinction recall relates to changes in vmPFC gamma activity. The present work thereby bridges findings from prior rodent electrophysiological research and human neuroimaging studies and indicates that EEG is a valuable tool for future fear extinction research.

Effect of time delay on recognition memory for pictures: the modulatory role of emotion

This study investigated the modulatory role of emotion in the effect of time delay on recognition memory for pictures. Participants viewed neutral, positive and negative pictures, and took a recognition memory test 5 minutes, 24 hours, or 1 week after learning. The findings are: 1) For neutral, positive and negative pictures, overall recognition accuracy in the 5-min delay did not significantly differ from that in the 24-h delay. For neutral and positive pictures, overall recognition accuracy in the 1-week delay was lower than in the 24-h delay; for negative pictures, overall recognition in the 24-h and 1-week delay did not significantly differ. Therefore negative emotion modulates the effect of time delay on recognition memory, maintaining retention of overall recognition accuracy only within a certain frame of time. 2) For the three types of pictures, recollection and familiarity in the 5-min delay did not significantly differ from that in the 24-h and the 1-week delay. Thus emotion does not appear to modulate the effect of time delay on recollection and familiarity. However, recollection in the 24-h delay was higher than in the 1-week delay, whereas familiarity in the 24-h delay was lower than in the 1-week delay.

Fear conditioning and extinction across development: evidence from human studies and animal models

The ability to differentiate danger and safety through associative processes emerges early in life. Understanding the mechanisms underlying associative learning of threat and safety can clarify the processes that shape development of normative fears and pathological anxiety. Considerable research has used fear conditioning and extinction paradigms to delineate underlying mechanisms in animals and human adults; however, little is known about these mechanisms in children and adolescents. The current paper summarizes the empirical data on the development of fear conditioning and extinction. It reviews methodological considerations and future directions for research on fear conditioning and extinction in pediatric populations.

Dorsal hippocampus inactivation impairs spontaneous recovery of Pavlovian magazine approach responding in rats

Destruction or inactivation of the dorsal hippocampus (DH) has been shown to eliminate the renewal of extinguished fear [1-4]. However, it has recently been reported that the contextual control of responding to extinguished appetitive stimuli is not disrupted when the DH is destroyed or inactivated prior to tests for renewal of Pavlovian conditioned magazine approach [5]. In the present study we extend the analysis of DH control of appetitive extinction learning to the spontaneous recovery of Pavlovian conditioned magazine approach responding. Subjects were trained to associate two separate stimuli with the delivery of food and had muscimol or vehicle infused into the DH prior to a single test-session for spontaneous recovery occurring immediately following extinction of one of these stimuli, but one week following extinction of the other. While vehicle treated subjects showed more recovery to the distally extinguished stimulus than the proximal one, muscimol treated subjects failed to show spontaneous recovery to either stimulus. This result suggests that, while the DH is not involved in the control of extinction by physical contexts [5], it may be involved when time is the gating factor controlling recovery of extinguished responding.

Mechanisms of renewal after the extinction of discriminated operant behavior

Three experiments demonstrated, and examined the mechanisms that underlie, the renewal of extinguished discriminated operant behavior. In Experiment 1, rats were trained to perform 1 response (lever press or chain pull) in the presence of one discriminative stimulus (S; light or tone) in Context A, and to perform the other response in the presence of the other S in Context B. Next, each of the original S/response combinations was extinguished in the alternate context. When the S/response combinations were tested back in the context in which they had been trained, responding in the presence of S returned (an ABA renewal effect was observed). This renewal could not be due to differential context-reinforcer associations, suggesting instead that the extinction context inhibits either the response and/or the effectiveness of the S. Consistent with the latter mechanism, in Experiment 2, ABA renewal was still observed when both the extinction and renewal contexts inhibited the same response. However, in Experiment 3, previous extinction of the response in the renewing context (occasioned by a different S) reduced AAB renewal more than did extinction of the different response. Taken together, the results suggest at least 2 mechanisms of renewal after instrumental extinction. First, extinction performance is at least partly controlled by a direct inhibitory association that is formed between the context and the response. Second, in the discriminated operant procedure, extinction performance can sometimes be partly controlled by a reduction in the effectiveness of the S in the extinction context. Renewal of discriminated operant behavior can be produced by a release from either of these forms of inhibition.

Psychological and neural mechanisms of experimental extinction: a selective review

The present review examines key psychological concepts in the study of experimental extinction and implications these have for an understanding of the underlying neurobiology of extinction learning. We suggest that many of the signature characteristics of extinction learning (spontaneous recovery, renewal, reinstatement, rapid reacquisition) can be accommodated by the standard associative learning theory assumption that extinction results in partial erasure of the original learning together with new inhibitory learning. Moreover, we consider recent behavioral and neural evidence that supports the partial erasure view of extinction, but also note shortcomings in our understanding of extinction circuits as these relate to the negative prediction error concept. Recent work suggests that common prediction error and stimulus-specific prediction error terms both may be required to explain neural plasticity both in acquisition and extinction learning. In addition, we suggest that many issues in the content of extinction learning have not been fully addressed in current research, but that neurobiological approaches should be especially helpful in addressing such issues. These include questions about the nature of extinction learning (excitatory CS-No US, inhibitory CS-US learning, occasion setting processes), especially as this relates to studies of the micro-circuitry of extinction, as well as its representational content (sensory, motivational, response). An additional understudied problem in extinction research is the role played by attention processes and their underlying neural networks, although some research and theory converge on the idea that extinction is accompanied by attention decrements (i.e., habituation-like processes).

Fear of the unexpected: hippocampus mediates novelty-induced return of extinguished fear in rats

Several lines of evidence indicate an important role for the hippocampus in the recovery of fear memory after extinction. For example, hippocampal inactivation prevents the renewal of fear to an extinguished conditioned stimulus (CS) when it is presented outside the extinction context. Renewal of extinguished responding is accompanied by associative novelty (an unexpected occurrence of a familiar CS in a familiar place), the detection of which may require the hippocampus. We therefore examined whether the hippocampus also mediates the recovery of extinguished fear caused by other unexpected events, including presenting a familiar CS in a novel context or presenting a novel cue with the CS in a familiar context (e.g., external disinhibition). Rats underwent Pavlovian fear conditioning and extinction using an auditory CS and freezing behavior served as the index of conditioned fear. In Experiment 1, conditioned freezing to the extinguished CS was renewed in a novel context and this was eliminated by intra-hippocampal infusions of the GABAA agonist, muscimol, prior to the test. In Experiment 2, muscimol inactivation of the hippocampus reduced the external disinhibition of conditioned freezing that occurred when a novel white noise accompanied the extinguished tone CS. Collectively, these results suggest that the hippocampus mediates the return of fear when extinguished CSs are unexpected, or when unexpected stimuli accompany CS presentation. Ultimately, a violation of expectations about when, where, and with what other stimuli an extinguished CS will occur may form the basis of spontaneous recovery, renewal, and external disinhibition.

Spatial memory extinction: a c-Fos protein mapping study

While the neuronal basis of spatial memory consolidation has been thoroughly studied, the substrates mediating the process of extinction remain largely unknown. This study aimed to evaluate the functional contribution of selected brain regions during the extinction of a previously acquired spatial memory task in the Morris water maze. For that purpose, we used adult male Wistar rats trained in a spatial reference memory task. Learning-related changes in c-Fos inmunoreactive cells after training were evaluated in cortical and subcortical regions. Results show that removal of the hidden platform in the water maze induced extinction of the previously reinforced escape behavior after 16 trials, without spontaneous recovery 24h later. Extinction was related with significantly higher numbers of c-Fos positive nuclei in amygdala nuclei and prefrontal cortex. On the other hand, the lateral mammillary bodies showed higher number of c-Fos positive cells than the control group. Therefore, in contrast with the results obtained in studies of classical conditioning, we show the involvement of diencephalic structures mediating this kind of learning. In summary, our findings suggest that medial prefrontal cortex, the amygdala complex and diencephalic structures like the lateral mammillary nuclei are relevant for the extinction of spatial memory.

The role of conditioning, learning and dopamine in sexual behavior: a narrative review of animal and human studies

Many theories of human sexual behavior assume that sexual stimuli obtain arousing properties through associative learning processes. It is widely accepted that classical conditioning contributes to the etiology of both normal and maladaptive human behaviors. Despite the hypothesized importance of basic learning processes in sexual behavior, research on classical conditioning of the sexual response in humans is scarce. In the present paper, animal studies and studies in humans on the role of pavlovian conditioning on sexual responses are reviewed. Animal research shows robust, direct effects of conditioning processes on partner- and place preference. On the contrast, the empirical research with humans in this area is limited and earlier studies within this field are plagued by methodological confounds. Although recent experimental demonstrations of human sexual conditioning are neither numerous nor robust, sexual arousal showed to be conditionable in both men and women. The present paper serves to highlight the major empirical findings and to renew the insight in how stimuli can acquire sexually arousing value. Hereby also related neurobiological processes in reward learning are discussed. Finally, the connections between animal and human research on the conditionability of sexual responses are discussed, and suggestions for future directions in human research are given.

Nature and causes of the immediate extinction deficit: a brief review

Recent data in both rodents and humans suggests that the timing of extinction trials after conditioning influences the magnitude and duration of extinction. For example, administering extinction trials soon after Pavlovian fear conditioning in rats, mice, and humans results in minimal fear suppression - the so-called immediate extinction deficit. Here I review recent work examining the behavioral and neural substrates of the immediate extinction deficit. I suggest that extinction is most effective at some delay after conditioning, because brain systems involved in encoding and retrieving extinction memories function sub-optimally under stress.

Memory reconsolidation in aversive and appetitive settings

Memory reconsolidation has been observed across species and in a number of behavioral paradigms. The majority of memory reconsolidation studies have been carried out in Pavlovian fear conditioning and other aversive memory settings, with potential implications for the treatment of post-traumatic stress disorder. However, there is a growing literature on memory reconsolidation in appetitive reward-related memory paradigms, including translational models of drug addiction. While there appears to be substantial similarity in the basic phenomenon and underlying mechanisms of memory reconsolidation across unconditioned stimulus valence, there are also notable discrepancies. These arise both when comparing aversive to appetitive paradigms and also across different paradigms within the same valence of memory. We review the demonstration of memory reconsolidation across different aversive and appetitive memory paradigms, the commonalities and differences in underlying mechanisms and the conditions under which each memory undergoes reconsolidation. We focus particularly on whether principles derived from the aversive literature are applicable to appetitive settings, and also whether the expanding literature in appetitive paradigms is informative for fear memory reconsolidation.

Behavioral and neurobiological mechanisms of extinction in Pavlovian and instrumental learning

This article reviews research on the behavioral and neural mechanisms of extinction as it is represented in both Pavlovian and instrumental learning. In Pavlovian extinction, repeated presentation of a signal without its reinforcer weakens behavior evoked by the signal; in instrumental extinction, repeated occurrence of a voluntary action without its reinforcer weakens the strength of the action. In either case, contemporary research at both the behavioral and neural levels of analysis has been guided by a set of extinction principles that were first generated by research conducted at the behavioral level. The review discusses these principles and illustrates how they have informed the study of both Pavlovian and instrumental extinction. It shows that behavioral and neurobiological research efforts have been tightly linked and that their results are readily integrated. Pavlovian and instrumental extinction are also controlled by compatible behavioral and neural processes. Since many behavioral effects observed in extinction can be multiply determined, we suggest that the current close connection between behavioral-level and neural-level analyses will need to continue.

Orbitofrontal cortex inactivation impairs between- but not within-session Pavlovian extinction: an associative analysis

The orbitofrontal cortex (OFC) is argued to be the neural locus of Pavlovian outcome expectancies. Reinforcement learning theories argue that extinction learning in Pavlovian procedures is caused by the discrepancy between the expected value of the outcome (US) that is elicited by a predictive stimulus (CS), and the lack of experienced US. If the OFC represents Pavlovian outcome expectancies that are necessary for extinction learning, then disrupting OFC function prior to extinction training should impair extinction learning. This was tested. In experiment 1, Long Evans rats received infusions of saline or muscimol targeting the lateral OFC prior to three appetitive Pavlovian extinction sessions. Muscimol infused into the OFC disrupted between-session but not within-session extinction behaviour. This finding was not due to muscimol infusions disrupting the memory consolidation process per se as there was no effect of muscimol infusion when administered immediately post session (experiment 2). These findings support a role for the OFC in representing outcome expectancies that are necessary for learning. A number of ways in which disrupting outcome expectancy information might block learning will be discussed in the context of traditional associative learning theories and the associative structures they depend on.

Stress impairs retrieval of extinguished and unextinguished associations in a predictive learning task

Recovery effects which can frequently be observed after a seemingly successful extinction procedure indicate that extinction does not lead to an erasure of the memory trace. Investigating factors which modulate the retrieval of extinction memory is highly relevant for basic science and clinical applications alike. This study investigated the effect of stress on the retrieval of extinguished and unextinguished stimulus-outcome associations in a predictive learning task. In this task, participants had to imagine being the doctor of a patient who sometimes suffers from stomach trouble after meals in his favorite restaurants. They were presented with different food stimuli while having to predict the occurrence or non-occurrence of stomach trouble. As extinction memory is modulated by context, we manipulated contextual cues so that initial acquisition of critical associations occurred in context (restaurant frame) A on day one, whereas associations were reversed in context B (extinction, day two). On the third day, participants were either stressed (exposed to the socially evaluated cold pressor task (SECPT); n=21) or subjected to a control condition (n=21) shortly before extinction memory retrieval was tested (in contexts A and B). Salivary cortisol and blood pressure measures as well as subjective ratings indicated that stress induction was successful. When retrieval of extinguished associations was tested on day three, participants' predictions reflected a renewal effect, as indicated by stronger recovery of responding in the acquisition context compared to the extinction context. Compared to controls, stressed participants showed impaired retrieval of extinguished and unextinguished associations. Contextual cues abolished the stress-induced memory impairment for unextinguished but not for extinguished associations. These findings might help to explain why stress leads to the reoccurrence of symptoms in affective disorders.

ABA and ABC renewal of conditioned magazine approach are not impaired by dorsal hippocampus inactivation or lesions

Three experiments investigated the role of the dorsal hippocampus (DH) in renewal of conditioned and then extinguished magazine approach responding in rats. Experiments 1 and 2 found no effect of muscimol inactivation of the DH during testing on ABA and ABC renewal, respectively. However, subjects from these studies were subsequently found to be impaired on a delayed non-matching-to-place task following muscimol but not saline infusions. Experiment 3 found no effects of post-training excitotoxic lesions of the DH on ABA and ABC renewal. Lesioned subjects were, however, impaired on the delayed non-matching-to-place task compared to control subjects. These findings suggest that the DH may not play a similar role in Pavlovian extinction in appetitive learning tasks as has previously been reported in aversive learning.

Exposure to a fearful context during periods of memory plasticity impairs extinction via hyperactivation of frontal-amygdalar circuits

An issue of increasing theoretical and translational importance is to understand the conditions under which learned fear can be suppressed, or even eliminated. Basic research has pointed to extinction, in which an organism is exposed to a fearful stimulus (such as a context) in the absence of an expected aversive outcome (such as a shock). This extinction process results in the suppression of fear responses, but is generally thought to leave the original fearful memory intact. Here, we investigate the effects of extinction during periods of memory lability on behavioral responses and on expression of the immediate-early gene c-Fos within fear conditioning and extinction circuits. Our results show that long-term extinction is impaired when it occurs during time periods during which the memory should be most vulnerable to disruption (soon after conditioning or retrieval). These behavioral effects are correlated with hyperactivation of medial prefrontal cortex and amygdala subregions associated with fear expression rather than fear extinction. These findings demonstrate that behavioral experiences during periods of heightened fear prevent extinction and prolong the conditioned fear response.

Preferential responses to extinguished face stimuli are preserved in frontal and occipito-temporal cortex at initial but not later stages of processing

Magnetoencephalographic correlates of rapid emotional responses (50-80 ms) in frontal and occipito-temporal regions have recently been reported using a novel MultiCS Conditioning paradigm with odor-conditioned faces. As those short-latency responses were supposed to partially reflect initial access to nonextinguished emotional memories, it could be predicted that they outlast the extinction phase. To test this hypothesis, appetitively and aversively odor-conditioned faces were frequently presented during extinction while event-related magnetic fields were recorded. Affect-specific responses in frontal and occipito-temporal areas were found in the early (50-80 ms) but not in the later (130-190 ms) time interval following extinction learning. These results suggest that previously acquired emotional memories can be accessed at initial processing stages but become ineffective in modulating processing at later stages as extinction proceeds.

Appetitive Pavlovian goal-tracking memories reconsolidate only under specific conditions

Despite extensive evidence that appetitive memories undergo reconsolidation, two notable failures to observe reconsolidation have been reported: instrumental responding and goal-tracking. However, these studies do not provide conclusive evidence for a lack of memory reconsolidation due to the numerous boundary conditions that dictate whether a memory will undergo reconsolidation. In this study we sought to reexamine reconsolidation in an appetitive, Pavlovian conditioned approach procedure and the behavioral boundary conditions within which memories are destabilized and reconsolidated. This study demonstrated that a Pavlovian goal-tracking memory, previously thought to be resistant to destabilization, will undergo memory reconsolidation under discrete conditions that favor reconsolidation as opposed to extinction, and that this is dependent on the amount of training rats received. With restricted training, systemic administration of MK-801 impaired memory extinction. In contrast, with more extended training, MK-801 administration impaired memory reconsolidation. We also demonstrate that behavioral boundary conditions that exist for appetitive memory reconsolidation are much more complex than simple parametric calculations. Moreover, extinction per se is not a boundary on reconsolidation, in that MK-801 also has no behavioral effect under some conditions.

A model of amygdala-hippocampal-prefrontal interaction in fear conditioning and extinction in animals

Empirical research has shown that the amygdala, hippocampus, and ventromedial prefrontal cortex (vmPFC) are involved in fear conditioning. However, the functional contribution of each brain area and the nature of their interactions are not clearly understood. Here, we extend existing neural network models of the functional roles of the hippocampus in classical conditioning to include interactions with the amygdala and prefrontal cortex. We apply the model to fear conditioning, in which animals learn physiological (e.g. heart rate) and behavioral (e.g. freezing) responses to stimuli that have been paired with a highly aversive event (e.g. electrical shock). The key feature of our model is that learning of these conditioned responses in the central nucleus of the amygdala is modulated by two separate processes, one from basolateral amygdala and signaling a positive prediction error, and one from the vmPFC, via the intercalated cells of the amygdala, and signaling a negative prediction error. In addition, we propose that hippocampal input to both vmPFC and basolateral amygdala is essential for contextual modulation of fear acquisition and extinction. The model is sufficient to account for a body of data from various animal fear conditioning paradigms, including acquisition, extinction, reacquisition, and context specificity effects. Consistent with studies on lesioned animals, our model shows that damage to the vmPFC impairs extinction, while damage to the hippocampus impairs extinction in a different context (e.g., a different conditioning chamber from that used in initial training in animal experiments). We also discuss model limitations and predictions, including the effects of number of training trials on fear conditioning.

Neural functions of calcineurin in synaptic plasticity and memory

Major brain functions depend on neuronal processes that favor the plasticity of neuronal circuits while at the same time maintaining their stability. The mechanisms that regulate brain plasticity are complex and engage multiple cascades of molecular components that modulate synaptic efficacy. Protein kinases (PKs) and phosphatases (PPs) are among the most important of these components that act as positive and negative regulators of neuronal signaling and plasticity, respectively. In these cascades, the PP protein phosphatase 2B or calcineurin (CaN) is of particular interest because it is the only Ca(2+)-activated PP in the brain and a major regulator of key proteins essential for synaptic transmission and neuronal excitability. This review describes the primary properties of CaN and illustrates its functions and modes of action by focusing on several representative targets, in particular glutamate receptors, striatal enriched protein phosphatase (STEP), and neuromodulin (GAP43), and their functional significance for synaptic plasticity and memory.

Neural and cellular mechanisms of fear and extinction memory formation

Over the course of natural history, countless animal species have evolved adaptive behavioral systems to cope with dangerous situations and promote survival. Emotional memories are central to these defense systems because they are rapidly acquired and prepare organisms for future threat. Unfortunately, the persistence and intrusion of memories of fearful experiences are quite common and can lead to pathogenic conditions, such as anxiety and phobias. Over the course of the last 30 years, neuroscientists and psychologists alike have attempted to understand the mechanisms by which the brain encodes and maintains these aversive memories. Of equal interest, though, is the neurobiology of extinction memory formation as this may shape current therapeutic techniques. Here we review the extant literature on the neurobiology of fear and extinction memory formation, with a strong focus on the cellular and molecular mechanisms underlying these processes.

Increasing histone acetylation in the hippocampus-infralimbic network enhances fear extinction

BACKGROUND

A key finding from recent studies of epigenetic mechanisms of memory is that increasing histone acetylation after a learning experience enhances memory consolidation. This has been demonstrated in several preparations, but little is known about whether excitatory and inhibitory memories are equally sensitive to drugs that promote histone acetylation and how transcriptional changes in the hippocampal-medial prefrontal cortex network contribute to these drug effects.

METHODS

We compare the long-term behavioral consequences of systemic, intrahippocampal and intra-medial prefrontal cortex administration of the histone deacetylase inhibitor sodium butyrate (NaB) after contextual fear conditioning and extinction 1 and/or 14 days later in male c57BL/6J mice (n = 302). Levels of histone acetylation and expression of the product of the immediate-early gene c-Fos were assessed by immunohistochemistry following infusion of NaB into the hippocampus (n = 26).

RESULTS

Across a variety of conditions, the effects of NaB on extinction were larger and more persistent compared to the effects on initial memory formation. NaB administered following weak extinction induced behavioral extinction, infralimbic histone acetylation and c-Fos expression consistent with strong extinction. No similar effect was seen in the prelimbic cortex. The involvement of the infralimbic cortex was confirmed as infusions of NaB into the infralimbic, but not prelimbic cortex, induced extinction enhancements.

CONCLUSIONS

These studies show that the memory modulating ability of drugs that enhance acetylation is sensitive to a variety of behavioral and molecular conditions. We further identify transcriptional changes in the hippocampal-infralimbic circuit associated with extinction enhancements induced by the histone deacetylase inhibitor NaB.

Rats distinguish between absence of events and lack of evidence in contingency learning

The goal of three experiments was to study whether rats are aware of the difference between absence of events and lack of evidence. We used a Pavlovian extinction paradigm in which lights consistently signaling sucrose were suddenly paired with the absence of sucrose. The crucial manipulation involved the absent outcomes in the extinction phase. Whereas in the Cover conditions, access to the drinking receptacle was blocked by a metal plate, in the No Cover conditions, the drinking receptacle was accessible. The Test phase showed that in the Cover conditions, the measured expectancies of sucrose were clearly at a higher level than in the No Cover conditions. We compare two competing theories potentially explaining the findings. A cognitive theory interprets the observed effect as evidence that the rats were able to understand that the cover blocked informational access to the outcome information, and therefore the changed learning input did not necessarily signify a change of the underlying contingency in the world. An alternative associationist account, renewal theory, might instead explain the relative sparing of extinction in the Cover condition as a consequence of context change. We discuss the merits of both theories as accounts of our data and conclude that the cognitive explanation is in this case preferred.

HDAC1 regulates fear extinction in mice

Histone acetylation has been implicated with the pathogenesis of neuropsychiatric disorders and targeting histone deacetylases (HDACs) using HDAC inhibitors was shown to be neuroprotective and to initiate neuroregenerative processes. However, little is known about the role of individual HDAC proteins during the pathogenesis of brain diseases. HDAC1 was found to be upregulated in patients suffering from neuropsychiatric diseases. Here, we show that virus-mediated overexpression of neuronal HDAC1 in the adult mouse hippocampus specifically affects the extinction of contextual fear memories, while other cognitive abilities were unaffected. In subsequent experiments we show that under physiological conditions, hippocampal HDAC1 is required for extinction learning via a mechanism that involves H3K9 deacetylation and subsequent trimethylation of target genes. In conclusion, our data show that hippocampal HDAC1 has a specific role in memory function.

Facets of Pavlovian and operant extinction

Research on extinction is of fundamental importance in both Pavlovian and operant approaches to the experimental analysis of learning. Although these approaches are often motivated by different empirical and theoretical questions, extinction has emerged as a research area in which common themes unite the two approaches. In this review, we focus on some common considerations in the analysis of Pavlovian and operant extinction. These include methodological challenges and interpretational issues in analyzing behavior during and after extinction. We consider the different roles that theory has played in the development of research on extinction in these preparations and conclude with some attention to applications of extinction.

Dietary Cholesterol Concentration and Duration Degrade Long-Term Memory of Classical Conditioning of the Rabbit's Nictitating Membrane Response

A rabbit model of Alzheimer's disease based on feeding a cholesterol diet for eight weeks shows sixteen hallmarks of the disease, including learning and memory changes. Although we have shown 2% cholesterol and copper in water can retard learning, other studies show feeding dietary cholesterol before learning can improve acquisition whereas feeding cholesterol after learning can degrade long-term memory. We explored this issue by manipulating cholesterol concentration and duration following classical trace conditioning of the rabbit's nictitating membrane response and assessed conditioned responding after eight weeks on cholesterol. First, rabbits given trace classical conditioning followed by 0.5%, 1%, or 2% cholesterol for eight weeks showed body weight and serum cholesterol levels that were a function of dietary cholesterol. Although all concentrations of cholesterol showed some sign of retarding long-term memory, the level of memory retardation was correlated with serum cholesterol levels. Second, rabbits given trace conditioning followed by different durations of a 2% cholesterol diet combined with different durations of a 0% control diet for 8 weeks showed duration and timing of a 2% cholesterol diet were important in affecting recall. The data support the idea that dietary cholesterol may retard long-term memory.

A parametric analysis of factors affecting acquisition and extinction of contextual fear in C57BL/6 and DBA/2 mice

Behavioral analyses of genetically modified and inbred strains of mice have revealed neural systems and molecules that are involved in memory formation. Many of these studies have examined memories that form in contextual fear conditioning, in which an organism learns that a particular context signals the occurrence of a footshock. During fear extinction, nonreinforced exposure to the context results in the loss of the conditioned fear response. The study of extinction has been instrumental for behavioral and molecular theories of memory. However, many of the transgenic, knockout, and inbred strains of mice that have been widely studied in memory have behavioral deficits in contextual fear conditioning, which makes the study of extinction in these mice particularly challenging. Here we explore several strategies for studying extinction in C57BL/6 and DBA/2 mice, two strains known to differ in contextual fear conditioning. First, we attempt to equate performance prior to extinction through several extensive conditioning protocols. Second, we examine extinction in subsets of mice matched for initial levels of context conditioning. Third, we examine within-strain effects of variables known to affect extinction. Differences between the strains persisted across extensive conditioning and extinction protocols, but both strains were sensitive to session duration and context manipulations during extinction. We describe the implications of our results for behavioral and neurobiological approaches to extinction, and we examine the general challenges in studying extinction in subjects that differ in learning or performance prior to extinction.

Issues in the extinction of specific stimulus-outcome associations in Pavlovian conditioning

This paper reviews a variety of studies designed to examine the effects of extinction upon control by specific stimulus-outcome (S-O) associations in Pavlovian conditioning. Studies conducted with rats in a magazine approach conditioning paradigm have shown that control by specific S-O associations is normally unaffected by extinction treatments, although other aspects of conditioned responding seem affected in a more enduring way. However, recent work suggests that extinction can undermine control by such associations if it is administered after the conditioned stimulus is weakly encoded. The results from these studies suggest that it may be important to consider multiple response systems in assessing the impact of extinction. Studies conducted with the flavor preference learning paradigm in rats also show that specific S-O associations can be undermined by procedures that involve presenting a flavor cue in the absence of its associated nutrient. These findings provide no support for the view that flavor preference learning necessarily entails some unique learning process that differs from more conventional processes. As in other situations, some of these effects likely involve a masking process, but the extent to which masking or true associative weakening occurs in extinction more generally is a topic that is not well understood. Finally, we present some data to suggest that extinction also involves conditional "occasion-setting" control by contextual cues. Special procedures are recommended in assessing such learning when the goal is to distinguish this form of learning from other more conventional mechanisms of extinction.

NMDA receptors and fear extinction: implications for cognitive behavioral therapy

Based primarily on studies that employ Pavlovian fear conditioning, extinction of conditioned fear has been found to be mediated by N-methyl-D-aspartate (NMDA) receptors in the amygdala and medial prefrontal cortex. This led to the discovery that an NMDA partial agonist, D-cycloserine, could facilitate fear extinction when given systemically or locally into the amygdala. Because many forms of cognitive behavioral therapy depend on fear extinction, this led to the successful use of D-cycloserine as an adjunct to psychotherapy in patients with so-called simple phobias (fear of heights), social phobia, obsessive-compulsive behavior, and panic disorder. Data in support of these conclusions are reviewed, along with some of the possible limitations of D-cycloserine as an adjunct to psychotherapy.