Blocking, Unblocking, and Overexpectation of Fear: A Role for Opioid Receptors in the Regulation of Pavlovian Association Formation

Loneliness and the persistence of fear: Perceived social isolation reduces evaluative fear extinction

Loneliness has been linked to a host of harmful physical and mental health outcomes, detrimental effects that may stem from increases in threat-responding caused by altered fear learning in lonely individuals. In particular, the heightened threat-vigilance that is a hallmark of loneliness may augment the processes by which fear learning occurs, ultimately resulting in a greater number of perceived threatening cues in the environment. However, almost no research has examined how loneliness alters fear learning processes in humans. Here, we investigated the effect of loneliness on fear learning during an evaluative learning procedure in which participants (n = 782) were taught to associate fearful, positive, or neutral control stimuli with neutral images. Results showed that reduced extinction of evaluative fear associations occurred in high (vs. low) lonely individuals, but there was no difference in extinction of evaluative appetitive (also known as positive or reward) associations, suggesting this effect is specific to fear learning. In addition to shedding light on the link between loneliness and poor health, these results represent an important step forward in the growing understanding of the powerful impact of social bonds on fear learning processes.

The Rescorla-Wagner model, prediction error, and fear learning

The Rescorla-Wagner model remains one of the most important and influential theoretical accounts of the conditions under which Pavlovian learning occurs. Moreover, the experimental approaches that inspired the model continue to provide powerful behavioral tools to advance mechanistic understanding of how we and other animals learn to fear and learn to reduce fear. Here we consider key features of the Rescorla-Wagner model as applied to study of fear learning. We review evidence for key insights of the model. First, learning to fear and learning to reduce fear are governed by a common, signed prediction error. Second, this error drives variations in effectiveness of the shock US that are causal to whether and how much fear is learned or lost during a conditioning trial. We also consider behavioral and neural findings inconsistent with the model and which will be essential to understand and advance understanding of fear learning.

Anesthesia and the neurobiology of fear and posttraumatic stress disorder

PURPOSE OF REVIEW

Dysfunction of fear memory systems underlie a cluster of clinically important and highly prevalent psychological morbidities seen in perioperative and critical care patients, most archetypally posttraumatic stress disorder (PTSD). Several sedative-hypnotics and analgesics are known to modulate fear systems, and it is theoretically plausible that clinical decisions of the anesthesiologist could impact psychological outcomes. This review aims to provide a focused synthesis of relevant literature from multiple fields of research.

RECENT FINDINGS

There is evidence in some contexts that unconscious fear memory systems are less sensitive to anesthetics than are conscious memory systems. Opiates may suppress the activation of fear systems and have benefit in the prevention of PTSD following trauma. There is inconsistent evidence that the use of propofol and benzodiazepines for sedation following trauma may potentiate the development of PTSD relative to other drugs. The benefits of ketamine seen in the treatment of major depression are not clearly replicated in PTSD-cluster psychopathologies, and its effects on fear processes are complex.

SUMMARY

There are multiple theoretical mechanisms by which anesthetic drugs can modulate fear systems and clinically important fear-based psychopathologies. The current state of research provides some evidence to support further hypothesis investigation. However, the absence of effectiveness studies and the inconsistent signals from smaller studies provide insufficient evidence to currently offer firm clinical guidance.

Reclassifying the Unique Inhibitory Properties of Social Support Figures: A Roadmap for Exploring Prepared Fear Suppression

Recent work has revealed that social support cues are powerful inhibitors of the fear response. They are endowed with a unique combination of inhibitory properties, enabling them to both inhibit fear in the short term and reduce fear in the long term. While these findings had previously been thought to suggest that social support cues belong to a category of prepared safety stimuli, mounting evidence clearly shows that the mechanisms underlying safety signaling cannot account for the unique effects of social support cues. Here, we propose a reclassification of social support cues as members of a prepared fear suppressor category. We present an argument for the prepared fear suppressor classification, discuss potential mechanisms underlying the unique effects of prepared fear suppressors, and outline next steps to build an understanding of this category and its clinical implications. This review is meant to serve as a roadmap for exploring this novel category of prepared fear suppressors, whose never-before-seen range of inhibitory effects makes them an important and impactful discovery with implications for both fear learning theory and clinical application.

The Opioid Receptor Antagonist Naloxone Enhances First-Order Fear Conditioning, Second-Order Fear Conditioning and Sensory Preconditioning in Rats

The opioid receptor antagonist naloxone enhances Pavlovian fear conditioning when rats are exposed to pairings of an initially neutral stimulus, such as a tone, and a painful foot shock unconditioned stimulus (US; so-called first-order fear conditioning; Pavlov, 1927). The present series of experiments examined whether naloxone has the same effect when conditioning occurs in the absence of US exposure. In Experiments 1a and 1b, rats were exposed to tone-shock pairings in stage 1 (one trial per day for 4 days) and then to pairings of an initially neutral light with the already conditioned tone in stage 2 (one trial per day for 4 days). Experiment 1a confirmed that this training results in second-order fear of the light; and Experiment 1b showed that naloxone enhances this conditioning: rats injected with naloxone in stage 2 froze more than vehicle-injected controls when tested with the light alone (drug-free). In Experiments 2a and 2b, rats were exposed to light-tone pairings in stage 1 (one trial per day for 4 days) and then to tone-shock pairings in stage 2 (one trial per day for 2 days). Experiment 2a confirmed that this training results in sensory preconditioned fear of the light; and Experiment 2b showed that naloxone enhances sensory preconditioning when injected prior to each of the light-tone pairings: rats injected with naloxone in stage 1 froze more than vehicle-injected controls when tested with the light alone (drug-free). These results were taken to mean that naloxone enhances fear conditioning independently of its effect on US processing; and more generally, that opioids regulate the error-correction mechanisms that underlie associative formation.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Pre-training naltrexone increases conditioned fear learning independent of adolescent alcohol consumption history

Our previous research has shown a relationship between low voluntary alcohol consumption and high conditioned fear in male Long Evans rats. Here, we examined whether differences in the endogenous opioid systems might be responsible for these differences. Rats received 6 weeks of chronic intermittent to 20% alcohol (v/v) or water-only from PND 26-66. Based on their consumption during the last 2 weeks of alcohol access, the alcohol-access rats were divided into high drinking (>2.5 g/kg/24-h) or low drinking (<2 g/kg/24-h). Rats were then given injections of the preferential mu opioid receptor antagonist naltrexone (1 mg/kg, s.c.) or the selective kappa opioid receptor antagonist LY2456302 (10 mg/kg, s.c.) prior to fear conditioning and were then tested for conditioned fear 2 days later. Pre-training naltrexone increased conditioned suppression of lever-pressing during training and testing, with no differences between high versus low alcohol drinkers or between water-only versus alcohol access groups (averaged across drinking levels). There was no effect of LY2456302 on conditioned fear in any comparison. We also found no differences between high and low alcohol drinkers and no reliable effect of prior alcohol access (averaged across drinking levels) on conditioned fear. Our experiment replicates and extends previous demonstrations that a preferential MOR antagonist can increase fear learning using conditioned suppression of lever-pressing as a fear measure. However, additional research is needed to determine the cause of the differences in conditioned fear that we previously observed (as they were not observed in the current experiments).

Individual differences in voluntary alcohol consumption are associated with conditioned fear in the fear incubation model

Previous research in male Long Evans rats has shown a relationship between low voluntary alcohol consumption and high conditioned fear after a single training session. Here, we determined whether chronic intermittent access (CIA) to alcohol during adolescence/early adulthood or during adulthood would alter or be associated with auditory-cued conditioned fear levels using an extended training fear incubation procedure. This training procedure leads to low fear soon after training that grows over one month. Rats received 6 weeks of CIA to 20% alcohol or water from PND 26-66. Ten or eleven days later, the rats began behavioral testing that included 10 sessions of tone-shock pairings. Rats then received 4 weeks of CIA exposure during the 1-month fear incubation period and were tested for conditioned fear 6 days after the end of alcohol access. We found no evidence that voluntary alcohol consumption during adolescence/early adulthood or adulthood altered fear expression. However, we found that rats that consumed more alcohol during early adulthood (PND 54-66) had lower fear than low-consumption rats on day 1 of conditioned fear training and in the day 2 and 1-month tests. This extends associations we previously found between individual differences in alcohol consumption and conditioned fear to a different fear conditioning procedure. Combined with our previous data that show that the rate of instrumental extinction is associated with both alcohol consumption and conditioned fear, these data provide further support for the generality and reliability of a pair of phenotypes that encompass a wide variety of learning traits.

Prefrontal-limbic Functional Connectivity during Acquisition and Extinction of Conditioned Fear

This study is a new analysis to obtain novel metabolic data on the functional connectivity of prefrontal-limbic regions in Pavlovian fear acquisition and extinction of tone-footshock conditioning. Mice were analyzed with the fluorodeoxyglucose (FDG) autoradiographic method to metabolically map regional brain activity. New FDG data were sampled from the nuclei of the habenula and other regions implicated in aversive conditioning, such as infralimbic cortex, amygdala and periaqueductal gray regions. The activity patterns among these regions were inter-correlated during acquisition, extinction or pseudorandom training to develop a functional connectivity model. Two subdivisions of the habenular complex showed increased activity after acquisition relative to extinction, with the pseudorandom group intermediate between the other two groups. Significant acquisition activation effects were also found in centromedial amygdala, dorsomedial and ventrolateral periaqueductal gray. FDG uptake increases during extinction were found only in dorsal and ventral infralimbic cortex. The overall pattern of activity correlations between these regions revealed extensive but differential functional connectivity during acquisition and extinction training, with less functional connectivity found after pseudorandom training. Interestingly, habenula nuclei showed a distinct pattern of inter-correlations with amygdala nuclei during extinction. The functional connectivity model revealed changing interactions among infralimbic cortex, amygdala, habenula and periaqueductal gray regions through the stages of Pavlovian fear acquisition and extinction. This study provided new data on the contributions of the habenula to fear conditioning, and revealed previously unreported infralimbic-amygdala-habenula-periaqueductal gray interactions implicated in acquisition and extinction of conditioned fear.

Effects of chemogenetic excitation or inhibition of the ventrolateral periaqueductal gray on the acquisition and extinction of Pavlovian fear conditioning

The midbrain periaqueductal gray (PAG) has been implicated in the generation and transmission of a prediction error signal that instructs amygdala-based fear and extinction learning. However, the PAG also plays a key role in the expression of conditioned fear responses. The evidence for a role of the PAG in fear learning and extinction learning has been obtained almost exclusively using PAG-dependent fear responses. It is less clear whether the PAG regulates fear learning when other measures of learned fear are used. Here we combined a chemogenetic approach, permitting excitation or inhibition of neurons in the ventrolateral PAG (VLPAG), with conditioned suppression as the measure of learned fear to assess the role of VLPAG in the acquisition and extinction of fear learning. We show that chemogenetic excitation of VLPAG (with some encroachment on lateral PAG [LPAG]) impairs acquisition of fear and, conversely, chemogenetic inhibition impairs extinction of fear. These effects on fear and extinction learning were specific to the combination of DREADD expression and injection of CNO because they were observed relative to both eYFP controls injected with CNO as well as DREADD expressing controls injected with vehicle. Taken together, these results show that activity of L/VLPAG neurons regulates both the acquisition and extinction of Pavlovian fear learning.

Predictive mechanisms linking brain opioids to chronic pain vulnerability and resilience

Chronic pain is a major global healthcare problem that is currently inadequately treated. In addition, the current use of opioids for treatment has reached far beyond the paucity of evidence for long-term benefits relative to risks. Benefit-risk models for opioid and non-opioid treatments would benefit from a rational, mechanism-based understanding of neuroplastic and neurochemical contributions to chronic pain. Here, we evaluate the findings and limitations of representative research investigating brain neuroplasticity and neurochemistry in chronic pain. In sum, the mechanisms of pain-related neuroplasticity in the brain remain poorly understood because neuroimaging studies have been largely descriptive. We argue that definition is needed of optimal (pain-resilient) and suboptimal (pain-vulnerable) functioning of the endogenous opioid system in order to identify neurochemical contributions to aberrant neuroplasticity in chronic pain. We outline the potential benefits of computational approaches that utilize evolutionary and statistical optimality principles, illustrating this approach with mechanistic hypotheses on opioid function. In particular, we discuss the role of predictive mechanisms in perceptual and associative plasticity and evidence for their modulation by endogenous opioids. Future research should attempt to utilize formal computational models to provide evidence for the clinical validity of this approach, thereby providing a rational basis for future treatment and, ideally, prevention.

LINKED ARTICLES

This article is part of a themed section on Emerging Areas of Opioid Pharmacology. To view the other articles in this section visit http://onlinelibrary.wiley.com/doi/10.1111/bph.v175.14/issuetoc.

Opioid suppression of conditioned anticipatory brain responses to breathlessness

Opioid painkillers are a promising treatment for chronic breathlessness, but are associated with potentially fatal side effects. In the treatment of breathlessness, their mechanisms of action are unclear. A better understanding might help to identify safer alternatives. Learned associations between previously neutral stimuli (e.g. stairs) and repeated breathlessness induce an anticipatory threat response that may worsen breathlessness, contributing to the downward spiral of decline seen in clinical populations. As opioids are known to influence associative learning, we hypothesized that they may interfere with the brain processes underlying a conditioned anticipatory response to breathlessness in relevant brain areas, including the amygdala and the hippocampus.

Healthy volunteers viewed visual cues (neutral stimuli) immediately before induction of experimental breathlessness with inspiratory resistive loading. Thus, an association was formed between the cue and breathlessness. Subsequently, this paradigm was repeated in two identical neuroimaging sessions with intravenous infusions of either low-dose remifentanil (0.7 ng/ml target-controlled infusion) or saline (randomised).

During saline infusion, breathlessness anticipation activated the right anterior insula and the adjacent operculum. Breathlessness was associated with activity in a network including the insula, operculum, dorsolateral prefrontal cortex, anterior cingulate cortex and the primary sensory and motor cortices.

Remifentanil reduced breathlessness unpleasantness but not breathlessness intensity. Remifentanil depressed anticipatory activity in the amygdala and the hippocampus that correlated with reductions in breathlessness unpleasantness. During breathlessness, remifentanil decreased activity in the anterior insula, anterior cingulate cortex and sensory motor cortices. Remifentanil-induced reduction in breathlessness unpleasantness was associated with increased activity in the rostral anterior cingulate cortex and nucleus accumbens, components of the endogenous opioid system known to decrease the perception of aversive stimuli.

These findings suggest that in addition to effects on brainstem respiratory control, opioids palliate breathlessness through an interplay of altered associative learning mechanisms. These mechanisms provide potential targets for novel ways to develop and assess treatments for chronic breathlessness.

•

The mechanisms of how low-dose opioids relieve breathlessness are unknown.

•

We tested whether low-dose opioids affect conditioned anticipation and perception of breathlessness.

•

Low-dose opioids reduced unpleasantness, but not intensity of breathlessness.

•

Reduced breathlessness unpleasantness was associated with activation of the endogenous opioid system.

•

Breathlessness relief was predicted by decreased anticipatory activity in amygdala/hippocampus.

Disrupted Prediction Error Links Excessive Amygdala Activation to Excessive Fear

Basolateral amygdala (BLA) is critical for fear learning, and its heightened activation is widely thought to underpin a variety of anxiety disorders. Here we used chemogenetic techniques in rats to study the consequences of heightened BLA activation for fear learning and memory, and to specifically identify a mechanism linking increased activity of BLA glutamatergic neurons to aberrant fear. We expressed the excitatory hM3Dq DREADD in rat BLA glutamatergic neurons and showed that CNO acted selectively to increase their activity, depolarizing these neurons and increasing their firing rates. This chemogenetic excitation of BLA glutamatergic neurons had no effect on the acquisition of simple fear learning, regardless of whether this learning led to a weak or strong fear memory. However, in an associative blocking task, chemogenetic excitation of BLA glutamatergic neurons yielded significant learning to a blocked conditioned stimulus, which otherwise should not have been learned about. Moreover, in an overexpectation task, chemogenetic manipulation of BLA glutamatergic neurons prevented use of negative prediction error to reduce fear learning, leading to significant impairments in fear inhibition. These effects were not attributable to the chemogenetic manipulation enhancing arousal, increasing asymptotic levels of fear learning or fear memory consolidation. Instead, chemogenetic excitation of BLA glutamatergic neurons disrupted use of prediction error to regulate fear learning.

SIGNIFICANCE STATEMENT

Several neuropsychiatric disorders are characterized by heightened activation of the amygdala. This heightened activation has been hypothesized to underlie increased emotional reactivity, fear over generalization, and deficits in fear inhibition. Yet the mechanisms linking heightened amygdala activation to heightened emotional learning are elusive. Here we combined chemogenetic excitation of rat basolateral amygdala glutamatergic neurons with a variety of behavioral approaches to show that, although simple fear learning is unaffected, the use of prediction error to regulate this learning is profoundly disrupted, leading to formation of inappropriate fear associations and impaired fear inhibition.

The Origins and Organization of Vertebrate Pavlovian Conditioning

Pavlovian conditioning is the process by which we learn relationships between stimuli and thus constitutes a basic building block for how the brain constructs representations of the world. We first review the major concepts of Pavlovian conditioning and point out many of the pervasive misunderstandings about just what conditioning is. This brings us to a modern redefinition of conditioning as the process whereby experience with a conditional relationship between stimuli bestows these stimuli with the ability to promote adaptive behavior patterns that did not occur before the experience. Working from this framework, we provide an in-depth analysis of two examples, fear conditioning and food-based appetitive conditioning, which include a description of the only partially overlapping neural circuitry of each. We also describe how these circuits promote the basic characteristics that define Pavlovian conditioning, such as error-correction-driven regulation of learning.

Purkinje cell activity during classical conditioning with different conditional stimuli explains central tenet of Rescorla–Wagner model [corrected]

A central tenet of Rescorla and Wagner's model of associative learning is that the reinforcement value of a paired trial diminishes as the associative strength between the presented stimuli increases. Despite its fundamental importance to behavioral sciences, the neural mechanisms underlying the model have not been fully explored. Here, we present findings that, taken together, can explain why a stronger association leads to a reduced reinforcement value, within the context of eyeblink conditioning. Specifically, we show that learned pause responses in Purkinje cells, which trigger adaptively timed conditioned eyeblinks, suppress the unconditional stimulus (US) signal in a graded manner. Furthermore, by examining how Purkinje cells respond to two distinct conditional stimuli and to a compound stimulus, we provide evidence that could potentially help explain the somewhat counterintuitive overexpectation phenomenon, which was derived from the Rescorla-Wagner model.

Morphine administration during low ovarian hormone stage results in transient over expression of fear memories in females

Acute exposure to morphine after a traumatic event reduces trauma related symptoms in humans and conditioned fear expression in male rats. We aimed to determine whether acute administration of morphine alters consolidation of fear learning and extinction. Male and female rats in proestrus and metaestrus (high and low ovarian hormones respectively) underwent fear conditioning and received saline or morphine (2.5 mg/kg s.c.). The next day they underwent extinction. Results showed increased freezing during extinction only in the morphine metaestrus group while morphine did not affect males or proestrus females. Recall of extinction was similar on all groups. On a second experiment, a subset of rats conditioned during metaestrus was administered morphine prior to extinction producing no effects. We then measured mu opioid receptor (MOR) expression in the amygdala and periaqueductal gray (PAG) at the end of extinction (day 2). In males and proestrus females, morphine caused an increase in MOR in the amygdala but no in the PAG. In metaestrus females, morphine did not change MOR expression in either structure. These data suggests that ovarian hormones may interact with MORs in the amygdala to transiently alter memory consolidation. Morphine given after trauma to females with low ovarian hormones might increase the recall of fear responses, making recovery harder.

SSCC TD: a serial and simultaneous configural-cue compound stimuli representation for temporal difference learning

This paper presents a novel representational framework for the Temporal Difference (TD) model of learning, which allows the computation of configural stimuli--cumulative compounds of stimuli that generate perceptual emergents known as configural cues. This Simultaneous and Serial Configural-cue Compound Stimuli Temporal Difference model (SSCC TD) can model both simultaneous and serial stimulus compounds, as well as compounds including the experimental context. This modification significantly broadens the range of phenomena which the TD paradigm can explain, and allows it to predict phenomena which traditional TD solutions cannot, particularly effects that depend on compound stimuli functioning as a whole, such as pattern learning and serial structural discriminations, and context-related effects.

Developmental rodent models of fear and anxiety: from neurobiology to pharmacology

Anxiety disorders pose one of the biggest threats to mental health in the world, and they predominantly emerge early in life. However, research of anxiety disorders and fear-related memories during development has been largely neglected, and existing treatments have been developed based on adult models of anxiety. The present review describes animal models of anxiety disorders across development and what is currently known of their pharmacology. To summarize, the underlying mechanisms of intrinsic 'unlearned' fear are poorly understood, especially beyond the period of infancy. Models using 'learned' fear reveal that through development, rats exhibit a stress hyporesponsive period before postnatal day 10, where they paradoxically form odour-shock preferences, and then switch to more adult-like conditioned fear responses. Juvenile rats appear to forget these aversive associations more easily, as is observed with the phenomenon of infantile amnesia. Juvenile rats also undergo more robust extinction, until adolescence where they display increased resistance to extinction. Maturation of brain structures, such as the amygdala, prefrontal cortex and hippocampus, along with the different temporal recruitment and involvement of various neurotransmitter systems (including NMDA, GABA, corticosterone and opioids) are responsible for these developmental changes. Taken together, the studies described in this review highlight that there is a period early in development where rats appear to be more robust in overcoming adverse early life experience. We need to understand the fundamental pharmacological processes underlying anxiety early in life in order to take advantage of this period for the treatment of anxiety disorders.

Isomorphisms between psychological processes and neural mechanisms: from stimulus elements to genetic markers of activity

Traditional learning theory has developed models that can accurately predict and describe the course of learned behavior. These "psychological process" models rely on hypothetical constructs that are usually thought to be not directly measurable or manipulable. Recently, and mostly in parallel, the neural mechanisms underlying learning have been fairly well elucidated. The argument in this essay is that we can successfully uncover isomorphisms between process and mechanism and that this effort will help advance our theories about both processes and mechanisms. We start with a brief review of error-correction circuits as a successful example. Then we turn to the concept of stimulus elements, where the conditional stimulus is hypothesized to be constructed of a multitude of elements only some of which are sampled during any given experience. We discuss such elements with respect to how they explain acquisition of associative strength as an incremental process. Then we propose that for fear conditioning, stimulus elements and basolateral amygdala projection neurons are isomorphic and that the activational state of these "elements" can be monitored by the expression of the mRNA for activity-regulated cytoskeletal protein (ARC). Finally we apply these ideas to analyze recent data examining ARC expression during contextual fear conditioning and find that there are indeed many similarities between stimulus elements and amygdala neurons. The data also suggest some revisions in the conceptualization of how the population of stimulus elements is sampled from.

Error correction in latent inhibition and its disruption by opioid receptor blockade with naloxone

Latent inhibition refers to the retardation in the development of conditioned responding when a pre-exposed stimulus is used to signal an unconditioned stimulus. This effect is described by error-correction models as an attentional deficit and is commonly used as an animal model of schizophrenia. A series of experiments studied the role of error-correction mechanism in latent inhibition and its interaction with the endogenous opioid system. Systemic administration of the competitive opioid receptor antagonist naloxone before rats were pre-exposed to a target stimulus prevented latent inhibition of its subsequent fear conditioning; it was without effect on a non-pre-exposed stimulus and did not produce state-dependent learning (Experiments 1a and 1b). Naloxone did not reverse the latent inhibitory effect already accrued to a pre-exposed target. However, it did prevent the enhancement of latent inhibition by a long retention interval interpolated between its initial exposure and re-exposure (Experiment 2) or by a novel stimulus compounded with the pre-exposed target during re-exposure (Experiment 3). These results provide evidence that attentional loss in latent inhibition is instructed by an opioid-mediated error signal which diminishes with repeated stimulus exposures but recovers with the passage of time or reintroduction of novelty.

Behavioral and neurobiological mechanisms of extinction in Pavlovian and instrumental learning

This article reviews research on the behavioral and neural mechanisms of extinction as it is represented in both Pavlovian and instrumental learning. In Pavlovian extinction, repeated presentation of a signal without its reinforcer weakens behavior evoked by the signal; in instrumental extinction, repeated occurrence of a voluntary action without its reinforcer weakens the strength of the action. In either case, contemporary research at both the behavioral and neural levels of analysis has been guided by a set of extinction principles that were first generated by research conducted at the behavioral level. The review discusses these principles and illustrates how they have informed the study of both Pavlovian and instrumental extinction. It shows that behavioral and neurobiological research efforts have been tightly linked and that their results are readily integrated. Pavlovian and instrumental extinction are also controlled by compatible behavioral and neural processes. Since many behavioral effects observed in extinction can be multiply determined, we suggest that the current close connection between behavioral-level and neural-level analyses will need to continue.

The error in total error reduction

Most models of human and animal learning assume that learning is proportional to the discrepancy between a delivered outcome and the outcome predicted by all cues present during that trial (i.e., total error across a stimulus compound). This total error reduction (TER) view has been implemented in connectionist and artificial neural network models to describe the conditions under which weights between units change. Electrophysiological work has revealed that the activity of dopamine neurons is correlated with the total error signal in models of reward learning. Similar neural mechanisms presumably support fear conditioning, human contingency learning, and other types of learning. Using a computational modeling approach, we compared several TER models of associative learning to an alternative model that rejects the TER assumption in favor of local error reduction (LER), which assumes that learning about each cue is proportional to the discrepancy between the delivered outcome and the outcome predicted by that specific cue on that trial. The LER model provided a better fit to the reviewed data than the TER models. Given the superiority of the LER model with the present data sets, acceptance of TER should be tempered.

An extension of the Rescorla and Wagner Simulator for context conditioning

This paper introduces R&W Simulator version 4, which extends previous work by incorporating context simulation within standard Pavlovian designs. This addition allows the assessment of: (1) context-stimulus competition, by treating contextual cues as ordinary background stimuli present throughout the whole experimental session; (2) summation, by computing compound stimuli with contextual cues as an integrating feature, with and without the addition of specific configural cues; and (3) contingency effects in causal learning. These new functionalities broaden the range of experimental designs that the simulator is able to replicate, such as some recovery from extinction phenomena (e.g., renewal effects). In addition, the new version permits specifying probe trials among standard trials and extracting their values.

Neuropeptide regulation of fear and anxiety: Implications of cholecystokinin, endogenous opioids, and neuropeptide Y

The neural circuitry of fear likely underlies anxiety and fear-related disorders such as specific and social phobia, panic disorder, and posttraumatic stress disorder. The primary pharmacological treatments currently utilized for these disorders include benzodiazepines, which act on the GABAergic receptor system, and antidepressants, which modulate the monamine systems. However, recent work on the regulation of fear neural circuitry suggests that specific neuropeptide modulation of this system is of critical importance. Recent reviews have examined the roles of the hypothalamic-pituitary-adrenal axis neuropeptides as well as the roles of neurotrophic factors in regulating fear. The present review, instead, will focus on three neuropeptide systems which have received less attention in recent years but which are clearly involved in regulating fear and its extinction. The endogenous opioid system, particularly activating the μ opioid receptors, has been demonstrated to regulate fear expression and extinction, possibly through functioning as an error signal within the ventrolateral periaqueductal gray to mark unreinforced conditioned stimuli. The cholecystokinin (CCK) system initially led to much excitement through its potential role in panic disorder. More recent work in the CCK neuropeptide pathway suggests that it may act in concordance with the endogenous cannabinoid system in the modulation of fear inhibition and extinction. Finally, older as well as very recent data suggests that neuropeptide Y (NPY) may play a very interesting role in counteracting stress effects, enhancing extinction, and enhancing resilience in fear and stress preclinical models. Future work in understanding the mechanisms of neuropeptide functioning, particularly within well-known behavioral circuits, are likely to provide fascinating new clues into the understanding of fear behavior as well as suggesting novel therapeutics for treating disorders of anxiety and fear dysregulation.

Neurobiological mechanisms underlying the blocking effect in aversive learning

Current theories of classical conditioning assume that learning depends on the predictive relationship between events, not just on their temporal contiguity. Here we employ the classic experiment substantiating this reasoning-the blocking paradigm-in combination with functional magnetic resonance imaging (fMRI) to investigate whether human amygdala responses in aversive learning conform to these assumptions. In accordance with blocking, we demonstrate that significantly stronger behavioral and amygdala responses are evoked by conditioned stimuli that are predictive of the unconditioned stimulus than by conditioned stimuli that have received the same pairing with the unconditioned stimulus, yet have no predictive value. When studying the development of this effect, we not only observed that it was related to the strength of previous conditioned responses, but also that predictive compared with nonpredictive conditioned stimuli received more overt attention, as measured by fMRI-concurrent eye tracking, and that this went along with enhanced amygdala responses. We furthermore observed that prefrontal regions play a role in the development of the blocking effect: ventromedial prefrontal cortex (subgenual anterior cingulate) only exhibited responses when conditioned stimuli had to be established as nonpredictive for an outcome, whereas dorsolateral prefrontal cortex also showed responses when conditioned stimuli had to be established as predictive. Most importantly, dorsolateral prefrontal cortex connectivity to amygdala flexibly switched between positive and negative coupling, depending on the requirements posed by predictive relationships. Together, our findings highlight the role of predictive value in explaining amygdala responses and identify mechanisms that shape these responses in human fear conditioning.

Design of a neurally plausible model of fear learning

A neurally oriented conceptual and computational model of fear conditioning manifested by freezing behavior (FRAT), which accounts for many aspects of delay and context conditioning, has been constructed. Conditioning and extinction are the result of neuromodulation-controlled LTP at synapses of thalamic, cortical, and hippocampal afferents on principal cells and inhibitory interneurons of lateral and basal amygdala. The phenomena accounted for by the model (and simulated by the computational version) include conditioning, secondary reinforcement, blocking, the immediate shock deficit, extinction, renewal, and a range of empirically valid effects of pre- and post-training ablation or inactivation of hippocampus or amygdala nuclei.

Placing prediction into the fear circuit

Pavlovian fear conditioning depends on synaptic plasticity at amygdala neurons. Here, we review recent electrophysiological, molecular and behavioral evidence suggesting the existence of a distributed neural circuitry regulating amygdala synaptic plasticity during fear learning. This circuitry, which involves projections from the midbrain periaqueductal gray region, can be linked to prediction error and expectation modulation of fear learning, as described by associative and computational learning models. It controls whether, and how much, fear learning occurs by signaling aversive events when they are unexpected. Functional neuroimaging and clinical studies indicate that this prediction circuit is recruited in humans during fear learning and contributes to exposure-based treatments for clinical anxiety. This aversive prediction error circuit might represent a conserved mechanism for regulating fear learning in mammals.

Kappa opioid receptors mediate where fear is expressed following extinction training

Six experiments used a within-subjects renewal design to examine the involvement of kappa opioid receptors (KORs) in regulating the expression and recovery of extinguished fear. Rats were trained to fear a tone conditioned stimulus (CS) via pairings with foot shock in a distinctive context (A). This was followed by extinction training of the CS in a second context (B). Finally, all rats were tested for fear to the tone in the extinction context (ABB) and the training (ABA) or a novel (ABC) context. Intracerebroventricular (ICV) infusion of the KOR antagonist nor-binaltorphimine dihydrochloride (nor-BNI) dose-dependently prevented ABA renewal of fear, but had no effect on the expression of ABC renewal, the expression of extinction, or the expression of nonextinguished fear. Conversely, pretest infusion of the KOR agonist U50,488 hydrochloride (U50,488) selectively facilitated the expression of ABA renewal and had no effect on the expression of extinction. Pretest infusion of nor-BNI had no effect on the expression of context-specific latent inhibition. Collectively, these results suggest that KORs gate the expression of fear following extinction training and may comprise a critical neuropeptide component of the circuitry underlying context-dependent expression of fear.

Opioid regulation of Pavlovian overshadowing in fear conditioning

In Pavlovian overshadowing, a stimulus that predicts a biologically important event reduces conditioning to another, equally predictive stimulus. We tested the effects of an opioid antagonist and dopamine agonist on the ability of a salient white noise to overshadow a less salient light. Rats were conditioned to fear a light or a noise-light compound using a mild footshock. Compound-conditioned rats trained under the saline vehicle revealed significant overshadowing of the light by the noise. This overshadowing effect was significantly attenuated in rats trained under the opioid antagonist naltrexone, consistent with an opioid-mediated negative feedback model of conditioning. In line with predictions made by negative feedback-type models, we failed to obtain overshadowing with few trials, suggesting that the processes underlying conditioning during initial trials do not contribute to the opioid-dependent Pavlovian overshadowing obtained in our preparation. Lastly, we compared the involvement of dopamine-mediated and opioid-mediated processes in overshadowing by conditioning rats under the partial dopamine D1 receptor agonist SKF 38393 or the opioid antagonist naltrexone. Both naltrexone and SKF 38393 were found to attenuate overshadowing; however, the behavioral profiles produced by each pharmacological manipulation were distinct. Collectively, these studies demonstrate an important role for both opioid- and dopamine-mediated processes in multiple-trial overshadowing.

Central locomotor and cognitive effects of a NPFF receptor agonist in mouse

NPFF receptors are expressed in several brain regions directly or indirectly involved in cognition and behavior. However, the cognitive effects of the NPFF system have been poorly studied. Therefore, the aim of our study was to analyze the effects of i.c.v. injections of 1 DMe, a stable agonist of NPFF receptors, on behavioral and cognitive performances in C57BL/6J mice. We measured locomotor activity, and an open field with objects was used to estimate the ability of mice to react to spatial changes and to measure short-term retention of information. The Morris navigation task was used to evaluate the acquisition, as well as long-term retention of a hippocampo-dependent spatial memory with a distributed training procedure. Finally, 1 DMe was tested in a contextual fear conditioning paradigm to study its effect on long-term memory of contextual information acquired in a single training session. Altogether, our results demonstrate a small but complex influence of the NPFF system on mouse behavior. 1 DMe injected i.c.v. induces a delayed hyperlocomotion and mildly impairs both short-term and long-term spatial memory processing without affecting contextual fear memory.

Extinction of drug- and withdrawal-paired cues in animal models: relevance to the treatment of addiction

Conditioned drug craving and withdrawal elicited by cues paired with drug use or acute withdrawal are among the many factors contributing to compulsive drug taking. Understanding how to stop these cues from having these effects is a major goal of addiction research. Extinction is a form of learning in which associations between cues and the events they predict are weakened by exposure to the cues in the absence of those events. Evidence from animal models suggests that conditioned responses to drug cues can be extinguished, although the degree to which this occurs in humans is controversial. Investigations into the neurobiological substrates of extinction of conditioned drug craving and withdrawal may facilitate the successful use of drug cue extinction within clinical contexts. While this work is still in the early stages, there are indications that extinction of drug- and withdrawal-paired cues shares neural mechanisms with extinction of conditioned fear. Using the fear extinction literature as a template, it is possible to organize the observations on drug cue extinction into a cohesive framework.

Trace and contextual fear conditioning is enhanced in mice lacking the alpha4 subunit of the GABA(A) receptor

The GABA(A)R alpha4 subunit is highly expressed in the dentate gyrus region of the hippocampus at predominantly extra synaptic locations where, along with the GABA(A)R delta subunit, it forms GABA(A) receptors that mediate a tonic inhibitory current. The present study was designed to test hippocampus-dependent and hippocampus-independent learning and memory in GABA(A)R alpha4 subunit-deficient mice using trace and delay fear conditioning, respectively. Mice were of a mixed C57Bl/6J X 129S1/X1 genetic background from alpha4 heterozygous breeding pairs. The alpha4-knockout mice showed enhanced trace and contextual fear conditioning consistent with an enhancement of hippocampus-dependent learning and memory. These enhancements were sex-dependent, similar to previous studies in GABA(A)R delta knockout mice, but differences were present in both males and females. The convergent findings between alpha4 and delta knockout mice suggests that tonic inhibition mediated by alpha4betadelta GABA(A) receptors negatively modulates learning and memory processes and provides further evidence that tonic inhibition makes important functional contributions to learning and behavior.

Complementary roles for amygdala and periaqueductal gray in temporal-difference fear learning

Pavlovian fear conditioning is not a unitary process. At the neurobiological level multiple brain regions and neurotransmitters contribute to fear learning. At the behavioral level many variables contribute to fear learning including the physical salience of the events being learned about, the direction and magnitude of predictive error, and the rate at which these are learned about. These experiments used a serial compound conditioning design to determine the roles of basolateral amygdala (BLA) NMDA receptors and ventrolateral midbrain periaqueductal gray (vlPAG) mu-opioid receptors (MOR) in predictive fear learning. Rats received a three-stage design, which arranged for both positive and negative prediction errors producing bidirectional changes in fear learning within the same subjects during the test stage. Intra-BLA infusion of the NR2B receptor antagonist Ifenprodil prevented all learning. In contrast, intra-vlPAG infusion of the MOR antagonist CTAP enhanced learning in response to positive predictive error but impaired learning in response to negative predictive error--a pattern similar to Hebbian learning and an indication that fear learning had been divorced from predictive error. These findings identify complementary but dissociable roles for amygdala NMDA receptors and vlPAG MOR in temporal-difference predictive fear learning.

Blockade of endogenous opioid neurotransmission enhances acquisition of conditioned fear in humans

The endogenous opioid system is involved in fear learning in rodents, as opioid agonists attenuate and opioid antagonists facilitate the acquisition of conditioned fear. It has been suggested that an opioidergic signal, which is engaged through conditioning and acts inhibitory on unconditioned stimulus input, is the source of these effects. To clarify whether blockade of endogenous opioid neurotransmission enhances acquisition of conditioned fear in humans, and to elucidate the neural underpinnings of such an effect, we used functional magnetic resonance imaging in combination with behavioral recordings and a double-blind pharmacological intervention. All subjects underwent the same classical fear-conditioning paradigm, but subjects in the experimental group received the opioid antagonist naloxone before and during the experiment, in contrast to subjects in the control group, who received saline. Blocking endogenous opioid neurotransmission with naloxone led to more sustained responses to the unconditioned stimulus across trials, evident in both behavioral and blood oxygen level-dependent responses in pain responsive cortical regions. This effect was likely caused by naloxone blocking conditioned responses in a pain-inhibitory circuit involving opioid-rich areas such as the rostral anterior cingulate cortex, amygdala, and periaqueductal gray. Most importantly, naloxone enhanced the acquisition of fear on the behavioral level and changed the activation profile of the amygdala: whereas the control group showed rapidly decaying conditioned responses across trials, the naloxone group showed sustained conditioned responses in the amygdala. Together, these results demonstrate that in humans the endogenous opioid system has an inhibitory role in the acquisition of fear.

Neural mechanisms of extinction learning and retrieval

Emotional learning is necessary for individuals to survive and prosper. Once acquired, however, emotional associations are not always expressed. Indeed, the regulation of emotional expression under varying environmental conditions is essential for mental health. The simplest form of emotional regulation is extinction, in which conditioned responding to a stimulus decreases when the reinforcer is omitted. Two decades of research on the neural mechanisms of fear conditioning have laid the groundwork for understanding extinction. In this review, we summarize recent work on the neural mechanisms of extinction learning. Like other forms of learning, extinction occurs in three phases: acquisition, consolidation, and retrieval, each of which depends on specific structures (amygdala, prefrontal cortex, hippocampus) and molecular mechanisms (receptors and signaling pathways). Pharmacological methods to facilitate consolidation and retrieval of extinction, for both aversive and appetitive conditioning, are setting the stage for novel treatments for anxiety disorders and addictions.

Opioid receptors mediate direct predictive fear learning: evidence from one-trial blocking

Pavlovian fear learning depends on predictive error, so that fear learning occurs when the actual outcome of a conditioning trial exceeds the expected outcome. Previous research has shown that opioid receptors, including mu-opioid receptors in the ventrolateral quadrant of the midbrain periaqueductal gray (vlPAG), mediate such predictive fear learning. Four experiments reported here used a within-subject one-trial blocking design to study whether opioid receptors mediate a direct or indirect action of predictive error on Pavlovian association formation. In Stage I, rats were trained to fear conditioned stimulus (CS) A by pairing it with shock. In Stage II, CSA and CSB were co-presented once and co-terminated with shock. Two novel stimuli, CSC and CSD, were also co-presented once and co-terminated with shock in Stage II. The results showed one-trial blocking of fear learning (Experiment 1) as well as one-trial unblocking of fear learning when Stage II training employed a higher intensity footshock than was used in Stage I (Experiment 2). Systemic administrations of the opioid receptor antagonist naloxone (Experiment 3) or intra-vlPAG administrations of the selective mu-opioid receptor antagonist CTAP (Experiment 4) prior to Stage II training prevented one-trial blocking. These results show that opioid receptors mediate the direct actions of predictive error on Pavlovian association formation.

Renewal after overexpectation

Four magazine approach experiments were performed with rats to explore the decrement produced by reinforcing a compound of two previously conditioned stimuli. Compound presentation produced the usual over-expectation decrement in responding to the individual stimuli. However, testing in an alternative, but similarly treated, context allowed renewal of the initial responding to the individual stimuli. This renewal is similar to that seen after a decrement produced by nonreinforcement. It joins other results in suggesting that the decrement produced in extinction and overexpectation may be due to the same mechanism.

Mechanisms of fear extinction

Excessive fear and anxiety are hallmarks of a variety of disabling anxiety disorders that affect millions of people throughout the world. Hence, a greater understanding of the brain mechanisms involved in the inhibition of fear and anxiety is attracting increasing interest in the research community. In the laboratory, fear inhibition most often is studied through a procedure in which a previously fear conditioned organism is exposed to a fear-eliciting cue in the absence of any aversive event. This procedure results in a decline in conditioned fear responses that is attributed to a process called fear extinction. Extensive empirical work by behavioral psychologists has revealed basic behavioral characteristics of extinction, and theoretical accounts have emphasized extinction as a form of inhibitory learning as opposed to an erasure of acquired fear. Guided by this work, neuroscientists have begun to dissect the neural mechanisms involved, including the regions in which extinction-related plasticity occurs and the cellular and molecular processes that are engaged. The present paper will cover behavioral, theoretical and neurobiological work, and will conclude with a discussion of clinical implications.

Deepened extinction from compound stimulus presentation

Three experiments with rats and 2 with pigeons explored the effect of presenting 2 extinguished excitatory stimuli in compound. Four learning situations were used: Pavlovian magazine approach, Pavlovian fear conditioning, and instrumental discriminative instrumental learning in rats, as well as Pavlovian sign tracking in pigeons. All 5 experiments confirmed D. Reberg's (1972) observation that even after extinction of the individual stimuli, presenting them in compound evoked substantial responding. Moreover, nonreinforcement of that compound deepened extinction of an element more substantially than did additional presentation of that element alone. Such compound exposure reduced spontaneous recovery, reduced reinstatement, and slowed subsequent reconditioning. The primary determinant seemed to be the enhanced associative strength rather than the enhanced conditioned responding that occurred during the nonreinforced compound.

Opioid modulation of Fos protein expression and olfactory circuitry plays a pivotal role in what neonates remember

Paradoxically, fear conditioning (odor-0.5 mA shock) yields a learned odor preference in the neonate, presumably due to a unique learning and memory circuit that does not include apparent amygdala participation. Post-training opioid antagonism with naltrexone (NTX) blocks consolidation of this odor preference and instead yields memory of a learned odor aversion. Here we characterize the neural circuitry underlying this switch during memory consolidation. Experiment 1 assessed post-training opioid modulation of Fos protein expression within olfactory circuitry (olfactory bulb, piriform cortex, amygdala). Odor-shock conditioning with no post-training treatment (odor preference) induced significant changes in Fos protein expression in the granule cell layer of the olfactory bulb and anterior piriform cortex. Post-training opioid receptor antagonism (odor aversion) prevented the learning-induced changes in the anterior piriform cortex and also induced significant changes in Fos protein expression in the central nucleus of the amygdala. Experiment 2 assessed intra-amygdala opioid modulation of neonate memory consolidation. Post-training infusion of NTX within the amygdala permitted consolidation of an odor aversion, while vehicle-infused pups continued to demonstrate an odor preference. Overall, results demonstrate that opioids modulate memory consolidation in the neonate via modulating Fos protein expression in olfactory circuitry. Furthermore, these results suggest that opioids are instrumental in suppressing neonate fear behavior via modulating the amygdala.

Predicting danger: the nature, consequences, and neural mechanisms of predictive fear learning

The ability to detect and learn about the predictive relations existing between events in the world is essential for adaptive behavior. It allows us to use past events to predict the future and to adjust our behavior accordingly. Pavlovian fear conditioning allows anticipation of sources of danger in the environment. It guides attention away from poorer predictors toward better predictors of danger and elicits defensive behavior appropriate to these threats. This article reviews the differences between learning about predictive relations and learning about contiguous relations in Pavlovian fear conditioning. It then describes behavioral approaches to the study of these differences and to the examination of subtle variations in the nature and consequences of predictive learning. Finally, it reviews recent data from rodent and human studies that have begun to identify the neural mechanisms for direct and indirect predictive fear learning.

Facilitation of fear extinction by midbrain periaqueductal gray infusions of RB101(S), an inhibitor of enkephalin-degrading enzymes

Mu-opioid receptors in the ventrolateral quadrant midbrain periaqueductal gray (vIPAG) contribute to extinction of conditioned fear. The present experiment studied whether fear extinction could be facilitated by infusions of a peptidase inhibitor that reduces catabolism of vIPAG enkephalins. Rats were trained to fear an auditory conditioned stimulus. Fear was then extinguished. Extinction training was preceded by infusions of vehicle or RB101(S), an inhibitor of enkephalin catabolising enzymes. RB101(S) dose dependently facilitated extinction as indexed by performance during extinction and on a drug-free test. This facilitation was not observed when RB101(S) was infused outside the vIPAG. These results confirm that vIPAG endogenous opioids contribute to fear extinction and show that extinction can be facilitated by manipulations that increase vIPAG opioid neuromodulation.

Opioid receptors in the nucleus accumbens regulate attentional learning in the blocking paradigm

Fear learning depends on prediction error, or the discrepancy between the actual and expected outcome of a conditioning trial. These experiments used blocking and unblocking designs to study the role of opioid receptors in the nucleus accumbens (Acb) in predictive fear learning. Previous fear conditioning to a context blocked later fear conditioning to a conditioned stimulus (CS) in that context. Fear learning proceeded normally (i.e., unblocking occurred) if the CS signaled a more intense footshock than was used during previous context conditioning. Blocking and unblocking were mediated by Acb opioid receptors. Acb microinjections of a nonselective opioid receptor agonist prevented blocking, whereas a nonselective antagonist prevented unblocking. Examination of the associative mechanism for blocking and unblocking revealed that Acb opioid receptors mediate indirect predictive learning by controlling learned variations in attention. Mu-opioid and kappa-opioid receptors contribute to this learned regulation of attention because Acb microinjections of a mu-opioid receptor agonist impaired, whereas a kappa-opioid receptor agonist facilitated, blocking. Acb microinjections of a mu-opioid receptor antagonist also prevented unblocking. Microinjections of a delta-opioid receptor agonist or antagonist were without effect on blocking and unblocking. Our data show that the Acb mediates attentional selection between competing predictors of motivationally significant events to enable learning about the best predictor of such events at the expense of worse predictors. During fear learning, Acb mu-opioid receptors upregulate attention to conditioned stimuli that are predictive of shock, whereas kappa-opioid receptors downregulate attention to conditioned stimuli that are redundant or noninformative predictors of shock.