Pavlovian Extinction and Recovery Effects in Aversive Pavlovian to Instrumental Transfer

A laboratory model of canine search vigilance decrement, II: Noncontingent reward and Pavlovian appetitive stimuli

Detection dogs have demonstrated reduced performance in operational settings when required to search in an environment where few to no target odors are present. This study's purpose was to increase detection dog accuracy using noncontingent reward (NCR) and Pavlovian stimuli associated with reward. Eighteen dogs were randomly spilt into two groups and received four 40-trial sessions in an operational and training context at 90% odor prevalence (baseline). Following baseline, in the operational context (now at 10% odor prevalence), experimental dogs received an NCR schedule consisting of delivering food rewards at the end of 66% of trials. After the NCR Test, dogs returned to baseline. During baseline, the experimental dogs received 10 days of delayed Pavlovian conditioning to a tone. During the test phase, the conditioned stimulus (tone) was presented to experimental dogs on average every two trials for 30 s in the operational context (now at 10% odor prevalence). Overall, NCR showed a nonsignificant trend for increased responding in the experimental group but tended to increase false alerts; therefore, a permutation of an NCR-like reward schedule may maintain search. The Pavlovian conditioned stimulus didn't decrease timeouts or improve accuracy, but a within-session analysis indicated that the dogs were more likely to time out and less likely to false alert when the tone was on than when it was off.

General and Specific Aversive Modulation of Active Avoidance Require Central Amygdala

Three studies provide evidence that the central nucleus of the amygdala, a structure with a well-established role in conditioned freezing, is also required for conditioned facilitation of instrumental avoidance in rats. First, the immediate early gene c-Fos was measured following the presentation of a previously shock-paired tone in subjects trained either on an unsignaled avoidance task or not (in addition to tone only presentations in naïve controls). Significantly elevated expression of c-Fos was found in both the avoidance trained and Pavlovian trained conditions relative to naïve controls (but with no difference between the two trained conditions). In a subsequent study, intracranial infusions of muscimol into the central amygdala significantly attenuated the facilitation of shock-avoidance by a shock-paired Pavlovian cue relative to pre-operative responding. The final study used a virogenetic approach to inhibit the central amygdala prior to testing. This treatment eliminated the transfer of motivational control over shock-avoidance by both a shock-paired Pavlovian stimulus, as well as a cue paired with a perceptually distinct aversive event (i.e., klaxon). These findings provide compelling support for a role of central amygdala in producing aversive Pavlovian-instrumental transfer.

Conditional Control of Instrumental Avoidance by Context Following Extinction

Using rodents, three training arrangements (i.e., ABB vs. ABA, AAA vs. AAB and ABB vs. ABC) explored whether extinction influences the expression of avoidance in a manner controlled by context. Retention testing following extinction showed that more avoidance responding (i.e., renewal) was observed when extinguished cues were tested outside of the context where they had undergone extinction. In contrast, response rates were significantly lower when stimuli were tested within the context where extinction learning had occurred. These findings add to the emerging literature assessing the role of Pavlovian extinction processes in the development of instrumental avoidance responding by demonstrating conditional control over extinguished responding by context. This study was conducted using a within-subjects approach that minimized the potential for context-outcome associations to bias responding, and thus, reflects hierarchical control over behavior based on the specific associative status of each tested cue in each training context.

The lesser evil: Pavlovian-instrumental transfer & aversive motivation

While our understanding of appetitive motivation includes accounts of rich cognitive phenomena, such as choice, sensory-specificity and outcome valuation, the same is not true in aversive processes. A highly sophisticated picture has emerged of Pavlovian fear conditioning and extinction, but progress in aversive motivation has been somewhat limited to these fundamental behaviors. Many differences between appetitive and aversive stimuli permit different kinds of analyses; a widely used procedure in appetitive studies that can expand the scope of aversive motivation is Pavlovian-instrumental transfer (PIT). Recently, this motivational transfer effect has been used to examine issues pertaining to sensory-specificity and the nature of defensive control in avoidance learning. Given enduring controversies and unresolved criticisms surrounding avoidance research, PIT offers a valuable, well-controlled procedure with which to similarly probe this form of motivation. Furthermore, while avoidance itself can be criticized as artificial, PIT can be an effective model for how skills learned through avoidance can be practically applied to encounters with threatening or fearful stimuli and stress. Despite sensory-related challenges presented by the limited aversive unconditioned stimuli typically used in research, transfer testing can nevertheless provide valuable information on the psychological nature of this historically controversial phenomenon.

Motivational factors underlying aversive Pavlovian-instrumental transfer

While interest in active avoidance has recently been resurgent, many concerns relating to the nature of this form of learning remain unresolved. By separating stimulus and response acquisition, aversive Pavlovian-instrumental transfer can be used to measure the effect of avoidance learning on threat processing with more control than typical avoidance procedures. However, the motivational substrates that contribute to the aversive transfer effect have not been thoroughly examined. In three studies using rodents, the impact of a variety of aversive signals on shock-avoidance responding (i.e., two-way shuttling) was evaluated. Fox urine, as well as a tone paired with the delivery of the predator odor were insufficient modulatory stimuli for the avoidance response. Similarly, a signal for the absence of food did not generate appropriate aversive motivation to enhance shuttling. Only conditioned Pavlovian stimuli that had been paired with unconditioned threats were capable of augmenting shock-avoidance responding. This was true whether the signaled outcome was the same (e.g., shock) or different (e.g., klaxon) from the avoidance outcome (i.e., shock). These findings help to characterize the aversive transfer effect and provide a more thorough analysis of its generalization to warning signals for different kinds of threats. This feature of aversive motivation has not been demonstrated using conventional avoidance procedures and could be potentially useful for applying avoidance in treatment settings.

Chemogenetic Inhibition Reveals That Processing Relative But Not Absolute Threat Requires Basal Amygdala

While our understanding of appetitive motivation has benefited immensely from the use of selective outcome devaluation tools, the same cannot be said about aversive motivation. Findings from appetitive conditioning studies have shown that basal amygdala is required for behaviors that are sensitive to updates in outcome value, but similar results in aversive motivation are difficult to interpret due to a lack of outcome specificity. The studies reported here sought to develop procedures to isolate sensory-specific processes in aversive learning and behavior and to assess the possible contribution of the basal amygdala. Post-training changes to outcome value produced commensurate changes to subsequently tested conditioned responding in male rodents. Specifically, increases in shock intensity (i.e., inflation) augmented, while repeated exposure to (i.e., habituation of) an aversive sound (klaxon-horn) reduced freezing to conditioned stimuli previously paired with these outcomes. This was extended to a discriminative procedure, in which following revaluation of one event, but not the other, responding was found to be dependent on outcome value signaled by each cue. Chemogenetic inactivation of basal amygdala impaired this discrimination between stimuli signaling differently valued outcomes, but did not affect the revaluation process itself. These findings demonstrate a contribution of the basal amygdala to aversive outcome-dependent motivational processes.SIGNIFICANCE STATEMENT The specific content of pavlovian associative learning has been well studied in appetitive motivation, where the value of different foods can be easily manipulated. This has facilitated our understanding of the neural circuits that generate different forms of motivation (i.e., sensory specific vs general). Studies of aversive learning have not produced the same degree of understanding with regard to sensory specificity due to a lack of tools for evaluating sensory-specific processes. Here we use a variant of outcome devaluation procedures with aversive stimuli to study the role of basal amygdala in discriminating between aversive stimuli conveying different degrees of threat. These findings have implications for how we study generalized threat to identify dysregulation that can contribute to generalized anxiety.

Reward and avoidance learning in the context of aversive environments and possible implications for depressive symptoms

BACKGROUND

Aversive stimuli in the environment influence human actions. This includes valence-dependent influences on action selection, e.g., increased avoidance but decreased approach behavior. However, it is yet unclear how aversive stimuli interact with complex learning and decision-making in the reward and avoidance domain. Moreover, the underlying computational mechanisms of these decision-making biases are unknown.

METHODS

To elucidate these mechanisms, 54 healthy young male subjects performed a two-step sequential decision-making task, which allows to computationally model different aspects of learning, e.g., model-free, habitual, and model-based, goal-directed learning. We used a within-subject design, crossing task valence (reward vs. punishment learning) with emotional context (aversive vs. neutral background stimuli). We analyzed choice data, applied a computational model, and performed simulations.

RESULTS

Whereas model-based learning was not affected, aversive stimuli interacted with model-free learning in a way that depended on task valence. Thus, aversive stimuli increased model-free avoidance learning but decreased model-free reward learning. The computational model confirmed this effect: the parameter lambda that indicates the influence of reward prediction errors on decision values was increased in the punishment condition but decreased in the reward condition when aversive stimuli were present. Further, by using the inferred computational parameters to simulate choice data, our effects were captured. Exploratory analyses revealed that the observed biases were associated with subclinical depressive symptoms.

CONCLUSION

Our data show that aversive environmental stimuli affect complex learning and decision-making, which depends on task valence. Further, we provide a model of the underlying computations of this affective modulation. Finally, our finding of increased decision-making biases in subjects reporting subclinical depressive symptoms matches recent reports of amplified Pavlovian influences on action selection in depression and suggests a potential vulnerability factor for mood disorders. We discuss our findings in the light of the involvement of the neuromodulators serotonin and dopamine.

Common neurocircuitry mediating drug and fear relapse in preclinical models

BACKGROUND

Comorbidity of anxiety disorders, stressor- and trauma-related disorders, and substance use disorders is extremely common. Moreover, therapies that reduce pathological fear and anxiety on the one hand, and drug-seeking on the other, often prove short-lived and are susceptible to relapse. Considerable advances have been made in the study of the neurobiology of both aversive and appetitive extinction, and this work reveals shared neural circuits that contribute to both the suppression and relapse of conditioned responses associated with trauma or drug use.

OBJECTIVES

The goal of this review is to identify common neural circuits and mechanisms underlying relapse across domains of addiction biology and aversive learning in preclinical animal models. We focus primarily on neural circuits engaged during the expression of relapse.

KEY FINDINGS

After extinction, brain circuits involving the medial prefrontal cortex and hippocampus come to regulate the expression of conditioned responses by the amygdala, bed nucleus of the stria terminalis, and nucleus accumbens. During relapse, hippocampal projections to the prefrontal cortex inhibit the retrieval of extinction memories resulting in a loss of inhibitory control over fear- and drug-associated conditional responding.

CONCLUSIONS

The overlapping brain systems for both fear and drug memories may explain the co-occurrence of fear and drug-seeking behaviors.