Development of an aversive Pavlovian-to-instrumental transfer task in rat

General and Specific Aversive Modulation of Active Avoidance Require Central Amygdala

Three studies provide evidence that the central nucleus of the amygdala, a structure with a well-established role in conditioned freezing, is also required for conditioned facilitation of instrumental avoidance in rats. First, the immediate early gene c-Fos was measured following the presentation of a previously shock-paired tone in subjects trained either on an unsignaled avoidance task or not (in addition to tone only presentations in naïve controls). Significantly elevated expression of c-Fos was found in both the avoidance trained and Pavlovian trained conditions relative to naïve controls (but with no difference between the two trained conditions). In a subsequent study, intracranial infusions of muscimol into the central amygdala significantly attenuated the facilitation of shock-avoidance by a shock-paired Pavlovian cue relative to pre-operative responding. The final study used a virogenetic approach to inhibit the central amygdala prior to testing. This treatment eliminated the transfer of motivational control over shock-avoidance by both a shock-paired Pavlovian stimulus, as well as a cue paired with a perceptually distinct aversive event (i.e., klaxon). These findings provide compelling support for a role of central amygdala in producing aversive Pavlovian-instrumental transfer.

The lesser evil: Pavlovian-instrumental transfer & aversive motivation

While our understanding of appetitive motivation includes accounts of rich cognitive phenomena, such as choice, sensory-specificity and outcome valuation, the same is not true in aversive processes. A highly sophisticated picture has emerged of Pavlovian fear conditioning and extinction, but progress in aversive motivation has been somewhat limited to these fundamental behaviors. Many differences between appetitive and aversive stimuli permit different kinds of analyses; a widely used procedure in appetitive studies that can expand the scope of aversive motivation is Pavlovian-instrumental transfer (PIT). Recently, this motivational transfer effect has been used to examine issues pertaining to sensory-specificity and the nature of defensive control in avoidance learning. Given enduring controversies and unresolved criticisms surrounding avoidance research, PIT offers a valuable, well-controlled procedure with which to similarly probe this form of motivation. Furthermore, while avoidance itself can be criticized as artificial, PIT can be an effective model for how skills learned through avoidance can be practically applied to encounters with threatening or fearful stimuli and stress. Despite sensory-related challenges presented by the limited aversive unconditioned stimuli typically used in research, transfer testing can nevertheless provide valuable information on the psychological nature of this historically controversial phenomenon.

mGlu5: A double-edged sword for aversive learning related therapeutics

Aversive memories underlie many types of anxiety disorders. One area of research to more effectively treat anxiety disorders has therefore been identifying pharmacological targets to affect memory processes. Among these targets, the metabotropic glutamate 5 receptor (mGlu5) has received attention due to the availability of drugs to utilize its role in learning and memory. In this review, we highlight preclinical studies examining the role of mGlu5 at various stages of aversive learning and its inhibition via extinction in order to gain a better understanding of its therapeutic potential. We suggest that mGlu5 has distinct roles at different stages of memory that not only makes it a tricky target, but a double-edged sword as a therapeutic. However, the selective involvement of mGlu5 in different memory stages allows for certain precision that could be harnessed clinically. We therefore suggest potential applications, limitations, and pitfalls when considering use of mGlu5 modulators as therapeutics. In addition, we recommend future studies to address important gaps in this literature, such as sex and age factors in light of anxiety disorders being more prevalent in those demographics.

Motivational factors underlying aversive Pavlovian-instrumental transfer

While interest in active avoidance has recently been resurgent, many concerns relating to the nature of this form of learning remain unresolved. By separating stimulus and response acquisition, aversive Pavlovian-instrumental transfer can be used to measure the effect of avoidance learning on threat processing with more control than typical avoidance procedures. However, the motivational substrates that contribute to the aversive transfer effect have not been thoroughly examined. In three studies using rodents, the impact of a variety of aversive signals on shock-avoidance responding (i.e., two-way shuttling) was evaluated. Fox urine, as well as a tone paired with the delivery of the predator odor were insufficient modulatory stimuli for the avoidance response. Similarly, a signal for the absence of food did not generate appropriate aversive motivation to enhance shuttling. Only conditioned Pavlovian stimuli that had been paired with unconditioned threats were capable of augmenting shock-avoidance responding. This was true whether the signaled outcome was the same (e.g., shock) or different (e.g., klaxon) from the avoidance outcome (i.e., shock). These findings help to characterize the aversive transfer effect and provide a more thorough analysis of its generalization to warning signals for different kinds of threats. This feature of aversive motivation has not been demonstrated using conventional avoidance procedures and could be potentially useful for applying avoidance in treatment settings.

Psychological mechanisms and functions of 5-HT and SSRIs in potential therapeutic change: Lessons from the serotonergic modulation of action selection, learning, affect, and social cognition

Uncertainty regarding which psychological mechanisms are fundamental in mediating SSRI treatment outcomes and wide-ranging variability in their efficacy has raised more questions than it has solved. Since subjective mood states are an abstract scientific construct, only available through self-report in humans, and likely involving input from multiple top-down and bottom-up signals, it has been difficult to model at what level SSRIs interact with this process. Converging translational evidence indicates a role for serotonin in modulating context-dependent parameters of action selection, affect, and social cognition; and concurrently supporting learning mechanisms, which promote adaptability and behavioural flexibility. We examine the theoretical basis, ecological validity, and interaction of these constructs and how they may or may not exert a clinical benefit. Specifically, we bridge crucial gaps between disparate lines of research, particularly findings from animal models and human clinical trials, which often seem to present irreconcilable differences. In determining how SSRIs exert their effects, our approach examines the endogenous functions of 5-HT neurons, how 5-HT manipulations affect behaviour in different contexts, and how their therapeutic effects may be exerted in humans - which may illuminate issues of translational models, hierarchical mechanisms, idiographic variables, and social cognition.

Dopamine modulates individual differences in avoidance behavior: A pharmacological, immunohistochemical, neurochemical and volumetric investigation

Avoidance behavior is a hallmark in pathological anxiety disorders and results in impairment of daily activities. Individual differences in avoidance responses are critical in determining vulnerability or resistance to anxiety disorders. Dopaminergic activation is implicated in the processing of avoidance responses; however, the mechanisms underlying these responses are unknown. In this sense, we used a preclinical model of avoidance behavior to investigate the possibility of an intrinsic differential dopaminergic pattern between good and poor performers. The specific goal was to assess the participation of dopamine (DA) through pharmacological manipulation, and we further evaluated the effects of systemic injections of the dopaminergic receptor type 1 (D1 antagonist - SCH23390) and dopaminergic receptor type 2 (D2 antagonist - sulpiride) antagonists in the good performers. Additionally, we evaluated the effects of intra-amygdala microinjection of a D1 antagonist (SCH23390) and a D2 antagonist (sulpiride) in good performers as well as intra-amygdala microinjection of a D1 agonist (SKF38393) and D2 agonist (quinpirole) in poor performers. Furthermore, we quantified the contents of dopamine and metabolites (3,4-dihydroxyphenylacetic acid (DOPAC) and homovanillic acid (HVA)) in the amygdala, evaluated the basal levels of tyrosine hydroxylase expression (catecholamine synthesis enzyme) and measured the volume of the substantia nigra, ventral tegmental area and locus coeruleus. Our results showed that it could be possible to convert animals from good to poor performers, and vice versa, by intra-amygdala (basolateral and central nucleus) injections of D1 receptor antagonists in good performers or D2 receptor agonists in poor performers. Additionally, the good performers had lower levels of DOPAC and HVA in the amygdala, an increase in the total volume of the amygdala (AMG), substantia nigra (SN), ventral tegmental area (VTA) and locus coeruleus (LC), and an increase in the number of tyrosine hydroxylase-positive cells in SN, VTA and LC, which positively correlates with the avoidance behavior. Taken together, our data show evidence for a dopaminergic signature of avoidance performers, emphasizing the role of distinct dopaminergic receptors in individual differences in avoidance behavior based on pharmacological, immunohistochemical, neurochemical and volumetric analyses. Our findings provide a better understanding of the role of the dopaminergic system in the execution of avoidance behavior.

•

The role of dopamine in individual differences in avoidance behavior.

•

Dopamine modulates avoidance behavior.

•

Dopaminergic evidence of individual difference in avoidance behavior.

•

Good and poor avoiders distinction based on dopaminergic signature.

•

Dopaminergic signature of avoidance performers: poor versus good avoiders.

Motivational sensitivity of outcome-response priming: Experimental research and theoretical models

Outcome-response (O-R) priming is at the core of various associative theories of human intentional action. This is a simple and parsimonious mechanism by which activation of outcome representations (e.g. thinking about the light coming on) leads to activation of the associated motor patterns required to achieve it (e.g. pushing the light switch). In the current manuscript, we review the evidence for such O-R associative links demonstrated by converging (yet until now, separate) strands of research. While there is a wealth of evidence that both the perceptual and motivational properties of an outcome can be encoded in the O-R association and mediate O-R priming, we critically examine the integration of these mechanisms and the conditions under which motivational factors constrain the sensory O-R priming effect. We discuss the clinical relevance of this O-R priming mechanism, whether it can satisfactorily account for human goal-directed behaviour, and the implications for theories of human action control.

Pattern of access determines influence of junk food diet on cue sensitivity and palatability

AIMS

Like drug addiction, cues associated with palatable foods can trigger food-seeking, even when sated. However, whether susceptibility to the motivating influence of food-related cues is a predisposing factor in overeating or a consequence of poor diet is difficult to determine in humans. Using a rodent model, we explored whether a highly palatable 'junk food' diet impacts responses to reward-paired cues in a Pavlovian-to-instrumental transfer test, using sweetened condensed milk (SCM) as the reward. The hedonic impact of SCM consumption was also assessed by analyzing licking microstructure.

METHODS

To probe the effects of pattern and duration of junk food exposure, we provided rats with either regular chow ad libitum (controls) or chow plus access to junk food for either 2 or 24 h per day for 1, 3, or 6 weeks. We also examined how individual susceptibility to weight gain related to these measures.

RESULTS

Rats provided 24 h access to the junk food diet were insensitive to the motivational effects of a SCM-paired cue when tested sated even though their hedonic experience upon reward consumption was similar to controls. In contrast, rats provided restricted, 2 h access to junk food exhibited a cue generalization phenotype under sated conditions, lever-pressing with increased vigor in response to both a SCM-paired cue, and a cue not previously paired with reward. Hedonic response was also significantly higher in these animals relative to controls.

CONCLUSIONS

These data demonstrate that the pattern of junk food exposure differentially alters the hedonic impact of palatable foods and susceptibility to the motivating influence of cues in the environment to promote food-seeking actions when sated, which may be consequential for understanding overeating and obesity.

Pavlovian Extinction and Recovery Effects in Aversive Pavlovian to Instrumental Transfer

Three studies explored the sensitivity of aversive Pavlovian to instrumental transfer (PIT) to Pavlovian extinction in rodents. Rats underwent Pavlovian conditioning prior to avoidance training. The PIT test then involved assessment of the effects of the Pavlovian conditioned stimulus (CS) on the performance of the avoidance response (AR). Conducting extinction prior to avoidance training and transfer testing, allowed spontaneous recovery and shock reinstatement of extinguished motivation, whereas conducting extinction following avoidance training and just prior to PIT testing successfully reduced transfer effects. This was also the case in a design that compared responding to an extinguished CS against a non-extinguished CS rather than comparing extinguished and non-extinguished groups to one another. While extinction treatments in many appetitive PIT studies do not successfully reduce transfer, and can sometimes enhance the effect, the current findings show that an extinction treatment temporally close to transfer testing can reduce the motivational impact of the aversive Pavlovian CS on instrumental avoidance responding.

Noradrenergic Regulation of Central Amygdala in Aversive Pavlovian-to-Instrumental Transfer

The neural mechanisms through which a Pavlovian conditioned stimulus (CS) elicits innate defense responses are well understood. But a Pavlovian CS can also invigorate ongoing instrumental responding, as shown by studies of aversive Pavlovian-to-instrumental transfer (PIT). While the neural circuitry of appetitive PIT has been studied extensively, little is known about the brain mechanisms of aversive PIT. We recently showed the central amygdala (CeA) is essential for aversive PIT. In the current studies, using pharmacology and designer receptors in rodents, we demonstrate that noradrenergic (NE) activity negatively regulates PIT via brainstem locus coeruleus (LC) activity and LC projections to CeA. Our results provide evidence for a novel pathway through which response modulation occurs between brainstem neuromodulatory systems and CeA to invigorate adaptive behavior in the face of threat.

The Neural Foundations of Reaction and Action in Aversive Motivation

Much of the early research in aversive learning concerned motivation and reinforcement in avoidance conditioning and related paradigms. When the field transitioned toward the focus on Pavlovian threat conditioning in isolation, this paved the way for the clear understanding of the psychological principles and neural and molecular mechanisms responsible for this type of learning and memory that has unfolded over recent decades. Currently, avoidance conditioning is being revisited, and with what has been learned about associative aversive learning, rapid progress is being made. We review, below, the literature on the neural substrates critical for learning in instrumental active avoidance tasks and conditioned aversive motivation.

Prospective and Pavlovian mechanisms in aversive behaviour

Studying aversive behaviour is critical for understanding negative emotions and associated psychopathologies. However a comprehensive picture of the mechanisms underlying aversion is lacking, with associative learning theories focusing on Pavlovian reactions and decision-making theoretic approaches on prospective functions. We propose a computational model of aversion that combines goal-directed and Pavlovian forms of control into a unifying framework in which their relative importance is regulated by factors such as threat distance and controllability. Using simulations, we test whether the model can reproduce available empirical findings and discuss its relevance to understanding factors underlying negative emotions such as fear and anxiety. Furthermore, the specific method used to construct the model permits a natural mapping from its components to brain structure and function. Our model provides a basis for a unifying account of aversion that can guide empirical and interventional study contexts.

Modulation of instrumental responding by a conditioned threat stimulus requires lateral and central amygdala

Two studies explored the role of the amygdala in response modulation by an aversive conditioned stimulus (CS) in rats. Experiment 1 investigated the role of amygdala circuitry in conditioned suppression using a paradigm in which licking for sucrose was inhibited by a tone CS that had been previously paired with footshock. Electrolytic lesions of the lateral amygdala (LA) impaired suppression relative to sham-operated animals, and produced the same pattern of results when applied to central amygdala. In addition, disconnection of the lateral and central amygdala, by unilateral lesion of each on opposite sides of the brain, also impaired suppression relative to control subjects that received lesions of both areas on the same side. In each case, lesions were placed following Pavlovian conditioning and instrumental training, but before testing. This procedure produced within-subjects measures of the effects of lesion on freezing and between-group comparisons for the effects on suppression. Experiment 2 extended this analysis to a task where an aversive CS suppressed shuttling responses that had been previously food reinforced and also found effects of bilateral lesions of the central amygdala in a pre-post design. Together, these studies demonstrate that connections between the lateral and central amygdala constitute a serial circuit involved in processing aversive Pavlovian stimuli, and add to a growing body of findings implicating central amygdala in the modulation of instrumental behavior.

Learning and Motivational Processes Contributing to Pavlovian-Instrumental Transfer and Their Neural Bases: Dopamine and Beyond

Pavlovian stimuli exert a range of effects on behavior from simple conditioned reflexes, such as salivation, to altering the vigor and direction of instrumental actions. It is currently accepted that these distinct behavioral effects stem from two sources (i) the various associative connections between predictive stimuli and the component features of the events that these stimuli predict and (ii) the distinct motivational and cognitive functions served by cues, particularly their arousing and informational effects on the selection and performance of specific actions. Here, we describe studies that have assessed these latter phenomena using a paradigm that has come to be called Pavlovian-instrumental transfer. We focus first on behavioral experiments that have described distinct sources of stimulus control derived from the general affective and outcome-specific predictions of conditioned stimuli, referred to as general transfer and specific transfer, respectively. Subsequently, we describe research efforts attempting to establish the neural bases of these transfer effects, largely in the afferent and efferent connections of the nucleus accumbens (NAc) core and shell. Finally, we examine the role of predictive cues in examples of aberrant stimulus control associated with psychiatric disorders and addiction.

Medial amygdala lesions selectively block aversive pavlovian-instrumental transfer in rats

Pavlovian conditioned stimuli (CSs) play an important role in the reinforcement and motivation of instrumental active avoidance (AA). Conditioned threats can also invigorate ongoing AA responding [aversive Pavlovian-instrumental transfer (PIT)]. The neural circuits mediating AA are poorly understood, although lesion studies suggest that lateral, basal, and central amygdala nuclei, as well as infralimbic prefrontal cortex, make key, and sometimes opposing, contributions. We recently completed an extensive analysis of brain c-Fos expression in good vs. poor avoiders following an AA test (Martinez et al., 2013, Learning and Memory). This analysis identified medial amygdala (MeA) as a potentially important region for Pavlovian motivation of instrumental actions. MeA is known to mediate defensive responding to innate threats as well as social behaviors, but its role in mediating aversive Pavlovian-instrumental interactions is unknown. We evaluated the effect of MeA lesions on Pavlovian conditioning, Sidman two-way AA conditioning (shuttling) and aversive PIT in rats. Mild footshocks served as the unconditioned stimulus in all conditioning phases. MeA lesions had no effect on AA but blocked the expression of aversive PIT and 22 kHz ultrasonic vocalizations in the AA context. Interestingly, MeA lesions failed to affect Pavlovian freezing to discrete threats but reduced freezing to contextual threats when assessed outside of the AA chamber. These findings differentiate MeA from lateral and central amygdala, as lesions of these nuclei disrupt Pavlovian freezing and aversive PIT, but have opposite effects on AA performance. Taken together, these results suggest that MeA plays a selective role in the motivation of instrumental avoidance by general or uncertain Pavlovian threats.

Lesions of lateral or central amygdala abolish aversive Pavlovian-to-instrumental transfer in rats

Aversive Pavlovian conditioned stimuli (CSs) elicit defensive reactions (e.g., freezing) and motivate instrumental actions like active avoidance (AA). Pavlovian reactions require connections between the lateral (LA) and central (CeA) nuclei of the amygdala, whereas AA depends on LA and basal amygdala (BA). Thus, the neural circuits mediating conditioned reactions and motivation appear to diverge in the amygdala. However, AA is not ideal for studying conditioned motivation, because Pavlovian and instrumental learning are intermixed. Pavlovian-to-instrumental transfer (PIT) allows for the study of conditioned motivation in isolation. PIT refers to the ability of a Pavlovian CS to modulate a separately-trained instrumental action. The role of the amygdala in aversive PIT is unknown. We designed an aversive PIT procedure in rats and tested the effects of LA, BA, and CeA lesions. Rats received Pavlovian tone-shock pairings followed by Sidman shock-avoidance training. PIT was assessed by comparing shuttling rates in the presence and absence of the tone. Tone presentations facilitated instrumental responding. Aversive PIT was abolished by lesions of LA or CeA, but was unaffected by lesions of BA. These results suggest that LA and CeA are essential for aversive conditioned motivation. More specifically, the results are consistent with a model of amygdala processing in which the CS is encoded in the LA and then, via connections to CeA, the motivation to perform the aversive task is enhanced. These findings have implications for understanding the contribution of amygdala circuits to aversive instrumental motivation, but also for the relation of aversive and appetitive behavioral control.