Roles of nucleus accumbens and basolateral amygdala in autoshaped lever pressing

Role of nucleus accumbens D1-type medium spiny neurons in the expression and extinction of sign-tracking

While sign-tracking, also known as autoshaping, has been studied for many decades, only recently has the tendency to show sign-tracking behavior been linked to the development and persistence of addiction. Sign-tracking is dependent upon dopamine activity in the nucleus accumbens (NAc). The NAc is comprised predominantly of medium spiny projection neurons (MSN) that can be differentiated by their D1-like or D2-like dopamine receptor expression. Here we determined how reducing activity of D1-type MSNs in the NAc affects the expression and extinction of sign-tracking. To address this, we transfected the NAc of transgenic male and female rats that selectively express Cre recombinase in D1-type MSNs with a DIO viral vector expressing hM4Di. Cre- rats were given the same viral infusion but did not express the hM4Di receptor and therefore served as controls. Rats were then conditioned to associate lever presentations with pellet delivery. After sign-tracking was established, all rats were administered clozapine-n-oxide (CNO) prior to three additional conditioning sessions to assess the effects of NAc D1-MSNs inhibition on sign-tracking in the presence of reward. CNO treatment did not alter the expression of sign-tracking in Cre+ or Cre- rats. Next rats underwent extinction training where lever presentations occurred without pellet delivery and all rats received a CNO injection prior to each extinction session. In these extinction conditions, Cre+ rats exhibited robust extinction of sign-tracking across sessions, whereas Cre- rats did not. To determine if D1-MSN inhibition merely produced a temporary cessation of sign-tracking or instead had facilitated a persistent loss of sign-tracking, we evaluated the reemergence of sign-tracking in a test for reconditioning. During testing, reintroduction of the CS-US pairing did not promote the reemergence of sign-tracking in Cre+ rats, but restored sign-tracking in Cre- rats. Thus, chemogenetic inhibition of NAc D1-MSNs promoted extinction of sign-tracking. Collectively, these data suggest that D1-MSNs play an important role in resistance to extinction that typifies sign-tracking behavior.

Cue- versus reward-encoding basolateral amygdala projections to nucleus accumbens

In substance use disorders, drug use as unconditioned stimulus (US) reinforces drug taking. Meanwhile, drug-associated cues (conditioned stimulus [CS]) also gain incentive salience to promote drug seeking. The basolateral amygdala (BLA) is implicated in both US- and CS-mediated responses. Here, we show that two genetically distinct BLA neuronal types, expressing Rspo2 versus Ppp1r1b, respectively, project to the nucleus accumbens (NAc) and form monosynaptic connections with both dopamine D1 and D2 receptor-expressing neurons. While intra-NAc stimulation of Rspo2 or Ppp1r1b presynaptic terminals establishes intracranial self-stimulation (ICSS), only Ppp1r1b-stimulated mice exhibit cue-induced ICSS seeking. Furthermore, increasing versus decreasing the Ppp1r1b-to-NAc, but not Rspo2-to-NAc, subprojection increases versus decreases cue-induced cocaine seeking after cocaine withdrawal. Thus, while both BLA-to-NAc subprojections contribute to US-mediated responses, the Ppp1r1b subprojection selectively encodes CS-mediated reward and drug reinforcement. Such differential circuit representations may provide insights into precise understanding and manipulation of drug- versus cue-induced drug seeking and relapse.

Contexts facilitate dynamic value encoding in the mesolimbic dopamine system

Adaptive behavior in a dynamic environment often requires rapid revaluation of stimuli that deviates from well-learned associations. The divergence between stable value-encoding and appropriate behavioral output remains a critical test to theories of dopamine's function in learning, motivation, and motor control. Yet how dopamine neurons are involved in the revaluation of cues when the world changes to alter our behavior remains unclear. Here we make use of pharmacology, in vivo electrophysiology, fiber photometry, and optogenetics to resolve the contributions of the mesolimbic dopamine system to the dynamic reorganization of reward-seeking. Male and female rats were trained to discriminate when a conditioned stimulus would be followed by sucrose reward by exploiting the prior, non-overlapping presentation of a separate discrete cue - an occasion setter. Only when the occasion setter's presentation preceded the conditioned stimulus did the conditioned stimulus predict sucrose delivery. As a result, in this task we were able to dissociate the average value of the conditioned stimulus from its immediate expected value on a trial-to-trial basis. Both the activity of ventral tegmental area dopamine neurons and dopamine signaling in the nucleus accumbens were essential for rats to successfully update behavioral responding in response to the occasion setter. Moreover, dopamine release in the nucleus accumbens following the conditioned stimulus only occurred when the occasion setter indicated it would predict reward. Downstream of dopamine release, we found that single neurons in the nucleus accumbens dynamically tracked the value of the conditioned stimulus. Together these results reveal a novel mechanism within the mesolimbic dopamine system for the rapid revaluation of motivation.

Activation of the basolateral or the central amygdala dampened the incentive motivation for food reward on high fixed-ratio schedules

The amygdala plays crucial roles in emotional processing, motivated behaviors, and stress responses. It receives sensory information and modulates fear- and anxiety-related behaviors. Neuronal activations are induced in the basolateral complex of the amygdala (BLA) and the central nucleus of the amygdala (CeA) when exposing to acute stress, leading to increased alertness and proper behavioral adaptation. Previous studies have shown that animals displayed a decrease in appetitive motivated behaviors under stress conditions. However, whether the hyperactive amygdala is responsible for the decrease in appetitive motivated behaviors remains unknown. In this study, we aimed to examine the role of BLA or CeA activation in effort-based motivated behavior. We pharmacologically activated the BLA or the CeA with N-methyl-D-aspartate (NMDA) before the lever-pressing for food reward test on different fixed-ratio (FR) schedules (FR1, FR16, or FR32) in male Long-Evans rats. Our data showed that activation of either the BLA or the CeA with NMDA (0.05 μg in 0.5 μl per site) decreased the lever-pressing behavior on higher FR schedules of FR16 and FR32, but not on the FR1 test. Importantly, locomotor activity and free-feeding food intake were intact under amygdala activation, suggesting that the decrease in lever-pressing behavior was not due to motor disablement or decreased appetite. These results suggested that activation of the BLA or the CeA negatively impaired the effort-based motivated behavior that the animals were less willing to work for food reward.

Basolateral amygdala and orbitofrontal cortex, but not dorsal hippocampus, are necessary for the control of reward-seeking by occasion setters

Reward-seeking in the world is driven by cues that can have ambiguous predictive and motivational value. To produce adaptive, flexible reward-seeking, it is necessary to exploit occasion setters, other distinct features in the environment, to resolve the ambiguity of Pavlovian reward-paired cues. Despite this, very little research has investigated the neurobiological underpinnings of occasion setting, and as a result little is known about which brain regions are critical for occasion setting. To address this, we exploited a recently developed task that was amenable to neurobiological inquiry where a conditioned stimulus is only predictive of reward delivery if preceded in time by the non-overlapping presentation of a separate cue-an occasion setter. This task required male rats to maintain and link cue-triggered expectations across time to produce adaptive reward-seeking. We interrogated the contributions of the basolateral amygdala and orbitofrontal cortex to occasion setting as these regions are thought to be critical for the computation and exploitation of state value, respectively. Reversible inactivation of either structure prior to the occasion-setting task resulted in a profound inability of rats to use the occasion setter to guide reward-seeking. In contrast, inactivation of the dorsal hippocampus, a region fundamental for context-specific responding was without effect nor did inactivation of the basolateral amygdala or orbitofrontal cortex in a standard Pavlovian conditioning preparation affect conditioned responding. We conclude that neural activity within the orbitofrontal cortex and basolateral amygdala circuit is necessary to update and resolve ambiguity in the environment to promote cue-driven reward-seeking.

Inactivation of the Basolateral Amygdala to Insular Cortex Pathway Makes Sign-Tracking Sensitive to Outcome Devaluation

Goal-tracking (GT) rats are sensitive to Pavlovian outcome devaluation while sign-tracking (ST) rats are devaluation insensitive. During outcome devaluation, GT rats flexibly modify responding to cues based on the current value of the associated outcome. However, ST rats rigidly respond to cues regardless of the current outcome value. Prior work demonstrated disconnection of the basolateral amygdala (BLA) and anterior insular cortex (aIC) decreased both GT and ST behaviors. Given the role of these regions in appetitive motivation and behavioral flexibility, we predicted that disrupting BLA to aIC pathway during outcome devaluation would reduce flexibility in GT rats and reduce rigid appetitive motivation in ST rats. We inhibited the BLA to aIC pathway by infusing inhibitory DREADDs (hM4Di-mcherry) or control (mCherry) virus into the BLA and implanted cannulae into the aIC to inhibit BLA terminals using intracranial injections of clozapine N-oxide (CNO). After training, we used a within-subject satiety-induced outcome devaluation procedure in which we sated rats on training pellets (devalued condition) or homecage chow (valued condition). All rats received bilateral CNO infusions into the aIC before brief nonreinforced test sessions. Contrary to our hypothesis, BLA-IC inhibition did not interfere with devaluation sensitivity in GT rats but did make ST behaviors sensitive to devaluation. Intermediate rats showed the opposite effect, showing rigid responding to cues with BLA-aIC pathway inactivation. Together, these results demonstrate BLA-IC projections mediate tracking-specific Pavlovian devaluation sensitivity and highlights the importance of considering individual differences in Pavlovian approach when evaluating circuitry contributions to behavioral flexibility.

Discordant Effects of Cannabinoid 2 Receptor Antagonism/Inverse Agonism During Adolescence on Pavlovian and Instrumental Reward Learning in Adult Male Rats

The endocannabinoid system is responsible for regulating a spectrum of physiological activities and plays a critical role in the developing brain. During adolescence, the endocannabinoid system is particularly sensitive to external insults that may change the brain’s developmental trajectory. Cannabinoid receptor type 2 (CB2R) was initially thought to predominantly function in the peripheral nervous system, but more recent studies have implicated its role in the mesolimbic pathway, a network largely attributed to reward circuitry and reward motivated behavior, which undergoes extensive changes during adolescence. It is therefore important to understand how CB2R modulation during adolescence can impact reward-related behaviors in adulthood. In this study, adolescent male rats (postnatal days 28–41) were exposed to a low or high dose of the CB2R antagonist/inverse agonist SR144528 and Pavlovian autoshaping and instrumental conditional behavioral outcomes were measured in adulthood. SR144528-treated rats had significantly slower acquisition of the autoshaping task, seen by less lever pressing behavior over time [F_{(2, 19)} = 5.964, p = 0.010]. Conversely, there was no effect of adolescent SR144528 exposure on instrumental conditioning. These results suggest that modulation of the CB2R in adolescence differentially impacts reward-learning behaviors in adulthood.

Choose your path: Divergent basolateral amygdala efferents differentially mediate incentive motivation, flexibility and decision-making

To survive in a complex environment, individuals form associations between environmental stimuli and rewards to organize and optimize reward seeking behaviors. The basolateral amygdala (BLA) uses these learned associations to inform decision-making processes. In this review, we describe functional projections between BLA and its cortical and striatal targets that promote learning and motivational processes central to decision-making. Specifically, we compare and contrast divergent projections from the BLA to the orbitofrontal (OFC) and to the nucleus accumbens (NAc) and examine the roles of these pathways in associative learning, value-guided decision-making, choice behaviors, as well as cue and context-driven drug seeking. Finally, we consider how these projections are involved in disorders of motivation, with a focus on Substance Use Disorder.

Basolateral Amygdala to Nucleus Accumbens Communication Differentially Mediates Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Rats rely on communication between the basolateral amygdala (BLA) and nucleus accumbens (NAc) to express lever directed approach in a Pavlovian lever autoshaping (PLA) task that distinguishes sign- and goal-tracking rats. During PLA, sign-tracking rats preferentially approach an insertable lever cue, while goal-tracking rats approach a foodcup where rewards are delivered. While sign-tracking rats inflexibly respond to cues even after the associated reward is devalued, goal-tracking rats flexibly reduce responding to cues during outcome devaluation. Here, we sought to determine whether BLA-NAc communication, which is necessary for sign, but not goal-tracking, drives a rigid appetitive approach of sign-tracking rats that are insensitive to manipulations of outcome value. Using a contralateral chemogenetic inactivation design, we injected contralateral BLA and NAc core with inhibitory DREADD (hm4Di-mCherry) or control (mCherry) constructs. To determine sign- and goal-tracking groups, we trained rats in five PLA sessions in which brief lever insertion predicts food pellet delivery. We sated rats on training pellets (devalued condition) or chow (valued condition) before systemic clozapine injections (0.1 mg/kg) to inactivate BLA and contralateral NAc during two outcome devaluation probe tests, in which we measured lever and foodcup approach. Contralateral BLA-NAc chemogenetic inactivation promoted a flexible lever approach in sign-tracking rats but disrupted the flexible foodcup approach in goal-tracking rats. Consistent with a prior BLA-NAc disconnection lesion study, we find contralateral chemogenetic inactivation of BLA and NAc core reduces lever, but not the foodcup approach in PLA. Together these findings suggest rigid appetitive associative encoding in BLA-NAc of sign-tracking rats hinders the expression of flexible behavior when outcome value changes.

Neurons of the Ventral Tegmental Area Encode Individual Differences in Motivational "Wanting" for Reward Cues

It has been argued that the dopaminergic system is involved in the attribution of motivational value to reward predictive cues as well as prediction error. To evaluate, dopamine neurons were recorded from male rats performing a Pavlovian approach task containing cues that have both "predictive" and "incentive" properties. All animals learned the predictive nature of the cue (illuminated lever entry into cage), but some also found the cue to be attractive and were motivated toward it ("sign-trackers," STs). "Goal-trackers" (GTs) predominantly approached the location of reward receptacle. Rats were implanted with tetrodes for neural electrophysiological recordings in the ventral tegmental area. Cells were characterized by spike waveform shape and firing rate. Firing rates and magnitudes of responses in relation to Pavlovian behaviors, cue presentation, and reward delivery were assessed. We identified 103 dopamine and 141 nondopamine neurons. GTs and STs both showed responses to the initial lever presentation (CS1) and lever retraction (CS2). However, higher firing rates were sustained during the lever interaction period only in STs. Further, dopamine cells of STs showed a significantly higher proportion of cells responding to both CS1 and CS2. These are the first results to show that neurons from the VTA encode both predictive and incentive cues, support an important role for dopamine neurons in the attribution of incentive salience to reward-paired cues, and underscore the consequences of potential differences in motivational behavior between individuals.SIGNIFICANCE STATEMENT This project serves to determine whether dopamine neurons encode differences in cued approach behaviors and incentive salience. How neurons of the VTA affect signaling through the NAcc and subsequent dopamine release is still not well known. All cues that precede a reward are predictive in nature. Some, however, also have incentive value, in that they elicit approach toward them. We quantified the attribution of incentive salience through cue approach behavior and cue interaction, and the corresponding magnitude of VTA neural firing. We found dopamine neurons of the VTA encode strength of incentive salience of reward cues. This suggests that dopamine neurons specifically in the VTA encode motivation.

Neural circuits linking sleep and addiction: Animal models to understand why select individuals are more vulnerable to substance use disorders after sleep deprivation

Individuals differ widely in their drug-craving behaviors. One reason for these differences involves sleep. Sleep disturbances lead to an increased risk of substance use disorders and relapse in only some individuals. While animal studies have examined the impact of sleep on reward circuitry, few have addressed the role of individual differences in the effects of altered sleep. There does, however, exist a rodent model of individual differences in reward-seeking behavior: the sign/goal-tracker model of Pavlovian conditioned approach. In this model, only some rats show the key behavioral traits associated with addiction, including impulsivity and poor attentional control, making this an ideal model system to examine individually distinct sleep-reward interactions. Here, we describe how the limbic neural circuits responsible for individual differences in incentive motivation overlap with those involved in sleep-wake regulation, and how this model can elucidate the common underlying mechanisms. Consideration of individual differences in preclinical models would improve our understanding of how sleep interacts with motivational systems, and why sleep deprivation contributes to addiction in only select individuals.

The role of the nucleus accumbens in learned approach behavior diminishes with training

Nucleus accumbens dopamine plays a key role in reward-directed approach. Past findings suggest that dopamine's role in the expression of learned behavior diminishes with extended training. However, little is known about the central substrates that mediate the shift to dopamine-independent reward approach. In the present study, rats approached and inserted the head into a reward compartment in response to a cue signaling food delivery. On days 4 and 5 of 28-trial-per-day sessions, D1 receptor antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH23390) infused to the NAc core reduced the probability and speed of cued approach. The disruptive effect of D1 receptor blockade was specific to the nucleus accumbens core and not seen with drug infusions to nearby dopamine target regions. In rats that received drug infusions after extended training (days 10 or 11), accumbens core D1 receptor blockade produced little effect on the expression of the same behavior. These results could have been due to a continued accumbens mediation of cued approach even after the behavior had become independent of accumbens D1 receptors. However, accumbens core ionotropic glutamate receptor blockade disrupted cued approach during early but not late stages of training, similar to the effects of D1 antagonist infusions. The results suggest that with extended training, a nucleus accumbens D1-dependent behavior becomes less dependent not only on nucleus accumbens D1 transmission but also on excitatory transmission in the nucleus accumbens. These findings fill an important gap in a growing literature on reorganization of striatal function over the course of training.

Occasion setting

Occasion setting refers to the ability of 1 stimulus, an occasion setter, to modulate the efficacy of the association between another, conditioned stimulus (CS) and an unconditioned stimulus (US) or reinforcer. Occasion setters and simple CSs are readily distinguished. For example, occasion setters are relatively immune to extinction and counterconditioning, and their combination and transfer functions differ substantially from those of simple CSs. Similarly, the acquisition of occasion setting is favored when stimuli are separated by longer intervals, by empty trace intervals, and are of different modalities, whereas the opposite conditions typically favor the acquisition of simple associations. Furthermore, the simple conditioning and occasion setting properties of a single stimulus can be independent, for example, that stimulus may simultaneously predict the occurrence of a reinforcer and indicate that another stimulus will not be reinforced. Many behavioral phenomena that are intractable to simple associative analysis are better understood within an occasion setting framework. Besides capturing the distinction between direct and modulatory control common to many arenas in neuroscience, occasion setting provides a model for the hierarchical organization of memory for events and event relations, and for contextual control more broadly. Although early lesion studies further differentiated between occasion setting and simple conditioning functions, little is known about the neurobiology of occasion setting. Modern techniques for precise manipulation and monitoring of neuronal activity in multiple brain regions are ideally suited for disentangling contributions of simple conditioning and occasion setting in associative learning. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Sex differences in nicotine-enhanced Pavlovian conditioned approach in rats

Background

Nicotine exposure enhances Pavlovian conditioned approach (PCA), or the learned approach to reward-predictive cues. While females show elevated approach to conditioned stimuli compared to males, potentially indicating heightened addiction vulnerability, it is unknown how sex may interact with nicotine to influence approach behavior. Additionally, brain-derived neurotrophic factor (BDNF) levels can be altered significantly after repeated nicotine exposure, suggesting a potential mechanism contributing to nicotine-induced behavioral phenotypes. The present study investigated the role of sex on nicotine-induced changes to stimulus-response behavior and associated BDNF protein levels.

Methods

Male and female rats were exposed to nicotine (0.4 mg/kg, subcutaneously) or saline 15 min prior to each PCA session. PCA training consisted of 29 sessions of 15 trials, in which a 30-s cue presentation ended concurrently with a sucrose reward (20% w/v in water, 100 μL), and a 120-s variable intertrial interval occurred between trials. Approach behavior to the cue and reward receptacle was recorded. Preference toward the reward receptacle indicated a goal-tracking phenotype, and preference toward the cue indicated a sign-tracking phenotype, demonstrating that the cue had gained incentive salience. Twenty-four hours after the last PCA session, brain tissue was collected and BDNF levels were measured in the basolateral amygdala, orbitofrontal cortex, and nucleus accumbens using Western blot analysis.

Results

Nicotine exposure enhanced both sign- and goal-tracking conditioned approach, and females showed elevated sign-tracking compared to males. There were no sex-by-drug interactions on conditioned approach. Day-to-day variability in conditioned approach was similar between sexes. In contrast to prior studies, neither repeated exposure to nicotine nor sex significantly affected BDNF expression.

Conclusions

Drug-naïve females exhibited heightened sign-tracking compared to males, and nicotine enhanced conditioned approach similarly in males and females. Further, non-significant changes to BDNF expression in brain regions highly associated with PCA indicate that BDNF is unlikely to drive nicotine-enhanced conditioned behavior.

Circuit directionality for motivation: Lateral accumbens-pallidum, but not pallidum-accumbens, connections regulate motivational attraction to reward cues

Sign-tracking behavior, in which animals interact with a cue that predicts reward, provides an example of how incentive salience can be attributed to cues and elicit motivation. The nucleus accumbens (NAc) and ventral pallidum (VP) are two regions involved in cue-driven motivation. The VP, and NAc subregions including the medial shell and core, are critical for sign-tracking. Further, connections between the medial shell and VP are known to participate in sign-tracking and other motivated behaviors. The NAc lateral shell (NAcLSh) is a distinct and understudied subdivision of the NAc, and its contribution to the process by which reward cues acquire value remains unclear. The NAcLSh has been implicated in reward-directed behavior, and has reciprocal connections with the VP, suggesting that NAcLSh and VP interactions could be important mechanisms for incentive salience. Here, we use DREADDs (Designer Receptors Exclusively Activated by Designer Drugs) and an intersectional viral delivery strategy to produce a biased inhibition of NAcLSh neurons projecting to the VP, and vice versa. We find that disruption of connections from NAcLSh to VP reduces sign-tracking behavior while not affecting consumption of food rewards. In contrast, VP to NAcLSh disruption affected neither sign-tracking nor reward consumption, but did produce a greater shift in animals' behavior more towards the reward source when it was available. These findings indicate that the NAcLSh → VP pathway plays an important role in guiding animals towards reward cues, while VP → NAcLSh back-projections may not and may instead bias motivated behavior towards rewards.

Voluntary alcohol access during adolescence/early adulthood, but not during adulthood, causes faster omission contingency learning

In omission contingency training, rodents learn to suppress their natural tendency to approach or touch a reward-predictive cue (termed "autoshaping" or "sign-tracking" responses) if the approach/touching responses lead to the omission of the reward. Previous research has shown that high levels of alcohol exposure (through alcohol vapor exposure) or adolescent alcohol consumption (with some versions of the omission contingency task) can lead to faster omission contingency learning. However, the alcohol exposure procedures and/or omission contingency task parameters differed between these different demonstrations. It was unclear whether the same voluntary alcohol consumption procedures during adolescence/early adulthood and/or adulthood would lead to faster omission contingency learning in one or both age groups. Here, rats received 6 weeks of chronic intermittent access to 20% alcohol or water from PND 26-66 (adolescence/early adulthood in Exp. 1) or PND 68-108 (adulthood in Exp. 2) and began behavioral training (autoshaping training followed by omission contingency training) several days later. We found no evidence that alcohol access at either age altered the number of trials with a sign-tracking response on the levers during autoshaping training. However, alcohol access during adolescence/early adulthood, but not during adulthood, led to faster learning to withhold responding on the lever under omission contingencies during the subsequent phase. We also found no evidence that the level of alcohol consumption was correlated with sign-tracking behavior in the autoshaping phase or with the suppression of lever-pressing during the omission contingency phase. Our results suggest that adolescent/early adult rats have increased vulnerability, compared with adults, to some long-term behavioral effects of voluntary alcohol consumption.

Sign Tracking and Goal Tracking Are Characterized by Distinct Patterns of Nucleus Accumbens Activity

During Pavlovian conditioning, if a cue (e.g., lever extension) predicts reward delivery in a different location (e.g., a food magazine), some individuals will come to approach and interact with the cue, a behavior known as sign tracking (ST), and others will approach the site of reward, a behavior known as goal tracking (GT). In rats, the acquisition of ST versus GT behavior is associated with distinct profiles of dopamine release in the nucleus accumbens (NAc), but it is unknown whether it is associated with different patterns of accumbens neural activity. Therefore, we recorded from individual neurons in the NAc core during the acquisition, maintenance, and extinction of ST and GT behavior. Even though NAc dopamine is specifically important for the acquisition and expression of ST, we found that cue-evoked excitatory responses encode the vigor of both ST and GT behavior. In contrast, among sign trackers only, there was a prominent decrease in reward-related activity over the course of training, which may reflect the decreasing reward prediction error encoded by phasic dopamine. Finally, both behavior and cue-evoked activity were relatively resistant to extinction in sign trackers, as compared with goal trackers, although a subset of neurons in both groups retained their cue-evoked responses. Overall, the results point to the convergence of multiple forms of reward learning in the NAc.

Optogenetic Dissection of Temporal Dynamics of Amygdala-Striatal Interplay during Risk/Reward Decision Making

Decision making often requires weighing costs and benefits of different options that vary in terms of reward magnitude and uncertainty. Previous studies using pharmacological inactivations have shown that the basolateral amygdala (BLA) to nucleus accumbens (NAc) pathway promotes choice towards larger/riskier rewards. Neural activity in BLA and NAc shows distinct, phasic changes in firing prior to choice and following action outcomes, yet, how these temporally-discrete patterns of activity within BLA→NAc circuitry influence choice is unclear. We assessed how optogenetic silencing of BLA terminals in the NAc altered action selection during probabilistic decision making. Rats received intra-BLA infusions of viruses encoding the inhibitory opsin eArchT and were well trained on a probabilistic discounting task, where they chose between smaller/certain rewards and larger rewards delivered in a probabilistic manner, with the odds of obtaining the larger reward changing over a session (50–12.5%). During testing, activity of BLA→NAc inputs were suppressed with 4- to 7-s pulses of light delivered via optic fibers into the NAc during discrete task events: prior to choice or after choice outcomes. Inhibition prior to choice reduced selection of the preferred option, suggesting that during deliberation, BLA→NAc activity biases choice towards preferred rewards. Inhibition during reward omissions increased risky choice during the low-probability block, indicating that activity after non-rewarded actions serves to modify subsequent choice. In contrast, silencing during rewarded outcomes did not reliably affect choice. These data demonstrate how patterns of activity in BLA→NAc circuitry convey different types of information that guide action selection in situations involving reward uncertainty.

Neurochemical fingerprinting of amygdalostriatal and intra-amygdaloid projections: a tracing-immunofluorescence study in the rat

Amygdalostriatal and intra-amygdaloid fiber connectivity was studied in rats via injections of one of the tracers Phaseolus vulgaris leucoagglutinin (PHA-L) or biotinylated dextran amine (BDA) into various amygdaloid nuclei. To determine the neurotransmitter identity of labeled fibers we combined tracer detection with immunofluorescence staining, using antibodies against vesicular transporters (VTs) associated with glutamatergic (VGluT1, VGluT2) or GABAergic (VGAT) neurotransmission. High-magnification confocal laser scanning images were screened for overlap: occurrence inside tracer labeled fibers or axon terminals of immunofluorescence signal associated with one of the VTs. Labeled amygdalostriatal fibers were seen when tracer had been injected into the magnocellular and parvicellular portions of the basal amygdaloid nucleus and the lateral amygdaloid nucleus (nuclei belonging to 'cortical type' amygdaloid nuclei). Intra-amygdaloidal projection fibers were mostly found after tracer injections in the central and medial amygdaloid nuclei ('striatal type' amygdaloid nuclei). Terminals of tracer-labeled amygdalostriatal fibers contained immunofluorescence signal associated mostly with VGluT1 and to a lesser degree with VGluT2 or VGAT. Intra-amygdaloid labeled fibers showed colocalization mostly of VGluT1, followed by VGAT. VGluT2 co-occurred in a minority of intra-amygdaloid tracer-containing fiber terminals. We conclude from our observations that both amygdalostriatal and intra-amygdaloid projections, arising from, respectively, 'cortical type' and 'striatal type' amygdaloid nuclei contain strong glutamatergic and modest GABAergic components. The glutamatergic fibers express either VGluT1 or VGluT2. The absence in large numbers of tracer labeled fibers of expression of one of the selected VTs leads us to suspect that amygdalostriatal projection fibers may contain hitherto neglected neurotransmitters in these connections, e.g., aspartate.

Persistent coding of outcome-predictive cue features in the rat nucleus accumbens

The nucleus accumbens (NAc) is important for learning from feedback, and for biasing and invigorating behaviour in response to cues that predict motivationally relevant outcomes. NAc encodes outcome-related cue features such as the magnitude and identity of reward. However, little is known about how features of cues themselves are encoded. We designed a decision making task where rats learned multiple sets of outcome-predictive cues, and recorded single-unit activity in the NAc during performance. We found that coding of cue identity and location occurred alongside coding of expected outcome. Furthermore, this coding persisted both during a delay period, after the rat made a decision and was waiting for an outcome, and after the outcome was revealed. Encoding of cue features in the NAc may enable contextual modulation of on-going behaviour, and provide an eligibility trace of outcome-predictive stimuli for updating stimulus-outcome associations to inform future behaviour.

Nucleus Accumbens Dopamine D1-Receptor-Expressing Neurons Control the Acquisition of Sign-Tracking to Conditioned Cues in Mice

Following repeated pairings, the reinforcing and motivational properties (incentive salience) of a reward can be transferred onto an environmental stimulus which can then elicit conditioned responses, including Pavlovian approach behavior to the stimulus (a sign-tracking response). In rodents, acquisition of sign-tracking in autoshaping paradigms is sensitive to lesions and dopamine D1 receptor antagonism of the nucleus accumbens (NAc) of the ventral striatum. However, currently, the possible roles of dorsal striatal subregions, as well as of the two major striatal neuron types, dopamine D1-/D2-expressing medium spiny neurons (MSNs), in controlling the development of conditioned responses is still unclear and warrants further study. Here, for the first time, we used a transgenic mouse line combined with striatal subregion-specific AAV virus injections to separately express tetanus toxin in D1-/D2- MSNs in the NAc, dorsomedial striatum, and dorsolateral striatum, to permanently block neurotransmission in these neurons during acquisition of an autoshaping task. Neurotransmission blocking of NAc D1-MSNs inhibited the acquisition of sign-tracking responses when the initial conditioned response for each conditioned stimulus presentation was examined, confirming our initial hypothesis. These findings suggest that activity in NAc D1-MSNs contributes to the attribution of incentive salience to conditioned stimuli.

Behavioral predictors of alcohol drinking in a neurodevelopmental rat model of schizophrenia and co-occurring alcohol use disorder

Alcohol use disorder commonly occurs in patients with schizophrenia and contributes greatly to its morbidity. Unfortunately, the neural and behavioral underpinnings of alcohol drinking in these patients are not well understood. In order to begin to understand the cognitive and reward-related changes that may contribute to alcohol drinking, this study was designed to address: 1) latent inhibition; 2) conditioning; and 3) extinction of autoshaping in a neurodevelopmental rat model with relevance to co-occurring schizophrenia and alcohol use disorders, the neonatal ventral hippocampal lesioned (NVHL) rat. NVHL lesions (or sham surgeries) were performed on post-natal day 7 (PND7) and animals were given brief exposure to alcohol during adolescent (PND 28-42). Latent inhibition of autoshaping, conditioning and extinction were assessed between PND 72-90. On PND90 animals were given alcohol again and allowed to establish stable drinking. Latent inhibition of autoshaping was found to be prolonged in the NVHL rats; the NVHL rats pre-exposed to the lever stimulus were slower to acquire autoshaping than sham pre-exposed rats. NVHL rats that were not pre-exposed to the lever stimulus did not differ during conditioning, but were slower to extinguish conditioned responding compared to sham controls. Finally, the NVHL rats from both groups drank significantly more alcohol than sham rats, and the extent of latent inhibition predicted future alcohol intake in the pre-exposed animals. These findings suggest that the latent inhibition of autoshaping procedure can be used to model cognitive- and reward-related dysfunctions in schizophrenia, and these dysfunctions may contribute to the development of co-occurring alcohol use.

Paradoxical accentuation of motivation following accumbens-pallidum disconnection

The nucleus accumbens (NAc) and ventral pallidum (VP) are reciprocally connected, and activity within this circuit is thought to promote reward learning. Inconsistent with this notion, we find that disconnecting NAc medial shell and VP greatly enhances the attribution of value to a cue that is paired with reward. This result suggests that medial NAc shell and VP are both needed for attributing value to cues yet can also oppose one-another's functional contribution.

Disconnection of basolateral amygdala and insular cortex disrupts conditioned approach in Pavlovian lever autoshaping

Previously established individual differences in appetitive approach and devaluation sensitivity observed in goal- and sign-trackers may be attributed to differences in the acquisition, modification, or use of associative information in basolateral amygdala (BLA) pathways. Here, we sought to determine the extent to which communication of associative information between BLA and anterior portions of insular cortex (IC) supports ongoing Pavlovian conditioned approach behaviors in sign- and goal-tracking rats, in the absence of manipulations to outcome value. We hypothesized that the BLA mediates goal-, but not sign- tracking approach through interactions with the IC, a brain region involved in supporting flexible behavior. We first trained rats in Pavlovian lever autoshaping to determine their sign- or goal-tracking tendency. During alternating test sessions, we gave unilateral intracranial injections of vehicle or a cocktail of gamma-aminobutyric acid (GABA) receptor agonists, baclofen and muscimol, unilaterally into the BLA and contralaterally or ipsilaterally into the IC prior to reinforced lever autoshaping sessions. Consistent with our hypothesis we found that contralateral inactivation of BLA and IC increased the latency to approach the food cup and decreased the number of food cup contacts in goal-trackers. While contralateral inactivation of BLA and IC did not affect the total number of lever contacts in sign-trackers, this manipulation increased the latency to approach the lever. Ipsilateral inactivation of BLA and IC did not impact approach behaviors in Pavlovian lever autoshaping. These findings, contrary to our hypothesis, suggest that communication between BLA and IC maintains a representation of initially learned appetitive associations that commonly support the initiation of Pavlovian conditioned approach behavior regardless of whether it is directed at the cue or the location of reward delivery.

NMDA receptor blockade specifically impedes the acquisition of incentive salience attribution

Glutamatergic signaling plays an important role in learning and memory. Using Pavlovian conditioned approach procedures, the mechanisms that drive stimulus-reward learning and memory have been investigated. However, there are instances where reward-predictive stimuli can function beyond being solely predictive and can be attributed with "motivational value" or incentive salience. Using a Pavlovian conditioned approach procedure consisting of two different but equally predictive stimuli (lever vs. tone) we investigated the role NMDA receptor function has in the attribution of incentive salience. The results revealed that the administration of MK-801, an NMDA receptor antagonist, during acquisition of Pavlovian conditioned approach promoted goal-tracking to a lever stimulus, while control animals learned to sign-track. Moreover, within the same animals, the use of a tone stimulus elicited goal-tracking responses that were unaffected by MK-801 pretreatments. Furthermore, a lever CS that elicited sign-tracking served as a more robust conditioned reinforcer than a tone CS that elicited goal-tracking or a lever CS that elicited goal-tracking via MK-801 pretreatments. Collectively, these results demonstrate that NMDA receptor antagonism can alter the stimulus-reward relationship learned and prevent the attribution of incentive salience, rather than impede general learning.

Increased sign-tracking behavior in adolescent rats

An autoshaping procedure was used to test the notion that conditioned stimuli (CSs) gain greater incentive salience during adolescence than young adulthood under conditions of social isolation rearing and food restriction. Rats were single-housed and placed on food restriction during 10 daily training sessions in which a lever (CS⁺ ) was presented then followed immediately by a food unconditioned stimulus (US). A second lever (CS^- ) was presented on intermixed trials and was not reinforced. Despite the fact that food delivery was not contingent on the rats' behavior, all rats exhibited behaviors directed towards the lever (i.e., sign-tracking). In the adolescent group, the rate of lever pressing and the percentage of trials with a lever press were higher than in young adults. Initially, group differences were observed when rats were retrained when the adolescents had reached young adulthood. These findings support the hypothesis that cues that come to predict reward become imbued with excessive motivational value in adolescents, perhaps contributing to the hyper-responsiveness to reward-related stimuli typically observed during this period of development.

Limbic-motor integration by neural excitations and inhibitions in the nucleus accumbens

The nucleus accumbens (NAc) has often been described as a "limbic-motor interface," implying that the NAc integrates the value of expected rewards with the motor planning required to obtain them. However, there is little direct evidence that the signaling of individual NAc neurons combines information about predicted reward and behavioral response. We report that cue-evoked neural responses in the NAc form a likely physiological substrate for its limbic-motor integration function. Across task contexts, individual NAc neurons in behaving rats robustly encode the reward-predictive qualities of a cue, as well as the probability of behavioral response to the cue, as coexisting components of the neural signal. In addition, cue-evoked activity encodes spatial and locomotor aspects of the behavioral response, including proximity to a reward-associated target and the latency and speed of approach to the target. Notably, there are important limits to the ability of NAc neurons to integrate motivational information into behavior: in particular, updating of predicted reward value appears to occur on a relatively long timescale, since NAc neurons fail to discriminate between cues with reward associations that change frequently. Overall, these findings suggest that NAc cue-evoked signals, including inhibition of firing (as noted here for the first time), provide a mechanism for linking reward prediction and other motivationally relevant factors, such as spatial proximity, to the probability and vigor of a reward-seeking behavioral response.NEW & NOTEWORTHY The nucleus accumbens (NAc) is thought to link expected rewards and action planning, but evidence for this idea remains sparse. We show that, across contexts, both excitatory and inhibitory cue-evoked activity in the NAc jointly encode reward prediction and probability of behavioral responding to the cue, as well as spatial and locomotor properties of the response. Interestingly, although spatial information in the NAc is updated quickly, fine-grained updating of reward value occurs over a longer timescale.

Long-lasting contribution of dopamine in the nucleus accumbens core, but not dorsal lateral striatum, to sign-tracking

The attribution of incentive salience to reward-paired cues is dependent on dopamine release in the nucleus accumbens core (NAcC). These dopamine signals conform to traditional reward-prediction error signals and have been shown to diminish with time. Here we examined whether the diminishing dopamine signal in the NAcC has functional implications for the expression of sign-tracking, a Pavlovian conditioned response indicative of the attribution of incentive salience to reward-paired cues. Food-restricted male Sprague Dawley rats were trained in a Pavlovian paradigm in which an insertable lever predicted delivery of food reward in a nearby food cup. After 7 or 14 training sessions, rats received infusions of saline, the dopamine antagonist flupenthixol, or the GABA agonists baclofen and muscimol into the NAcC or the dorsal lateral striatum (DLS). Dopamine antagonism within the NAcC attenuated sign-tracking, whereas reversible inactivation did not affect sign-tracking but increased non-specific food cup checking behaviors. Neither drug in the DLS affected sign-tracking behavior. Critically, extended training did not alter these effects. Although extended experience with an incentive stimulus may reduce cue-evoked dopamine in the NAcC, this does not remove the dependence on dopamine in this region to promote Pavlovian cue approach nor result in the recruitment of dorsal lateral striatal systems for this behavior. These data support the notion that dopamine within the mesoaccumbal system, but not the nigrostriatal system, contributes critically to incentive motivational processes independent of the length of training.

Sensitive periods of substance abuse: Early risk for the transition to dependence

•

Early substance use dramatically increases the risk of substance use disorder (SUD).

•

Although many try drugs, only a small percentage transition to SUD.

•

High reactivity of reward, habit, and stress systems increase risk.

•

Identification of early risk enables targeted, preventative interventions for SUD.

•

Prevention must start before the sensitive adolescent period to maximize resilience.

Early adolescent substance use dramatically increases the risk of lifelong substance use disorder (SUD). An adolescent sensitive period evolved to allow the development of risk-taking traits that aid in survival; today these may manifest as a vulnerability to drugs of abuse. Early substance use interferes with ongoing neurodevelopment to induce neurobiological changes that further augment SUD risk. Although many individuals use drugs recreationally, only a small percentage transition to SUD. Current theories on the etiology of addiction can lend insights into the risk factors that increase vulnerability from early recreational use to addiction. Building on the work of others, we suggest individual risk for SUD emerges from an immature PFC combined with hyper-reactivity of reward salience, habit, and stress systems. Early identification of risk factors is critical to reducing the occurrence of SUD. We suggest preventative interventions for SUD that can be either tailored to individual risk profiles and/or implemented broadly, prior to the sensitive adolescent period, to maximize resilience to developing substance dependence. Recommendations for future research include a focus on the juvenile and adolescent periods as well as on sex differences to better understand early risk and identify the most efficacious preventions for SUD.

Chemogenetic manipulation of ventral pallidal neurons impairs acquisition of sign-tracking in rats

Cues associated with rewarding events acquire value themselves as a result of the incentive value of the reward being transferred to the cue. Consequently, presentation of a reward-paired cue can trigger reward-seeking behaviours towards the cue itself (i.e. sign-tracking). The ventral pallidum (VP) has been demonstrated to be involved in a number of motivated behaviours, both conditioned and unconditioned. However, its contribution to the acquisition of incentive value is unknown. Using a discriminative autoshaping procedure with levers, the effects of disrupting VP activity in rats on the emergence of sign-tracking was investigated using chemogenetics, i.e. Designer Receptors Exclusively Activated by Designer Drugs (DREADDs). Transient disruption of VP neurons [activation of the inhibitory hM4D(Gi) DREADD through systemic injections of clozapine N-oxide (CNO) prior to each autoshaping session] impaired acquisition of sign-tracking (lever press rate) without having any effect on approach to the site of reward delivery (i.e. goal-tracking) or on the expression of sign-tracking after it was acquired. In addition, electrophysiological recordings were conducted in freely behaving rats following VP DREADD activation. The majority of VP units that were responsive to CNO injections exhibited rapid inhibition relative to baseline, a subset of CNO-responsive units showed delayed excitation, and a smaller subset displayed a mixed response of inhibition and excitation following CNO injections. It is argued that disruption of VP during autoshaping specifically disrupted the transfer of incentive value that was attributed to the lever cue, suggesting a surprisingly fundamental role for the VP in acquiring, compared with expressing, Pavlovian incentive values.

The sensory features of a food cue influence its ability to act as an incentive stimulus and evoke dopamine release in the nucleus accumbens core

The sensory properties of a reward-paired cue (a conditioned stimulus; CS) may impact the motivational value attributed to the cue, and in turn influence the form of the conditioned response (CR) that develops. A cue with multiple sensory qualities, such as a moving lever-CS, may activate numerous neural pathways that process auditory and visual information, resulting in CRs that vary both within and between individuals. For example, CRs include approach to the lever-CS itself (rats that “sign-track”; ST), approach to the location of reward delivery (rats that “goal-track”; GT), or an “intermediate” combination of these behaviors. We found that the multimodal sensory features of the lever-CS were important to the development and expression of sign-tracking. When the lever-CS was covered, and thus could only be heard moving, STs not only continued to approach the lever location but also started to approach the food cup during the CS period. While still predictive of reward, the auditory component of the lever-CS was a much weaker conditioned reinforcer than the visible lever-CS. This plasticity in behavioral responding observed in STs closely resembled behaviors normally seen in rats classified as “intermediates.” Furthermore, the ability of both the lever-CS and the reward-delivery to evoke dopamine release in the nucleus accumbens was also altered by covering the lever—dopamine signaling in STs resembled neurotransmission observed in rats that normally only GT. These data suggest that while the visible lever-CS was attractive, wanted, and had incentive value for STs, when presented in isolation, the auditory component of the cue was simply predictive of reward, lacking incentive salience. Therefore, the specific sensory features of cues may differentially contribute to responding and ensure behavioral flexibility.

Lesions of the ventral hippocampus attenuate the acquisition but not expression of sign-tracking behavior in rats

Individual variation in the attribution of motivational salience to reward-related cues is believed to underlie addiction vulnerability. Pavlovian conditioned approach measures individual variation in motivational salience by identifying rats that are attracted to and motivated by reward cues (sign-trackers) or motivationally fixed on the reward itself (goal-trackers). Previously, it has been demonstrated that sign-trackers are more vulnerable to addiction-like behavior. Moreover, sign-trackers release more dopamine in the nucleus accumbens than goal-trackers in response to reward-related cues, and sign- but not goal-tracking behavior is dopamine-dependent. In the present study, we investigated whether the ventral hippocampus, a potent driver of dopaminergic activity in the nucleus accumbens, modulates the acquisition and expression of Pavlovian conditioned approach behavior. In Experiment 1, lesions of the ventral, but not dorsal or total hippocampus, decreased sign-tracking behavior. In Experiment 2, lesions of the ventral hippocampus did not affect the expression of sign- or goal-tracking behaviors nor conditioned reinforcement. In addition, temporary inactivation of the ventral subiculum, the main output pathway of the ventral hippocampus, did not affect the expression of sign- or goal-tracking behaviors. High-pressure liquid chromatography of nucleus accumbens tissue punches revealed that ventral hippocampal lesions decreased levels of homovanillic acid and the homovanillic acid/dopamine ratio (a marker of dopamine release and metabolism) in only sign-trackers, and decreased accumbal norepinephrine levels in both sign- and goal-trackers. These results suggest that the ventral hippocampus is important for the acquisition but not expression of sign-tracking behavior, possibly as a result of altered dopamine and norepinephrine in the nucleus accumbens. © 2016 Wiley Periodicals, Inc.

Toward isolating the role of dopamine in the acquisition of incentive salience attribution

Stimulus-reward learning has been heavily linked to the reward-prediction error learning hypothesis and dopaminergic function. However, some evidence suggests dopaminergic function may not strictly underlie reward-prediction error learning, but may be specific to incentive salience attribution. Utilizing a Pavlovian conditioned approach procedure consisting of two stimuli that were equally reward-predictive (both undergoing reward-prediction error learning) but functionally distinct in regard to incentive salience (levers that elicited sign-tracking and tones that elicited goal-tracking), we tested the differential role of D1 and D2 dopamine receptors and nucleus accumbens dopamine in the acquisition of sign- and goal-tracking behavior and their associated conditioned reinforcing value within individuals. Overall, the results revealed that both D1 and D2 inhibition disrupted performance of sign- and goal-tracking. However, D1 inhibition specifically prevented the acquisition of sign-tracking to a lever, instead promoting goal-tracking and decreasing its conditioned reinforcing value, while neither D1 nor D2 signaling was required for goal-tracking in response to a tone. Likewise, nucleus accumbens dopaminergic lesions disrupted acquisition of sign-tracking to a lever, while leaving goal-tracking in response to a tone unaffected. Collectively, these results are the first evidence of an intraindividual dissociation of dopaminergic function in incentive salience attribution from reward-prediction error learning, indicating that incentive salience, reward-prediction error, and their associated dopaminergic signaling exist within individuals and are stimulus-specific. Thus, individual differences in incentive salience attribution may be reflective of a differential balance in dopaminergic function that may bias toward the attribution of incentive salience, relative to reward-prediction error learning only.

Effects of amygdala lesions on overexpectation phenomena in food cup approach and autoshaping procedures

Prediction error (PE) plays a critical role in most modern theories of associative learning, by determining the effectiveness of conditioned stimuli (CS) or unconditioned stimuli (US). Here, we examined the effects of lesions of central (CeA) or basolateral (BLA) amygdala on performance in overexpectation tasks. In 2 experiments, after 2 CSs were separately paired with the US, they were combined and followed by the same US. In a subsequent test, we observed losses in strength of both CSs, as expected if the negative PE generated on reinforced compound trials encouraged inhibitory learning. CeA lesions, known to interfere with PE-induced enhancements in CS effectiveness, reduced those losses, suggesting that normally the negative PE also enhances cue associability in this task. BLA lesions had no effect. When a novel cue accompanied the reinforced compound, it acquired net conditioned inhibition, despite its consistent pairings with the US, consonant with US effectiveness models. That acquisition was unaffected by either CeA or BLA lesions, suggesting different rules for assignment of credit of changes in cue strength and cue associability. Finally, we examined a puzzling autoshaping phenomenon previously attributed to overexpectation effects. When a previously food-paired auditory cue was combined with the insertion of a lever and paired with the same food US, the auditory cue not only failed to block conditioning to the lever, but also lost strength, as in an overexpectation experiment. This effect was abolished by BLA lesions but unaffected by CeA lesions, suggesting it was unrelated to other overexpectation effects. (PsycINFO Database Record

Role of prefrontal 5-HT in the strain-dependent variation in sign-tracking behavior of C57BL/6 and DBA/2 mice

RATIONALE

The expression of sign-tracking (ST) phenotype over goal-tracking (GT) phenotype has been associated to different aspects of impulsive behavior, and depletions of brain serotonin (5-HT) have been shown to selectively increase impulsive action as well as ST.

OBJECTIVES

The present study aimed at testing the relationship between reduced brain 5-HT availability and expression of ST phenotype in a genetic model of individual variation in brain 5-HT functionality. Inbred DBA/2J (DBA) mice are homozygous for the allelic variant of the TPH-2 gene causing lower brain 5-HT function in comparison with C57BL/6J (C57) inbred mice.

MATERIALS

Young adult (10 weeks) and adult (14 weeks) C57 and DBA mice were trained in a Pavlovian conditioned approach (PCA) paradigm. Lever-directed (ST) and magazine-directed (GT) responses were measured in 12 daily conditioning sessions. In a second experiment, effect of the medial prefrontal cortex (mPFC) 5-HT depletion by the neurotoxin 5,7-dihydroxytryptamine (5,7-DHT) was assessed on acquisition of ST phenotype in adult C57 mice, according to their higher 5-HT functionality compared to DBA mice.

RESULTS

Young adult mice of both strains developed ST phenotype, but only adult DBA mice developed ST phenotype. 5-HT depletion in the mPFC of adult C57 mice completely changed their phenotype, as shown by their increased ST.

CONCLUSIONS

These findings indicate that ST phenotype can be the expression of a transitory late developmental stage and that genetic factors determine persistence of this phenotype in adulthood. These findings also support a role of 5-HT transmission in PFC in constraining development of ST phenotype.

An omission procedure reorganizes the microstructure of sign-tracking while preserving incentive salience

Appetitive sign-tracking, in which reward-paired cues elicit approach that can result in cue interaction, demonstrates how cues acquire motivational value. For example, rats will approach and subsequently interact with a lever insertion cue that signals food delivery upon its retraction. However, lever deflections are rapidly reduced once rats are trained on an omission schedule in which lever interactions cancel food delivery. Here we evaluated the change in sign-tracking response topography in rats exposed to such an omission procedure. Lever deflections dropped precipitously when they canceled reward. However, rats that were on an omission schedule continued to approach, sniff, and contact the lever without pressing it, and did so at comparable rates to rats that were not under an omission schedule. Thus, sign-tracking was maintained, albeit in a different manner, following omission. Such findings show that the motivational attraction to reward cues can be expressed with remarkable persistence and flexibility.

A Framework for Understanding the Emerging Role of Corticolimbic-Ventral Striatal Networks in OCD-Associated Repetitive Behaviors

Significant interest in the mechanistic underpinnings of obsessive-compulsive disorder (OCD) has fueled research on the neural origins of compulsive behaviors. Converging clinical and preclinical evidence suggests that abnormal repetitive behaviors are driven by dysfunction in cortico-striatal-thalamic-cortical (CSTC) circuits. These findings suggest that compulsive behaviors arise, in part, from aberrant communication between lateral orbitofrontal cortex (OFC) and dorsal striatum. An important body of work focused on the role of this network in OCD has been instrumental to progress in the field. Disease models focused primarily on these regions, however, fail to capture an important aspect of the disorder: affective dysregulation. High levels of anxiety are extremely prevalent in OCD, as is comorbidity with major depressive disorder. Furthermore, deficits in processing rewards and abnormalities in processing emotional stimuli are suggestive of aberrant encoding of affective information. Accordingly, OCD can be partially characterized as a disease in which behavioral selection is corrupted by exaggerated or dysregulated emotional states. This suggests that the networks producing OCD symptoms likely expand beyond traditional lateral OFC and dorsal striatum circuit models, and highlights the need to cast a wider net in our investigation of the circuits involved in generating and sustaining OCD symptoms. Here, we address the emerging role of medial OFC, amygdala, and ventral tegmental area projections to the ventral striatum (VS) in OCD pathophysiology. The VS receives strong innervation from these affect and reward processing regions, and is therefore poised to integrate information crucial to the generation of compulsive behaviors. Though it complements functions of dorsal striatum and lateral OFC, this corticolimbic-VS network is less commonly explored as a potential source of the pathology underlying OCD. In this review, we discuss this network's potential role as a locus of OCD pathology and effective treatment.

Individual variability in behavioral flexibility predicts sign-tracking tendency

Sign-tracking rats show heightened sensitivity to food- and drug-associated cues, which serve as strong incentives for driving reward seeking. We hypothesized that this enhanced incentive drive is accompanied by an inflexibility when incentive value changes. To examine this we tested rats in Pavlovian outcome devaluation or second-order conditioning prior to the assessment of sign-tracking tendency. To assess behavioral flexibility we trained rats to associate a light with a food outcome. After the food was devalued by pairing with illness, we measured conditioned responding (CR) to the light during an outcome devaluation probe test. The level of CR during outcome devaluation probe test correlated with the rats' subsequent tracking tendency, with sign-tracking rats failing to suppress CR to the light after outcome devaluation. To assess Pavlovian incentive learning, we trained rats on first-order (CS+, CS-) and second-order (SOCS+, SOCS-) discriminations. After second-order conditioning, we measured CR to the second-order cues during a probe test. Second-order conditioning was observed across all rats regardless of tracking tendency. The behavioral inflexibility of sign-trackers has potential relevance for understanding individual variation in vulnerability to drug addiction.

The basolateral amygdala in reward learning and addiction

Sophisticated behavioral paradigms partnered with the emergence of increasingly selective techniques to target the basolateral amygdala (BLA) have resulted in an enhanced understanding of the role of this nucleus in learning and using reward information. Due to the wide variety of behavioral approaches many questions remain on the circumscribed role of BLA in appetitive behavior. In this review, we integrate conclusions of BLA function in reward-related behavior using traditional interference techniques (lesion, pharmacological inactivation) with those using newer methodological approaches in experimental animals that allow in vivo manipulation of cell type-specific populations and neural recordings. Secondly, from a review of appetitive behavioral tasks in rodents and monkeys and recent computational models of reward procurement, we derive evidence for BLA as a neural integrator of reward value, history, and cost parameters. Taken together, BLA codes specific and temporally dynamic outcome representations in a distributed network to orchestrate adaptive responses. We provide evidence that experiences with opiates and psychostimulants alter these outcome representations in BLA, resulting in long-term modified action.

Rats that sign-track are resistant to Pavlovian but not instrumental extinction

Individuals vary in the extent to which they attribute incentive salience to a discrete cue (conditioned stimulus; CS) that predicts reward delivery (unconditioned stimulus; US), which results in some individuals approaching and interacting with the CS (sign-trackers; STs) more than others (goal-trackers; GTs). Here we asked how periods of non-reinforcement influence conditioned responding in STs vs. GTs, in both Pavlovian and instrumental tasks. After classifying rats as STs or GTs by pairing a retractable lever (the CS) with the delivery of a food pellet (US), we introduced periods of non-reinforcement, first by simply withholding the US (i.e., extinction training; experiment 1), then by signaling alternating periods of reward (R) and non-reward (NR) within the same session (experiments 2 and 3). We also examined how alternating R and NR periods influenced instrumental responding for food (experiment 4). STs and GTs did not differ in their ability to discriminate between R and NR periods in the instrumental task. However, in Pavlovian settings STs and GTs responded to periods of non-reward very differently. Relative to STs, GTs very rapidly modified their behavior in response to periods of non-reward, showing much faster extinction and better and faster discrimination between R and NR conditions. These results highlight differences between Pavlovian and instrumental extinction learning, and suggest that if a Pavlovian CS is strongly attributed with incentive salience, as in STs, it may continue to bias attention toward it, and to facilitate persistent and relatively inflexible responding, even when it is no longer followed by reward.

Repeated social defeat stress enhances the anxiogenic effect of bright light on operant reward-seeking behavior in rats

Repeated stress can trigger episodes of depression, along with symptoms of anhedonia and anxiety. Although often modeled separately, anxiogenic factors potently modulate hedonic, or appetitive, behavior. While repeated stress can increase anxiety and decrease appetitive behavior, it is not clear whether repeated stress can influence the impact of anxiogenic factors on appetitive behavior. This study tests whether repeated stress shifts behavior in a task that measures anxiogenic-appetitive balance. To test this, adult male rats were trained to lever press for sucrose pellet reward, and the effect of anxiogenic bright light on this behavior was measured. The impact of the bright light anxiogenic stimulus on lever pressing was compared between groups exposed to either daily repeated social defeat stress or control handling. We found that repeated stress reduced exploration in the open field and decreased social interaction, but had minimal effect on baseline lever pressing for reward. Repeated stress substantially enhanced the effect of anxiogenic bright light on lever pressing. This effect was greater two days after the last stress exposure, and began to diminish within two weeks. These data demonstrate that the anxiogenic and anhedonic features induced by repeated stress can be separately measured, and that the impact of anxiogenic stimuli can be greatly enhanced after repeated stress, even in the face of appetitive drive. The data also demonstrate that some apparent anhedonic-like effects of repeated stress can be due to increased sensitivity to anxiogenic stimuli, and may reflect an imbalance in an appetitive approach-withdrawal continuum.

Drug versus sweet reward: greater attraction to and preference for sweet versus drug cues

Despite the unique ability of addictive drugs to directly activate brain reward circuits, recent evidence suggests that drugs induce reinforcing and incentive effects that are comparable to, or even lower than some nondrug rewards. In particular, when rats have a choice between pressing a lever associated with intravenous cocaine or heroin delivery and another lever associated with sweet water delivery, most respond on the latter. This outcome suggests that sweet water is more reinforcing and attractive than either drug. However, this outcome may also be due to the differential ability of sweet versus drug levers to elicit Pavlovian feeding-like conditioned responses that can cause involuntary lever pressing, such as pawing and biting the lever. To test this hypothesis, rats first underwent Pavlovian conditioning to associate one lever with sweet water (0.2% saccharin) and a different lever with intravenous cocaine (0.25 mg) or heroin (0.01 mg). Choice between these two levers was then assessed under two operant choice procedures: one that permitted the expression of Pavlovian-conditioned lever press responses during choice, the other not. During conditioning, Pavlovian-conditioned lever press responses were considerably higher on the sweet lever than on either drug lever, and slightly greater on the heroin lever than on the cocaine lever. Importantly, though these differences in Pavlovian-conditioned behavior predicted subsequent preference for sweet water during choice, they were not required for its expression. Overall, this study confirms that rats prefer the sweet lever because sweet water is more reinforcing and attractive than cocaine or heroin.

Individual variation in the motivational and neurobiological effects of an opioid cue

A discrete cue associated with intravenous injections of cocaine acquires greater control over motivated behavior in some rats ('sign-trackers', STs) than others ('goal-trackers', GTs). It is not known, however, if such variation generalizes to cues associated with other drugs. We asked, therefore, whether a discrete cue (a light) associated with the intravenous administration of an opioid drug (the short-acting mu receptor agonist, remifentanil) acquires incentive motivational properties differently in STs and GTs, as indicated by tests of Pavlovian conditioned approach and conditioned reinforcement. Consistent with studies using cocaine, STs approached a classically conditioned opioid cue more readily than GTs, and in a test of conditioned reinforcement worked more avidly to get it. Interestingly, STs and GTs did not differ in the acquisition of a conditioned orienting response. In addition, the performance of conditioned approach behavior, but not conditioned orientation, was attenuated by pretreatment with the dopamine receptor antagonist, flupenthixol, into the core of the nucleus accumbens. Lastly, food and opioid cues engaged similar amygdalo-striatal-thalamic circuitry to a much greater extent in STs than GTs, as indicated by Fos expression. Taken together, these data demonstrate that, similar to food and cocaine cues: (1) a discrete opioid cue attains greater incentive motivational value in STs than GTs; (2) the attribution of incentive motivational properties to an opioid cue is dopamine dependent; and (3) an opioid cue engages the so-called 'motive circuit' only if it is imbued with incentive salience.

Appetitive associative learning recruits a distinct network with cortical, striatal, and hypothalamic regions

The amygdala, prefrontal cortex, striatum and other connected forebrain areas are important for reward-associated learning and subsequent behaviors. How these structurally and functionally dissociable regions are recruited during initial learning, however, is unclear. Recently, we showed amygdalar nuclei were differentially recruited across different stages of cue-food associations in a Pavlovian conditioning paradigm. Here, we systematically examined Fos induction in the forebrain, including areas associated with the amygdala, during early (day 1) and late (day 10) training sessions of cue-food conditioning. During training, rats in the conditioned group received tone-food pairings, while controls received presentations of the tone alone in the conditioning chamber followed by food delivery in their home cage. We found that a small subset of telencephalic and hypothalamic regions were differentially recruited during the early and late stages of training, suggesting evidence of learning-induced plasticity. Initial tone-food pairings recruited solely the amygdala, while late tone-food pairings came to induce Fos in distinct areas within the medial and lateral prefrontal cortex, the dorsal striatum, and the hypothalamus (lateral hypothalamus and paraventricular nucleus). Furthermore, within the perifornical lateral hypothalamus, tone-food pairings selectively recruited neurons that produce the orexigenic neuropeptide orexin/hypocretin. These data show a functional map of the forebrain areas recruited by appetitive associative learning and dependent on experience. These selectively activated regions include interconnected prefrontal, striatal, and hypothalamic regions that form a discrete but distributed network that is well placed to simultaneously inform cortical (cognitive) processing and behavioral (motivational) control during cue-food learning.

Isolating the incentive salience of reward-associated stimuli: value, choice, and persistence

Sign- and goal-tracking are differentially associated with drug abuse-related behavior. Recently, it has been hypothesized that sign- and goal-tracking behavior are mediated by different neurobehavioral valuation systems, including differential incentive salience attribution. Herein, we used different conditioned stimuli to preferentially elicit different response types to study the different incentive valuation characteristics of stimuli associated with sign- and goal-tracking within individuals. The results demonstrate that all stimuli used were equally effective conditioned stimuli; however, only a lever stimulus associated with sign-tracking behavior served as a robust conditioned reinforcer and was preferred over a tone associated with goal-tracking. Moreover, the incentive value attributed to the lever stimulus was capable of promoting suboptimal choice, leading to a significant reduction in reinforcers (food) earned. Furthermore, sign-tracking to a lever was more persistent than goal-tracking to a tone under omission and extinction contingencies. Finally, a conditional discrimination procedure demonstrated that sign-tracking to a lever and goal-tracking to a tone were dependent on learned stimulus–reinforcer relations. Collectively, these results suggest that the different neurobehavioral valuation processes proposed to govern sign- and goal-tracking behavior are independent but parallel processes within individuals. Examining these systems within individuals will provide a better understanding of how one system comes to dominate stimulus–reward learning, thus leading to the differential role these systems play in abuse-related behavior.

Nucleus accumbens core neurons encode value-independent associations necessary for sensory preconditioning

Reinforcement-based learning models predict that the strength of association between cues and outcomes is driven by aspects of outcome value. However, animals routinely make associations between contingent stimuli in the world, even if those associations hold no value to the organism. At the neural level, the nucleus accumbens (NAc) is known to encode associative information, but it is not known whether this encoding is specific for value-based information (consistent with reinforcement-based models) or if the NAc additionally plays a more general role in forming predictive associations, independent of outcome value. To test this, we employed a sensory preconditioning (SPC) task where rats initially (Preconditioning) received either contingent pairings of 2 neutral stimuli (e.g., tone [A] and light [X]; "Paired"), or random noncontingent presentations ("Unpaired"). After cue X was subsequently conditioned with food (First-Order Conditioning), the effect of preconditioning was assessed in Phase 3 (Test) by presentations of cue A alone. Electrophysiological recordings from the NAc core showed significant increases in phasic encoding for the stimuli in the Paired (but not Unpaired) condition as well as during test. Further, these effects were only seen in Paired rats that showed successful behavior during test (Good Learners), but not those who did not (Poor Learners) or Unpaired controls. These findings reveal a role for the NAc in the encoding of associative contingencies independent of value, and suggest that this structure also plays a more general role in forming associations necessary for predictive behavior.

Neurons in the ventral striatum exhibit cell-type-specific representations of outcome during learning

The ventromedial striatum (VMS) is a node in circuits underpinning both affect and reinforcement learning. The cellular bases of these functions and especially their potential linkages have been unclear. VMS cholinergic interneurons, however, have been singled out as being related both to affect and to reinforcement-based conditioning, raising the possibility that unique aspects of their signaling could account for these functions. Here we show that VMS tonically active neurons (TANs), including putative cholinergic interneurons, generate unique bidirectional outcome responses during reward-based learning, reporting both positive (reward) and negative (reward omission) outcomes when behavioral change is prompted by switches in reinforcement contingencies. VMS output neurons (SPNs), by contrast, are nearly insensitive to switches in reinforcement contingencies, gradually losing outcome signaling while maintaining responses at trial initiation and goal approach. Thus, TANs and SPNs in the VMS provide distinct signals optimized for different aspects of the learning process.