At the limbic-motor interface: disconnection of basolateral amygdala from nucleus accumbens core and shell reveals dissociable components of incentive motivation

Response-independent outcome presentations dissociate stimulus and value based choice

A stimulus that predicts the delivery of a specific food outcome can bias performance towards instrumental actions that earn that same outcome in a phenomenon known as specific Pavlovian-instrumental transfer (PIT). The precise mechanism by which the specific instrumental action is selected under these circumstances has remained elusive. The present set of experiments explored whether treatments that undermine the response-outcome (R-O) association also affect the expression of specific PIT. Consistent with previous work, in Experiment 1 we showed that specific PIT remains intact after an instrumental degradation treatment that attempted to undermine R-O associations. However, we additionally demonstrated that outcome-devaluation sensitivity also persisted after degradation, suggesting that R-O associations were impervious to the degradation treatment, and precluding any conclusions about the necessity of R-O associations for specific PIT expression. Nevertheless, given the two-lever two-outcome design of this experiment it is possible that R-O associations were indeed undermined by degradation and that the devaluation effect was driven by distinct, incidental Pavlovian lever-outcome associations. To nullify the obscuring effects of these incidental Pavlovian associations, we used a bidirectional lever for instrumental conditioning that could be pushed to the left or the right for distinct outcomes. In Experiment 2 we demonstrated that specific PIT could be observed on this bidirectional manipulandum whether the subjects were hungry or sated, consistent with the literature. The critical third Experiment used an identical design to Experiment 1 except that the two instrumental responses were made on the single bidirectional manipulanda. Here, specific PIT was intact after instrumental degradation and, crucially, we saw no evidence of outcome devaluation sensitivity in these same subjects, suggesting that the R-O associations were weakened or undermined by this treatment. We conclude that the expression of specific PIT is resistant to treatments that undermine R-O associations and disrupt value based choice, and discuss how these findings contribute to our understanding of the associative framework supporting behavioral control.

Ventral frontostriatal circuitry mediates the computation of reinforcement from symbolic gains and losses

Reinforcement learning (RL), particularly in primates, is often driven by symbolic outcomes. However, it is usually studied with primary reinforcers. To examine the neural mechanisms underlying learning from symbolic outcomes, we trained monkeys on a task in which they learned to choose options that led to gains of tokens and avoid choosing options that led to losses of tokens. We then recorded simultaneously from the orbitofrontal cortex (OFC), ventral striatum (VS), amygdala (AMY), and mediodorsal thalamus (MDt). We found that the OFC played a dominant role in coding token outcomes and token prediction errors. The other areas contributed complementary functions, with the VS coding appetitive outcomes and the AMY coding the salience of outcomes. The MDt coded actions and relayed information about tokens between the OFC and VS. Thus, the OFC leads the processing of symbolic RL in the ventral frontostriatal circuitry.

Behavioral outputs and overlapping circuits between conditional fear and active avoidance

Aversive learning can produce a wide variety of defensive behavioral responses depending on the circumstances, ranging from reactive responses like freezing to proactive avoidance responses. While most of this initial learning is behaviorally supported by an expectancy of an aversive outcome and neurally supported by activity within the basolateral amygdala, activity in other brain regions become necessary for the execution of defensive strategies that emerge in other aversive learning paradigms such as active avoidance. Here, we review the neural circuits that support both reactive and proactive defensive behaviors that are motivated by aversive learning, and identify commonalities between the neural substrates of these distinct (and often exclusive) behavioral strategies.

Ventral frontostriatal circuitry mediates the computation of reinforcement from symbolic gains and losses

Reinforcement learning (RL), particularly in primates, is often driven by symbolic outcomes. However, it is usually studied with primary reinforcers. To examine the neural mechanisms underlying learning from symbolic outcomes, we trained monkeys on a task in which they learned to choose options that led to gains of tokens and avoid choosing options that led to losses of tokens. We then recorded simultaneously from the orbitofrontal cortex (OFC), ventral striatum (VS), amygdala (AMY), and the mediodorsal thalamus (MDt). We found that the OFC played a dominant role in coding token outcomes and token prediction errors. The other areas contributed complementary functions with the VS coding appetitive outcomes and the AMY coding the salience of outcomes. The MDt coded actions and relayed information about tokens between the OFC and VS. Thus, OFC leads the process of symbolic reinforcement learning in the ventral frontostriatal circuitry.

Cue- versus reward-encoding basolateral amygdala projections to nucleus accumbens

In substance use disorders, drug use as unconditioned stimulus (US) reinforces drug taking. Meanwhile, drug-associated cues (conditioned stimulus [CS]) also gain incentive salience to promote drug seeking. The basolateral amygdala (BLA) is implicated in both US- and CS-mediated responses. Here, we show that two genetically distinct BLA neuronal types, expressing Rspo2 versus Ppp1r1b, respectively, project to the nucleus accumbens (NAc) and form monosynaptic connections with both dopamine D1 and D2 receptor-expressing neurons. While intra-NAc stimulation of Rspo2 or Ppp1r1b presynaptic terminals establishes intracranial self-stimulation (ICSS), only Ppp1r1b-stimulated mice exhibit cue-induced ICSS seeking. Furthermore, increasing versus decreasing the Ppp1r1b-to-NAc, but not Rspo2-to-NAc, subprojection increases versus decreases cue-induced cocaine seeking after cocaine withdrawal. Thus, while both BLA-to-NAc subprojections contribute to US-mediated responses, the Ppp1r1b subprojection selectively encodes CS-mediated reward and drug reinforcement. Such differential circuit representations may provide insights into precise understanding and manipulation of drug- versus cue-induced drug seeking and relapse.

Opioid Withdrawal Abruptly Disrupts Amygdala Circuit Function by Reducing Peptide Actions

While the physical signs of opioid withdrawal are most readily observable, withdrawal insidiously drives relapse and contributes to compulsive drug use, by disrupting emotional learning circuits. How these circuits become disrupted during withdrawal is poorly understood. Because amygdala neurons mediate relapse, and are highly opioid sensitive, we hypothesized that opioid withdrawal would induce adaptations in these neurons, opening a window of disrupted emotional learning circuit function. Under normal physiological conditions, synaptic transmission between the basolateral amygdala (BLA) and the neighboring main island (Im) of GABAergic intercalated cells (ITCs) is strongly inhibited by endogenous opioids. Using patch-clamp electrophysiology in brain slices prepared from male rats, we reveal that opioid withdrawal abruptly reduces the ability of these peptides to inhibit neurotransmission, a direct consequence of a protein kinase A (PKA)-driven increase in the synaptic activity of peptidases. Reduced peptide control of neurotransmission in the amygdala shifts the excitatory/inhibitory balance of inputs onto accumbens-projecting amygdala cells involved in relapse. These findings provide novel insights into how peptidases control synaptic activity within the amygdala and presents restoration of endogenous peptide activity during withdrawal as a viable option to mitigate withdrawal-induced disruptions in emotional learning circuits and rescue the relapse behaviors exhibited during opioid withdrawal and beyond into abstinence.SIGNIFICANCE STATEMENT We find that opioid withdrawal dials down inhibitory neuropeptide activity in the amygdala. This disrupts both GABAergic and glutamatergic transmission through amygdala circuits, including reward-related outputs to the nucleus accumbens. This likely disrupts peptide-dependent emotional learning processes in the amygdala during withdrawal and may direct behavior toward compulsive drug use.

Nucleus accumbens in the pathogenesis of major depressive disorder: A brief review

Major depressive disorder (MDD) is the most prevalent mental disorder characterized by anhedonia, loss of motivation, avolition, behavioral despair and cognitive abnormalities. Despite substantial advancements in the pathophysiology of MDD in recent years, the pathogenesis of this disorder is not fully understood. Meanwhile,the treatment of MDD with currently available antidepressants is inadequate, highlighting the urgent need for clarifying the pathophysiology of MDD and developing novel therapeutics. Extensive studies have demonstrated the involvement of nuclei such as the prefrontal cortex (PFC), hippocampus (HIP), nucleus accumbens (NAc), hypothalamus, etc., in MDD. NAc,a region critical for reward and motivation,dysregulation of its activity seems to be a hallmark of this mood disorder. In this paper, we present a review of NAc related circuits, cellular and molecular mechanisms underlying MDD and share an analysis of the gaps in current research and possible future research directions.

Modulation of methamphetamine memory reconsolidation by neural projection from basolateral amygdala to nucleus accumbens

Drug-associated conditioned cues promote subjects to recall drug reward memory, resulting in drug-seeking and reinstatement. A consolidated memory becomes unstable after recall, such that the amnestic agent can disrupt the memory during the reconsolidation stage, which implicates a potential therapeutic strategy for weakening maladaptive memories. The basolateral amygdala (BLA) involves the association of conditioned cues with reward and aversive valences and projects the information to the nucleus accumbens (NAc) that mediates reward-seeking. However, whether the BLA-NAc projection plays a role in drug-associated memory reactivation and reconsolidation is unknown. We used methamphetamine (MeAM) conditioned place preference (CPP) to investigate the role of BLA-NAc neural projection in the memory reconsolidation. Two weeks before CPP training, we infused adeno-associated virus (AAV) carrying the designer receptor exclusively activated by designer drugs (DREADD) or control constructs. We infused clozapine-N-oxide (CNO) after the recall test to manipulate the neural activity of BLA-NAc projections in mice. We found that after recall, DREADD-mediated inhibition of BLA neurons projecting to the NAc core blunted consolidated MeAM-associated memory. Inhibition of BLA glutamatergic nerve terminals in the NAc core 1 h after recall disrupted consolidated MeAM-associated memory. However, inhibiting this pathway after the time window of reconsolidation failed to affect memory. Furthermore, under the condition without memory retrieval, DREADD-mediated activation of BLA-NAc core projection was required for amnesic agents to disrupt consolidated MeAM-associated memory. Our findings provide evidence that the BLA-NAc pathway activity is involved in the post-retrieval processing of MeAM-associated memory in CPP.

A quadruple dissociation of reward-related behaviour in mice across excitatory inputs to the nucleus accumbens shell

The nucleus accumbens shell (NAcSh) is critically important for reward valuations, yet it remains unclear how valuation information is integrated in this region to drive behaviour during reinforcement learning. Using an optogenetic spatial self-stimulation task in mice, here we show that contingent activation of different excitatory inputs to the NAcSh change expression of different reward-related behaviours. Our data indicate that medial prefrontal inputs support place preference via repeated actions, ventral hippocampal inputs consistently promote place preferences, basolateral amygdala inputs produce modest place preferences but as a byproduct of increased sensitivity to time investments, and paraventricular inputs reduce place preferences yet do not produce full avoidance behaviour. These findings suggest that each excitatory input provides distinct information to the NAcSh, and we propose that this reflects the reinforcement of different credit assignment functions. Our finding of a quadruple dissociation of NAcSh input-specific behaviours provides insights into how types of information carried by distinct inputs to the NAcSh could be integrated to help drive reinforcement learning and situationally appropriate behavioural responses.

The role of the bed nucleus of the stria terminalis in the motivational control of instrumental action

We review recent studies assessing the role of the bed nucleus of the stria terminalis (BNST) in the motivational control of instrumental conditioning. This evidence suggests that the BNST and central nucleus of the amygdala (CeA) form a circuit that modulates the ventral tegmental area (VTA) input to the nucleus accumbens core (NAc core) to control the influence of Pavlovian cues on instrumental performance. In support of these claims, we found that activity in the oval region of BNST was increased by instrumental conditioning, as indexed by phosphorylated ERK activity (Experiment 1), but that this increase was not due to exposure to the instrumental contingency or to the instrumental outcome per se (Experiment 2). Instead, BNST activity was most significantly incremented in a test conducted when the instrumental outcome was anticipated but not delivered, suggesting a role for BNST in the motivational effects of anticipated outcomes on instrumental performance. To test this claim, we examined the effect of NMDA-induced cell body lesions of the BNST on general Pavlovian-to-instrumental transfer (Experiment 3). These lesions had no effect on instrumental performance or on conditioned responding during Pavlovian conditioning to either an excitory conditioned stimulus (CS) or a neutral CS (CS0) but significantly attenuated the excitatory effect of the Pavlovian CS on instrumental performance. These data are consistent with the claim that the BNST mediates the general excitatory influence of Pavlovian cues on instrumental performance and suggest BNST activity may be central to CeA-BNST modulation of a VTA-NAc core circuit in incentive motivation.

GABAB receptors in prelimbic cortex and basolateral amygdala differentially influence intertemporal decision making and decline with age

The ability to decide adaptively between immediate vs. delayed gratification (intertemporal choice) is critical for well-being and is associated with a range of factors that influence quality of life. In contrast to young adults, many older adults show enhanced preference for delayed gratification; however, the neural mechanisms underlying this age difference in intertemporal choice are largely un-studied. Changes in signaling through GABA_B receptors (GABA_BRs) mediate several age-associated differences in cognitive processes linked to intertemporal choice. The current study used a rat model to determine how GABA_BRs in two brain regions known to regulate intertemporal choice (prelimbic cortex; PrL and basolateral amygdala; BLA) contribute to age differences in this form of decision making in male rats. As in humans, aged rats showed enhanced preference for large, delayed over small, immediate rewards during performance in an intertemporal choice task in operant test chambers. Activation of PrL GABA_BRs via microinfusion of the agonist baclofen increased choice of large, delayed rewards in young adult rats but did not influence choice in aged rats. Conversely, infusion of baclofen into the BLA strongly reduced choice of large, delayed rewards in both young adult and aged rats. Aged rats further showed a significant reduction in expression of GABA_BR1 subunit isoforms in the prefrontal cortex, a discovery that is consonant with the null effect of intra-PrL baclofen on intertemporal choice in aged rats. In contrast, expression of GABA_BR subunits was generally conserved with age in the BLA. Jointly, these findings elucidate a role for GABA_BRs in intertemporal choice and identify fundamental features of brain maturation and aging that mediate an improved ability to delay gratification.

Volume and Connectivity Differences in Brain Networks Associated with Cognitive Constructs of Binge Eating

Bulimia nervosa (BN) and binge eating disorder (BED) are characterized by episodes of eating large amounts of food while experiencing a loss of control. Recent studies suggest that the underlying causes of BN/BED consist of a complex system of environmental cues, atypical processing of food stimuli, altered behavioral responding, and structural/functional brain differences compared with healthy controls (HC). In this narrative review, we provide an integrative account of the brain networks associated with the three cognitive constructs most integral to BN and BED, namely increased reward sensitivity, decreased cognitive control, and altered negative affect and stress responding. We show altered activity in BED/BN within several brain networks, specifically in the striatum, insula, prefrontal cortex (PFC) and orbitofrontal cortex (OFC), and cingulate gyrus. Numerous key nodes in these networks also differ in volume and connectivity compared with HC. We provide suggestions for how this integration may guide future research into these brain networks and cognitive constructs.

Cocaine-induced projection-specific and cell type-specific adaptations in the nucleus accumbens

Cocaine craving, seeking, and relapse are mediated, in part, by cocaine-induced adaptive changes in the brain reward circuits. The nucleus accumbens (NAc) integrates and prioritizes different emotional and motivational inputs to the reward system by processing convergent glutamatergic projections from the medial prefrontal cortex, basolateral amygdala, ventral hippocampus, and other limbic and paralimbic brain regions. Medium spiny neurons (MSNs) are the principal projection neurons in the NAc, which can be divided into two major subpopulations, namely dopamine receptor D1- versus D2-expressing MSNs, with complementing roles in reward-associated behaviors. After cocaine experience, NAc MSNs exhibit complex and differential adaptations dependent on cocaine regimen, withdrawal time, cell type, location (NAc core versus shell), and related input and output projections, or any combination of these factors. Detailed characterization of these cellular adaptations has been greatly facilitated by the recent development of optogenetic/chemogenetic techniques combined with transgenic tools. In this review, we discuss such cell type- and projection-specific adaptations induced by cocaine experience. Specifically, (1) D1 and D2 NAc MSNs frequently exhibit differential adaptations in spinogenesis, glutamatergic receptor trafficking, and intrinsic membrane excitability, (2) cocaine experience differentially changes the synaptic transmission at different afferent projections onto NAc MSNs, (3) cocaine-induced NAc adaptations exhibit output specificity, e.g., being different at NAc-ventral pallidum versus NAc-ventral tegmental area synapses, and (4) the input, output, subregion, and D1/D2 cell type may together determine cocaine-induced circuit plasticity in the NAc. In light of the projection- and cell-type specificity, we also briefly discuss ensemble and circuit mechanisms contributing to cocaine craving and relapse after drug withdrawal.

Metabotropic group II glutamate receptors in the basolateral amygdala mediate cue-triggered increases in incentive motivation

RATIONALE

Reward-associated cues can trigger incentive motivation for reward and invigorate reward-seeking behaviour via Pavlovian-to-instrumental transfer (PIT). Glutamate signaling within the basolateral amygdala (BLA) modulates cue-triggered increases in incentive motivation. However, the role of BLA metabotropic group II glutamate (mGlu_2/3) receptors is largely unknown.

OBJECTIVES

In Experiment 1, we characterized cue-triggered increases in incentive motivation for water reward using the PIT paradigm. In Experiment 2, we assessed the influence of intra-BLA microinjections of the mGlu_2/3 receptor agonist LY379268 on this effect.

METHODS

Water-restricted male Sprague-Dawley rats learned to press a lever for water. Separately, they learned to associate one of two auditory cues with free water. On test days, rats could lever press under extinction conditions (no water), with intermittent, non-contingent CS+ and CS- presentations. In Experiment 1, rats were tested under baseline conditions. In Experiment 2, rats received intra-BLA microinjections of LY379268 (0, 3 and 6 [Formula: see text]g/hemisphere) before testing.

RESULTS

Across experiments, CS+, but not CS-, presentations increased water-associated lever pressing during testing, even though responding was reinforced neither by water nor the CS+. Intra-BLA LY379268 abolished both CS+ potentiated pressing on the water-associated lever and CS+ evoked conditioned approach to the site of water delivery. LY379268 did not influence locomotion or instrumental and Pavlovian response rates during intervals between CS presentations or during the CS-, indicating no motor effects.

CONCLUSIONS

mGlu_2/3 receptor activity in the BLA mediates cue-triggered potentiation of incentive motivation for reward, suppressing both cue-induced increases in instrumental pursuit of the reward and anticipatory approach behaviour.

Choose your path: Divergent basolateral amygdala efferents differentially mediate incentive motivation, flexibility and decision-making

To survive in a complex environment, individuals form associations between environmental stimuli and rewards to organize and optimize reward seeking behaviors. The basolateral amygdala (BLA) uses these learned associations to inform decision-making processes. In this review, we describe functional projections between BLA and its cortical and striatal targets that promote learning and motivational processes central to decision-making. Specifically, we compare and contrast divergent projections from the BLA to the orbitofrontal (OFC) and to the nucleus accumbens (NAc) and examine the roles of these pathways in associative learning, value-guided decision-making, choice behaviors, as well as cue and context-driven drug seeking. Finally, we consider how these projections are involved in disorders of motivation, with a focus on Substance Use Disorder.

Defining an orbitofrontal compass: Functional and anatomical heterogeneity across anterior-posterior and medial-lateral axes

The orbitofrontal cortex (OFC) plays a critical role in the flexible control of behaviors and has been the focus of increasing research interest. However, there have been a number of controversies around the exact theoretical role of the OFC. One potential source of these issues is the comparison of evidence from different studies, particularly across species, which focus on different specific sub-regions within the OFC. Furthermore, there is emerging evidence that there may be functional diversity across the OFC which may account for these theoretical differences. Therefore, in this review we consider evidence supporting functional heterogeneity within the OFC and how it relates to underlying anatomical heterogeneity. We highlight the importance of anatomical and functional distinctions within the traditionally defined OFC subregions across the medial-lateral axis, which are often not differentiated for practical and historical reasons. We then consider emerging evidence of even finer-grained distinctions within these defined subregions along the anterior-posterior axis. These fine-grained anatomical considerations reveal a pattern of dissociable, but often complementary functions within the OFC. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

How predictive learning influences choice: Evidence for a GPCR-based memory process necessary for Pavlovian-instrumental transfer

Predictive learning endows stimuli with the capacity to signal both the sensory-specific and general motivational properties of their associated rewards or outcomes. These two signals can be distinguished behaviorally by their influence on the selection and performance of instrumental actions, respectively. This review focuses on how sensory-specific predictive learning guides choice between actions that earn otherwise equally desirable outcomes. We describe evidence that outcome-specific predictive learning is encoded in the basolateral amygdala and drives the accumulation of delta-opioid receptors on the surface of cholinergic interneurons located in the nucleus accumbens shell. This accumulation constitutes a novel form of cellular memory, not for outcome-specific predictive learning per se but for the selection of, and choice between, future instrumental actions. We describe recent evidence regarding the cascade of events necessary for the formation and expression of this cellular memory and point to open questions for future research into this process. Beyond these mechanistic considerations, the discovery of this new form of memory is consistent with recent evidence suggesting that intracellular rather than synaptic changes can mediate learning-related plasticity to modify brain circuitry to prepare for future significant events.

Reinforcement and Compensatory Mechanisms in Attention-Deficit Hyperactivity Disorder: A Systematic Review of Case-Control Studies

Attention-deficit hyperactivity disorder (ADHD) is a neuropsychological disorder that causes inattentiveness, hyperactivity, and impulsiveness in patients. Ventral striatal hypo-responsiveness, orbitofrontal cortex, and dopaminergic status in the brain are related to the pathogenesis of ADHD. Reinforcement tasks by monetary incentive delay (MID) was shown to produce more responsiveness in patients. In this study, we reviewed how reinforcement interventions and compensatory mechanisms affect the behavior of ADHD patients. This systematic review was undertaken as per the Preferred Reporting Items for Systematic Review and Meta-Analysis guidelines, and PubMed database was used for literature search. The quality appraisal was completed using the Newcastle-Ottawa scale, and nine case-control studies were included in this systematic review. A total of 976 participants were included, with 493 cases and 330 controls. The studies included discuss reinforcement, attention networks, and compensatory mechanisms. Our review concludes that reinforcement improves responsiveness to gain and loss of rewards in ADHD patients. Reward processing is selectively associated with the salience network. While ADHD, predominantly the inattentive type, is insensitive to stimuli, ADHD combined type and controls showed similar responsiveness. The right visual cortex may also be related to compensatory mechanisms in ADHD. As we only included case-control studies from the last eight years, in the English language, we might have missed some relevant studies related to this research. Because the included studies have a relatively small sample size, we recommend future studies to explore larger cohorts of patients to improve the reliability of findings pertinent to this field.

Instrumental and Pavlovian Mechanisms in Alcohol Use Disorder

Purpose of Review

Current theories of alcohol use disorders (AUD) highlight the importance of Pavlovian and instrumental learning processes mainly based on preclinical animal studies. Here, we summarize available evidence for alterations of those processes in human participants with AUD with a focus on habitual versus goal-directed instrumental learning, Pavlovian conditioning, and Pavlovian-to-instrumental transfer (PIT) paradigms.

Recent Findings

The balance between habitual and goal-directed control in AUD participants has been studied using outcome devaluation or sequential decision-making procedures, which have found some evidence of reduced goal-directed/model-based control, but little evidence for stronger habitual responding. The employed Pavlovian learning and PIT paradigms have shown considerable differences regarding experimental procedures, e.g., alcohol-related or conventional reinforcers or stimuli.

Summary

While studies of basic learning processes in human participants with AUD support a role of Pavlovian and instrumental learning mechanisms in the development and maintenance of drug addiction, current studies are characterized by large variability regarding methodology, sample characteristics, and results, and translation from animal paradigms to human research remains challenging. Longitudinal approaches with reliable and ecologically valid paradigms of Pavlovian and instrumental processes, including alcohol-related cues and outcomes, are warranted and should be combined with state-of-the-art imaging techniques, computational approaches, and ecological momentary assessment methods.

Basolateral Amygdala to Nucleus Accumbens Communication Differentially Mediates Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Rats rely on communication between the basolateral amygdala (BLA) and nucleus accumbens (NAc) to express lever directed approach in a Pavlovian lever autoshaping (PLA) task that distinguishes sign- and goal-tracking rats. During PLA, sign-tracking rats preferentially approach an insertable lever cue, while goal-tracking rats approach a foodcup where rewards are delivered. While sign-tracking rats inflexibly respond to cues even after the associated reward is devalued, goal-tracking rats flexibly reduce responding to cues during outcome devaluation. Here, we sought to determine whether BLA-NAc communication, which is necessary for sign, but not goal-tracking, drives a rigid appetitive approach of sign-tracking rats that are insensitive to manipulations of outcome value. Using a contralateral chemogenetic inactivation design, we injected contralateral BLA and NAc core with inhibitory DREADD (hm4Di-mCherry) or control (mCherry) constructs. To determine sign- and goal-tracking groups, we trained rats in five PLA sessions in which brief lever insertion predicts food pellet delivery. We sated rats on training pellets (devalued condition) or chow (valued condition) before systemic clozapine injections (0.1 mg/kg) to inactivate BLA and contralateral NAc during two outcome devaluation probe tests, in which we measured lever and foodcup approach. Contralateral BLA-NAc chemogenetic inactivation promoted a flexible lever approach in sign-tracking rats but disrupted the flexible foodcup approach in goal-tracking rats. Consistent with a prior BLA-NAc disconnection lesion study, we find contralateral chemogenetic inactivation of BLA and NAc core reduces lever, but not the foodcup approach in PLA. Together these findings suggest rigid appetitive associative encoding in BLA-NAc of sign-tracking rats hinders the expression of flexible behavior when outcome value changes.

Motivational factors underlying aversive Pavlovian-instrumental transfer

While interest in active avoidance has recently been resurgent, many concerns relating to the nature of this form of learning remain unresolved. By separating stimulus and response acquisition, aversive Pavlovian-instrumental transfer can be used to measure the effect of avoidance learning on threat processing with more control than typical avoidance procedures. However, the motivational substrates that contribute to the aversive transfer effect have not been thoroughly examined. In three studies using rodents, the impact of a variety of aversive signals on shock-avoidance responding (i.e., two-way shuttling) was evaluated. Fox urine, as well as a tone paired with the delivery of the predator odor were insufficient modulatory stimuli for the avoidance response. Similarly, a signal for the absence of food did not generate appropriate aversive motivation to enhance shuttling. Only conditioned Pavlovian stimuli that had been paired with unconditioned threats were capable of augmenting shock-avoidance responding. This was true whether the signaled outcome was the same (e.g., shock) or different (e.g., klaxon) from the avoidance outcome (i.e., shock). These findings help to characterize the aversive transfer effect and provide a more thorough analysis of its generalization to warning signals for different kinds of threats. This feature of aversive motivation has not been demonstrated using conventional avoidance procedures and could be potentially useful for applying avoidance in treatment settings.

Organic cation transporter 3 and the dopamine transporter differentially regulate catecholamine uptake in the basolateral amygdala and nucleus accumbens

Regional alterations in kinetics of catecholamine uptake are due in part to variations in clearance mechanisms. The rate of clearance is a critical determinant of the strength of catecholamine signaling. Catecholamine transmission in the nucleus accumbens core (NAcc) and basolateral amygdala (BLA) is of particular interest due to involvement of these regions in cognition and motivation. Previous work has shown that catecholamine clearance in the NAcc is largely mediated by the dopamine transporter (DAT), but clearance in the BLA is less DAT-dependent. A growing body of literature suggests that organic cation transporter 3 (OCT3) also contributes to catecholamine clearance in both regions. Consistent with different clearance mechanisms between regions, catecholamine clearance is more rapid in the NAcc than in the BLA, though mechanisms underlying this have not been resolved. We compared the expression of DAT and OCT3 and their contributions to catecholamine clearance in the NAcc and BLA. We found DAT protein levels were ~ 4-fold higher in the NAcc than in the BLA, while OCT3 protein expression was similar between the two regions. Immunofluorescent labeling of the two transporters in brain sections confirmed these findings. Ex vivo voltammetry demonstrated that the magnitude of catecholamine release was greater, and the clearance rate was faster in the NAcc than in the BLA. Additionally, catecholamine clearance in the BLA was more sensitive to the OCT3 inhibitor corticosterone, while clearance in the NAcc was more cocaine sensitive. These distinctions in catecholamine clearance may underlie differential effects of catecholamines on behavioral outputs mediated by these regions.

Cortical and Thalamic Interaction with Amygdala-to-Accumbens Synapses

The nucleus accumbens shell (NAcSh) regulates emotional and motivational responses, a function mediated, in part, by integrating and prioritizing extensive glutamatergic projections from limbic and paralimbic brain regions. Each of these inputs is thought to encode unique aspects of emotional and motivational arousal. The projections do not operate alone, but rather are often activated simultaneously during motivated behaviors, during which they can interact and coordinate in shaping behavioral output. To understand the anatomic and physiological bases underlying these interprojection interactions, the current study in mice of both sexes focused on how the basolateral amygdala projection (BLAp) to the NAcSh regulates, and is regulated by, projections from the medial prefrontal cortex (mPFCp) and paraventricular nucleus of the thalamus (PVTp). Using a dual-color SynaptoTag technique combined with a backfilling spine imaging strategy, we found that all three afferent projections primarily targeted the secondary dendrites of NAcSh medium spiny neurons, forming putative synapses. We detected a low percentage of BLAp contacts closely adjacent to mPFCp or PVTp presumed synapses, and, on some rare occasions, the BLAp formed heterosynaptic interactions with mPFCp or PVTp profiles or appeared to contact the same spines. Using dual-rhodopsin optogenetics, we detected signs of dendritic summation of BLAp with PVTp and mPFCp inputs. Furthermore, high-frequency activation of BLAp synchronous with the PVTp or mPFCp resulted in a transient enhancement of the PVTp, but not mPFCp, transmission. These results provide anatomic and functional indices that the BLAp interacts with the mPFCp and PVTp for informational processing within the NAcSh.SIGNIFICANCE STATEMENT The nucleus accumbens regulates emotional and motivational responses by integrating extensive glutamatergic projections, but the anatomic and physiological bases on which these projections integrate and interact remain underexplored. Here, we used dual-color synaptic markers combined with backfilling of nucleus accumbens medium spiny neurons to reveal some unique anatomic alignments of presumed synapses from the basolateral amygdala, medial prefrontal cortex, and paraventricular nucleus of thalamus. We also used dual-rhodopsin optogenetics in brain slices, which reveal a nonlinear interaction between some, but not all, projections. These results provide compelling anatomic and physiological mechanisms through which different glutamatergic projections to the nucleus accumbens, and possibly different aspects of emotional and motivational arousal, interact with each other for final behavioral output.

Effects of hM4Di activation in CamKII basolateral amygdala neurons and CNO treatment on sensory-specific vs. general PIT: refining PIT circuits and considerations for using CNO

BACKGROUND

Pavlovian stimuli can influence instrumental behaviors via phenomena such as Pavlovian-to-instrumental transfer (PIT). PIT arises via dissociable processes as sensory-specific PIT (SS-PIT) and general PIT. The basolateral amygdala (BLA) mediates SS-PIT, but not general PIT. However, the specific BLA neuronal populations involved are unknown.

AIMS

To determine the contribution of glutamatergic BLA neurons to the expression of SS-PIT and to the recall of sensory-specific properties of stimulus-outcome associations.

METHODS

BLA neurons were transduced with virus containing either GFP or hM4Di, driven by the CamKII promoter. Rats were then tested for SS and general PIT and subsequently for expression of Pavlovian outcome devaluation effects and conditioned taste aversion following injections of vehicle or clozapine-N-oxide (CNO, the hM4Di agonist).

RESULTS

CNO selectively blocked SS-PIT in the hM4Di-expressing group, but not controls, without altering expression of Pavlovian outcome devaluation or sensory-specific taste aversion in either group. Unexpectedly, CNO disrupted general PIT in both groups.

CONCLUSIONS

CamKII BLA neurons mediate the expression of SS-PIT by enabling Pavlovian stimuli to trigger recall of the correct action-outcome associations rather than by mediating recall of the sensory-specific properties of the stimulus-outcome association. Separately, our data demonstrate that CNO alone is sufficient to disrupt affective, but not sensory-specific processes, an effect that was not due to generalized motor disruption. This non-specific effect on general PIT may be related to CNO-induced shifts in internal state. Together, these data identify BLA CamKII neurons as critical for the expression of SS-PIT and reveal important considerations for using CNO to study general affective motivation.

The Meaning of Behavior: Discriminating Reflex and Volition in the Brain

The ability to establish behaviorally what psychological capacity an animal is deploying-to discern accurately what an animal is doing-is key to functional analyses of the brain. Our current understanding of these capacities suggests, however, that this task is complex; there is evidence that multiple capacities are engaged simultaneously and contribute independently to the control of behavior. As such, establishing the contribution of a cell, circuit, or neural system to any one function requires careful dissection of that role from its influence on other functions and, therefore, the careful selection and design of behavioral tasks fit for that purpose. Here I describe recent research that has sought to utilize behavioral tools to investigate the neural bases of instrumental conditioning, particularly the circuits and systems supporting the capacity for goal-directed action, as opposed to conditioned reflexes and habits, and how these sources of action control interact to generate adaptive behavior.

Off-Target Influences of Arch-Mediated Axon Terminal Inhibition on Network Activity and Behavior

Archaerhodopsin (ArchT)-mediated photoinhibition of axon terminals is commonly used to test the involvement of specific long-range neural projections in behavior. Although sustained activation of this opsin in axon terminals has the unintended consequence of enhancing spontaneous vesicle release, it is unclear whether this desynchronized signaling is consequential for ArchT’s behavioral effects. Here, we compare axon terminal and cell body photoinhibition of nucleus accumbens (NAc) afferents to test the utility of these approaches for uncovering pathway-specific contributions of neural circuits to behavior. First, in brain slice recordings we confirmed that ArchT photoinhibition of glutamatergic axons reduces evoked synaptic currents and increases spontaneous transmitter release. A further consequence was increased interneuron activity, which served to broadly suppress glutamate input via presynaptic GABA_B receptors. In vivo, axon terminal photoinhibition increased feeding and reward-seeking behavior irrespective of the afferent pathway targeted. These behavioral effects are comparable to those obtained with broad inhibition of NAc neurons. In contrast, cell body inhibition of excitatory NAc afferents revealed a pathway-specific contribution of thalamic input to feeding behavior and amygdala input to reward-seeking under extinction conditions. These findings underscore the off-target behavioral consequences of ArchT-mediated axon terminal inhibition while highlighting cell body inhibition as a valuable alternative for pathway-specific optogenetic silencing.

BOLD differences normally attributed to inhibitory control predict symptoms, not task-directed inhibitory control in ADHD

Background

Altered brain activity that has been observed in attention deficit hyperactivity disorder (ADHD) while performing cognitive control tasks like the stop signal task (SST) has generally been interpreted as reflecting either weak (under-active) or compensatory (over-active) versions of the same functions as in healthy controls. If so, then regional activities that correlate with the efficiency of inhibitory control (i.e. stop signal reaction time, SSRT) in healthy subjects should also correlate with SSRT in ADHD. Here we test the alternate hypothesis that BOLD (blood-oxygen-level-dependent) differences might instead reflect the redirection of neural processing resources normally used for task-directed inhibitory control, towards actively managing symptomatic behaviour. If so, then activities that correlate with SSRT in TD should instead correlate with inattentive and hyperactive symptoms in ADHD.

Methods

We used fMRI (functional magnetic resonance imaging) in 14 typically developing (TD) and 14 ADHD adolescents performing the SST, and in a replication sample of 14 healthy adults. First, we identified significant group BOLD differences during all phases of activity in the SST (i.e. warning, response, reactive inhibition, error detection and post-error slowing). Next, we correlated these phases of activity with SSRT in TD and with SSRT, inattentive and hyperactive symptom scores in ADHD. We then identified whole brain significant correlations in regions of significant group difference in activity.

Results

Only three regions of significant group difference were correlated with SSRT in TD and replication groups (left and right inferior frontal gyri (IFG) during error detection and hypothalamus during post-error slowing). Consistent with regions of altered activity managing symptomatic behaviour instead of task-directed behaviour, left IFG correlated with greater inattentive score, right IFG correlated with lower hyperactive score and hypothalamus correlated with greater inattentive score and oppositely correlated with SSRT compared to TD.

Conclusions

Stimuli that elicit task-directed integration of neural processing in healthy subjects instead appear to be directing integrated function towards managing symptomatic behaviour in ADHD. The ability of the current approach to determine whether altered neural activities reflect comparable functions in ADHD and control groups has broad implications for the development and monitoring of therapeutic interventions.

Effects of Limited and Extended Pavlovian Training on Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Individual differences in Pavlovian approach predict differences in devaluation sensitivity. Recent studies indicate goal-tracking (GT) rats are sensitive to outcome devaluation while sign-tracking (ST) rats are not. With extended training in Pavlovian lever autoshaping (PLA), GT rats display more lever-directed behavior, typical of ST rats, suggesting they may become insensitive to devaluation with more Pavlovian training experience. Here, we use a within-subject satiety-induced outcome devaluation procedure to test devaluation sensitivity after limited and extended PLA training in GT and ST rats. We trained rats in PLA to determine GT and ST groups. Then, we sated rats on either the training pellets (devalued condition) or homecage chow (valued condition) prior to brief non-reinforced test sessions after limited (sessions 5/6) and extended (sessions 17/18) PLA training. GT rats decreased conditioned responding under devalued relative to valued conditions after both limited and extended training, demonstrating they are sensitive to satiety devaluation regardless of the amount of PLA training. While ST rats were insensitive to satiety devaluation after limited training, their lever directed behavior became devaluation sensitive after extended training. To determine whether sign-tracking rats also displayed sensitivity to illness-induced outcome devaluation after extended training, we trained a separate cohort of rats in extended PLA and devalued the outcome with lithium chloride injections after pellet consumption in the homecage. ST rats failed to decrease conditioned responding after illness-induced outcome devaluation, while Non-ST rats (GT and intermediates) were sensitive to illness-induced outcome devaluation after extended training. Together, our results confirm devaluation sensitivity is stable in GT rats across training and devaluation approaches. Extended training unmasks devaluation sensitivity in ST rats after satiety, but not illness-induced devaluation, suggesting ST rats respond appropriately by decreasing responding to cues during state-dependent but not inference-based devaluation. The differences in behavioral flexibility across tracking groups and devaluation paradigms have translational relevance for the understanding state- vs. inference-based reward devaluation as it pertains to drug addiction vulnerability.

Cell-Type- and Endocannabinoid-Specific Synapse Connectivity in the Adult Nucleus Accumbens Core

The nucleus accumbens (NAc) is a mesocorticolimbic structure that integrates cognitive, emotional and motor functions. Although its role in psychiatric disorders is widely acknowledged, the understanding of its circuitry is not complete. Here, we combined optogenetic and whole-cell recordings to draw a functional portrait of excitatory disambiguated synapses onto D1 and D2 medium spiny neurons (MSNs) in the adult male mouse NAc core. Comparing synaptic properties of ventral hippocampus (vHipp), basolateral amygdala (BLA) and prefrontal cortex (PFC) inputs revealed a hierarchy of synaptic inputs that depends on the identity of the postsynaptic target MSN. Thus, the BLA is the dominant excitatory pathway onto D1 MSNs (BLA > PFC = vHipp) while PFC inputs dominate D2 MSNs (PFC > vHipp > BLA). We also tested the hypothesis that endocannabinoids endow excitatory circuits with pathway- and cell-specific plasticity. Thus, whereas CB1 receptors (CB1R) uniformly depress excitatory pathways regardless of MSNs identity, TRPV1 receptors (TRPV1R) bidirectionally control inputs onto the NAc core in a pathway-specific manner. Finally, we show that the interplay of TRPV1R/CB1R shapes plasticity at BLA-NAc synapses. Together these data shed new light on synapse and circuit specificity in the adult NAc core and illustrate how endocannabinoids contribute to pathway-specific synaptic plasticity.SIGNIFICANCE STATEMENT We examined the impact of connections from the ventral hippocampus (vHipp,) basolateral amygdala (BLA) and prefrontal cortex (PFC) onto identified medium spiny neurons (MSNs) in the adult accumbens core. We found BLA inputs were strongest at D1 MSNs while PFC inputs dominate D2 MSNs. Pathway- and cell-specific circuit control was also facilitated by endocannabinoids that endow bidirectional synaptic plasticity at identified BLA-NAc synapses. These data provide mechanistic insights on synapse and circuit specificity in the adult NAc core.

Reward Devaluation Attenuates Cue-Evoked Sucrose Seeking and Is Associated with the Elimination of Excitability Differences between Ensemble and Non-ensemble Neurons in the Nucleus Accumbens

Animals must learn relationships between foods and the environmental cues that predict their availability for survival. Such cue–food associations are encoded in sparse sets of neurons or “neuronal ensembles” in the nucleus accumbens (NAc). For these ensemble-encoded, cue-controlled appetitive responses to remain adaptive, they must allow for their dynamic updating depending on acute changes in internal states such as physiological hunger or the perceived desirability of food. However, how these neuronal ensembles are recruited and physiologically modified following the update of such learned associations is unclear. To investigate this, we examined the effects of devaluation on ensemble plasticity at the levels of recruitment, intrinsic excitability, and synaptic physiology in sucrose-conditioned Fos-GFP mice that express green fluorescent protein (GFP) in recently activated neurons. Neuronal ensemble activation patterns and their physiology were examined using immunohistochemistry and slice electrophysiology, respectively. Reward-specific devaluation following 4 d of ad libitum sucrose consumption, but not general caloric devaluation, attenuated cue-evoked sucrose seeking. This suggests that changes in the hedonic and/or incentive value of sucrose, and not caloric need, drove this behavior. Moreover, devaluation attenuated the size of the neuronal ensemble recruited by the cue in the NAc shell. Finally, it eliminated the relative enhanced excitability of ensemble (GFP⁺) neurons against non-ensemble (GFP⁻) neurons observed under non-devalued conditions, and did not induce any ensemble-specific changes in excitatory synaptic physiology. Our findings provide new insights into neuronal ensemble mechanisms that underlie the changes in the incentive and/or hedonic impact of cues that support adaptive food seeking.

Context-Dependent and Context-Independent Effects of D1 Receptor Antagonism in the Basolateral and Central Amygdala during Cocaine Self-Administration

One way that drugs of abuse perturb the dopamine system is by triggering large amounts of extracellular dopamine to efflux into limbic regions. The basolateral (BLA) and central (CeA) nuclei of the amygdala have been shown to play distinct roles in value representation of primary and conditioned reward. However, the precise role of dopaminergic receptors in the BLA and the CeA during reward-related behaviors remains to be determined. Here we investigate the effects of dopamine D1 receptor blockade in the BLA and the CeA during asymptotic performance of cocaine self-administration and in a novel application of contextual renewal under continued access conditions. After more than three weeks of chained seek-take self-administration of cocaine, male Long Evans rats were given a bilateral intra-BLA or intra-CeA infusion of the D1 antagonist SCH-23390 (2 µg/0.3 µl) for multiple days. Intra-BLA D1 receptor blockade before, but not after the self-administration session, gradually suppressed drug seeking and taking responses and persisted with a change in context with continued D1 blockade. In contrast, intra-CeA D1 receptor blockade caused a rapid reduction in self-administration that showed renewal with a change in context with continued D1 blockade. Further, conditioned place aversion developed with intra-BLA but not intra-CeA infusions. Collectively, these results demonstrate that dopamine D1 receptors in the BLA and CeA both contribute to drug seeking and taking, but may do so through distinct mechanisms.

A glutamatergic insular-striatal projection regulates the reinstatement of cue-associated morphine-seeking behavior in mice

Recently, the insular cortex (IC) was identified as part of the neuronal circuit responsible for the reward expectations in cue-triggered behaviours. Moreover, there are evidences that connections between the IC and the ventral striatum, particularly with the nucleus accumbens (NAc), may mediate the retrieval and performance of actions based on incentive memory. However, the precise role of the IC-NAc connections in cue-related drug-seeking behaviour remains unclear. We used the morphine-induced conditioned place preference (CPP) paradigm to assess the formation and relapse of cue-related drug-seeking. cFos immunostaining was used to determine the activation of the brain regions. Chemogenetic and optogenetic methods were used to manipulate the activity of IC-to-NAc projection neurons. The result showed that neurons in IC and NAc core but not NAc shell were activated following cue-induced morphine-seeking behaviour. Negligible effect of inhibition of IC-to-NAc core projection (IC^{→NAc core}) on morphine CPP expression, whereas chemogenetic inactivation of this projection potently blocked the reinstatement of expressed morphine CPP. Furthermore, optogenetic inhibition of glutamatergic IC^{→NAc core} inputs significant suppressed the CPP reinstatement without significant effect on CPP expression. We demonstrated here, for the first time, that IC^{→NAc core} glutamatergic projection is required for the reinstatement of cue-associated drug seeking behaviour in mice. The present study provide insights into modulations of relapse of cue-associated drug-seeking behaviour following repeated overexposure to opioids in humans.

Optogenetic dissection of basolateral amygdala contributions to intertemporal choice in young and aged rats

Across species, aging is associated with an increased ability to choose delayed over immediate gratification. These experiments used young and aged rats to test the role of the basolateral amygdala (BLA) in intertemporal decision making. An optogenetic approach was used to inactivate the BLA in young and aged rats at discrete time points during choices between levers that yielded a small, immediate vs. a large, delayed food reward. BLA inactivation just prior to decisions attenuated impulsive choice in both young and aged rats. In contrast, inactivation during receipt of the small, immediate reward increased impulsive choice in young rats but had no effect in aged rats. BLA inactivation during the delay or intertrial interval had no effect at either age. These data demonstrate that the BLA plays multiple, temporally distinct roles during intertemporal choice, and show that the contribution of BLA to choice behavior changes across the lifespan.

Disrupted reinforcement learning during post-error slowing in ADHD

ADHD is associated with altered dopamine regulated reinforcement learning on prediction errors. Despite evidence of categorically altered error processing in ADHD, neuroimaging advances have largely investigated models of normal reinforcement learning in greater detail. Further, although reinforcement leaning critically relies on ventral striatum exerting error magnitude related thresholding influences on substantia nigra (SN) and dorsal striatum, these thresholding influences have never been identified with neuroimaging. To identify such thresholding influences, we propose that error magnitude related activities must first be separated from opposite activities in overlapping neural regions during error detection. Here we separate error detection from magnitude related adjustment (post-error slowing) during inhibition errors in the stop signal task in typically developing (TD) and ADHD adolescents using fMRI. In TD, we predicted that: 1) deactivation of dorsal striatum on error detection interrupts ongoing processing, and should be proportional to right frontoparietal response phase activity that has been observed in the SST; 2) deactivation of ventral striatum on post-error slowing exerts thresholding influences on, and should be proportional to activity in dorsal striatum. In ADHD, we predicted that ventral striatum would instead correlate with heightened amygdala responses to errors. We found deactivation of dorsal striatum on error detection correlated with response-phase activity in both groups. In TD, post-error slowing deactivation of ventral striatum correlated with activation of dorsal striatum. In ADHD, ventral striatum correlated with heightened amygdala activity. Further, heightened activities in locus coeruleus (norepinephrine), raphe nucleus (serotonin) and medial septal nuclei (acetylcholine), which all compete for control of DA, and are altered in ADHD, exhibited altered correlations with SN. All correlations in TD were replicated in healthy adults. Results in TD are consistent with dopamine regulated reinforcement learning on post-error slowing. In ADHD, results are consistent with heightened activities in the amygdala and non-dopaminergic neurotransmitter nuclei preventing reinforcement learning.

Common and distinct neural substrates of the money illusion in win and loss domains

People often evaluate money based on its face value and overlook its real purchasing power, a phenomenon known as the money illusion. In the present study, using functional magnetic resonance imaging (fMRI) combined with a gambling task, we examined the neural signatures of the money illusion in both win and loss domains. Behavioral results showed that self-reported satisfaction with outcomes was modulated by the face value but not the true value of money in both win and loss domains. At the neural level, activity in the posterior insula was associated with the true value of money in the win domain, but not in the loss domain. Importantly, we found that the ventral striatum, ventromedial prefrontal cortex (vmPFC) and amygdala encoded the money illusion in both domains, indicating a domain-general rather than domain-specific neural signature. Moreover, participants with a larger degree of money illusion at the behavioral level showed stronger functional connectivity between the ventral striatum and ventral anterior cingulate cortex (vACC) in the win domain, but stronger functional connectivity between the ventral striatum and amygdala in the loss domain. Our findings highlight the overlapping and distinct neural substrates underlying the money illusion in the context of wins and losses.

Behavioral and neural mechanisms underlying habitual and compulsive drug seeking

Addiction is characterized by compulsive drug use despite negative consequences. Here we review studies that indicate that compulsive drug use, and in particular punishment resistance in animal models of addiction, is related to impaired cortical control over habitual behavior. In humans and animals, instrumental behavior is supported by goal-directed and habitual systems that rely on distinct corticostriatal networks. Chronic exposure to addictive drugs or stress has been shown to bias instrumental response strategies toward habit learning, and impair prefrontal cortical (PFC) control over responding. Moreover, recent work has implicated prelimbic PFC hypofunction in the punishment resistance that has been observed in a subset of animals with an extended history of cocaine self-administration. This may be related to a broader role for prelimbic PFC in mediating adaptive responding and behavioral flexibility, including exerting goal-directed control over behavior. We hypothesize that impaired cortical control and reduced flexibility between habitual and goal-directed systems may be critically involved in the development of maladaptive, compulsive drug use.

Optogenetic Dissection of Temporal Dynamics of Amygdala-Striatal Interplay during Risk/Reward Decision Making

Decision making often requires weighing costs and benefits of different options that vary in terms of reward magnitude and uncertainty. Previous studies using pharmacological inactivations have shown that the basolateral amygdala (BLA) to nucleus accumbens (NAc) pathway promotes choice towards larger/riskier rewards. Neural activity in BLA and NAc shows distinct, phasic changes in firing prior to choice and following action outcomes, yet, how these temporally-discrete patterns of activity within BLA→NAc circuitry influence choice is unclear. We assessed how optogenetic silencing of BLA terminals in the NAc altered action selection during probabilistic decision making. Rats received intra-BLA infusions of viruses encoding the inhibitory opsin eArchT and were well trained on a probabilistic discounting task, where they chose between smaller/certain rewards and larger rewards delivered in a probabilistic manner, with the odds of obtaining the larger reward changing over a session (50–12.5%). During testing, activity of BLA→NAc inputs were suppressed with 4- to 7-s pulses of light delivered via optic fibers into the NAc during discrete task events: prior to choice or after choice outcomes. Inhibition prior to choice reduced selection of the preferred option, suggesting that during deliberation, BLA→NAc activity biases choice towards preferred rewards. Inhibition during reward omissions increased risky choice during the low-probability block, indicating that activity after non-rewarded actions serves to modify subsequent choice. In contrast, silencing during rewarded outcomes did not reliably affect choice. These data demonstrate how patterns of activity in BLA→NAc circuitry convey different types of information that guide action selection in situations involving reward uncertainty.

Dopaminergic Modulation of Goal-Directed Behavior in a Rodent Model of Attention-Deficit/Hyperactivity Disorder

Aside from its clinical symptoms of inattention, impulsivity and hyperactivity, patients with Attention/Deficit-Hyperactivity Disorder (ADHD) display reward and motivational impairments. These impairments may reflect a deficit in action control, that is, an inability to flexibly adapt behavior to changing consequences. We previously showed that spontaneously hypertensive rats (SHR), an inbred rodent model of ADHD, show impairments in goal-directed action control, and instead are predominated by habits. In this study, we examined the effects of specific dopamine receptor sub-type (D1 and D2) agonists and antagonists on goal-directed behavior in SHR and the normotensive inbred control strain Wistar-Kyoto (WKY) rats. Rats acquired an instrumental response for different-flavored food rewards. A selective-satiety outcome devaluation procedure followed by a choice test in extinction revealed outcome-insensitive habitual behavior in SHR rats. Outcome-sensitive goal-directed behavior was restored in SHR rats following injection prior to the choice test of the dopamine D2 receptor agonist Quinpirole or dopamine D1 receptor antagonist SCH23390, whereas WKY rats showed habitual responding following exposure to these drugs. This novel finding indicates that the core behavioral deficit in ADHD might not be a consequence of dopamine hypofunction, but rather is due to a misbalance between activation of dopamine D1 and D2 receptor pathways that govern action control.

Control of habitual instrumental actions by anticipation of postingestive sensations

In animal behavioral experiments, extended training causes instrumental actions that deliver ingestible substances to lose sensitivity to outcome devaluation and contingency degradation, and to gain direct sensitivity to the current motivational state. These features of habitual control have been attributed to a process that relies on stimulus-response (S-R) associations linking the context to instrumental actions and Pavlovian associations linking the same context to orally-sensed properties of substances attained there. This Pavlovian process was conceived based on the results of irrelevant incentive experiments, but it is not supported by the results of all such experiments. An alternative process is therefore proposed here. In this process, recall of the instrumental action is evoked by an S-R association, but the rate at which this action is then performed is controlled by anticipation of postingestive sensations that have frequently followed it. This anticipation relies on recall of an association linking the action directly to the postingestive sensations. This association is learned during the formation of a chunked action series that begins with the instrumental action and ends with a consummatory response. It enables a prediction of subjective value that is directly influenced by the current motivational state, but is not influenced by devaluation or non-contingent delivery of the substance that has produced the postingestive sensations.

Nucleus Accumbens Dopamine Signaling Regulates Sexual Preference for Females in Male Mice

Sexual preference for the opposite sex is a fundamental behavior underlying reproductive success, but the neural mechanisms remain unclear. Here, we examined the role of dopamine signaling in the nucleus accumbens core (NAcc) in governing chemosensory-mediated preference for females in TrpC2^-/- and wild-type male mice. TrpC2^-/- males, deficient in VNO-mediated signaling, do not display mating or olfactory preference toward females. We found that, during social interaction with females, TrpC2^-/- males do not show increased NAcc dopamine levels, observed in wild-type males. Optogenetic stimulation of VTA-NAcc dopaminergic neurons in TrpC2^-/- males during exposure to a female promoted preference response to female pheromones and elevated copulatory behavior toward females. Additionally, we found that signaling through the D1 receptor in the NAcc is necessary for the olfactory preference for female-soiled bedding. Our study establishes a critical role for the mesolimbic dopaminergic system in governing pheromone-mediated responses and mate choice in male mice.

Pavlovian Extinction and Recovery Effects in Aversive Pavlovian to Instrumental Transfer

Three studies explored the sensitivity of aversive Pavlovian to instrumental transfer (PIT) to Pavlovian extinction in rodents. Rats underwent Pavlovian conditioning prior to avoidance training. The PIT test then involved assessment of the effects of the Pavlovian conditioned stimulus (CS) on the performance of the avoidance response (AR). Conducting extinction prior to avoidance training and transfer testing, allowed spontaneous recovery and shock reinstatement of extinguished motivation, whereas conducting extinction following avoidance training and just prior to PIT testing successfully reduced transfer effects. This was also the case in a design that compared responding to an extinguished CS against a non-extinguished CS rather than comparing extinguished and non-extinguished groups to one another. While extinction treatments in many appetitive PIT studies do not successfully reduce transfer, and can sometimes enhance the effect, the current findings show that an extinction treatment temporally close to transfer testing can reduce the motivational impact of the aversive Pavlovian CS on instrumental avoidance responding.

Nucleus accumbens feedforward inhibition circuit promotes cocaine self-administration

The basolateral amygdala (BLA) sends excitatory projections to the nucleus accumbens (NAc) and regulates motivated behaviors partially by activating NAc medium spiny neurons (MSNs). Here, we characterized a feedforward inhibition circuit, through which BLA-evoked activation of NAc shell (NAcSh) MSNs was fine-tuned by GABAergic monosynaptic innervation from adjacent fast-spiking interneurons (FSIs). Specifically, BLA-to-NAcSh projections predominantly innervated NAcSh FSIs compared with MSNs and triggered action potentials in FSIs preceding BLA-mediated activation of MSNs. Due to these anatomical and temporal properties, activation of the BLA-to-NAcSh projection resulted in a rapid FSI-mediated inhibition of MSNs, timing-contingently dictating BLA-evoked activation of MSNs. Cocaine self-administration selectively and persistently up-regulated the presynaptic release probability of BLA-to-FSI synapses, entailing enhanced FSI-mediated feedforward inhibition of MSNs upon BLA activation. Experimentally enhancing the BLA-to-FSI transmission in vivo expedited the acquisition of cocaine self-administration. These results reveal a previously unidentified role of an FSI-embedded circuit in regulating NAc-based drug seeking and taking.

Ventral striatum lesions do not affect reinforcement learning with deterministic outcomes on slow time scales

A large body of work has implicated the ventral striatum (VS) in aspects of reinforcement learning (RL). However, less work has directly examined the effects of lesions in the VS, or other forms of inactivation, on 2-armed bandit RL tasks. We have recently found that lesions in the VS in macaque monkeys affect learning with stochastic schedules but have minimal effects with deterministic schedules. The reasons for this are not currently clear. Because our previous work used short intertrial intervals, one possibility is that the animals were using working memory to bridge stimulus-reward associations from 1 trial to the next. In the present study, we examined learning of 60 pairs of objects, in which the animals received only 1 trial per day with each pair. The large number of object pairs and the long interval (approximately 24 hr) between trials with a given pair minimized the chances that the animals could use working memory to bridge trials. We found that monkeys with VS lesions were unimpaired relative to controls, which suggests that animals with VS lesions can still learn to select rewarded objects even when they cannot make use of working memory. (PsycINFO Database Record

Extinction of specific stimulus-outcome (S-O) associations in Pavlovian learning with an extended CS procedure

Three experiments with male and female rats were conducted to examine the effects of Pavlovian extinction training on Pavlovian-to-instrumental transfer (PIT) in a task in which the unconditioned stimulus (US) was presented at an early time point within an extended conditioned stimulus (CS). Two instrumental responses were trained with different reinforcing outcomes (R1-O1, R2-O2) and then, independently, 2 stimuli were trained with those outcomes (S1-O1, S2-O2). One group then underwent an extinction treatment (S1-, S2-) and a second was merely exposed to the experimental contexts without any stimulus events. Finally, the effects of the 2 stimuli on instrumental responding were assessed in PIT tests. Across experiments we varied the number of Pavlovian training trials prior to extinction (8, 16, or 64 trials) and the length of time following extinction prior to test (i.e., 1 or 21 days, in a test for spontaneous recovery). We observed that outcome-specific PIT was reduced by extinction in all of our training conditions and that this extinction effect was durable, surviving a 3-week spontaneous recovery interval even though conditioned magazine approach spontaneously recovered over this interval. Although extinction reduced the magnitude of PIT, the temporal expression of PIT was mostly unaffected. We found these effects in both male and female rats, though in 1 study females were extinction-resistant. These data suggest that under the conditions studied here Pavlovian extinction may permanently weaken the ability of Pavlovian cues to retrieve a representation of their associated outcomes without impacting the temporal organization of responding. This suggests that different features of learning may be differentially sensitive to extinction. (PsycINFO Database Record

Effects of Ventral Striatum Lesions on Stimulus-Based versus Action-Based Reinforcement Learning

Learning the values of actions versus stimuli may depend on separable neural circuits. In the current study, we evaluated the performance of rhesus macaques with ventral striatum (VS) lesions on a two-arm bandit task that had randomly interleaved blocks of stimulus-based and action-based reinforcement learning (RL). Compared with controls, monkeys with VS lesions had deficits in learning to select rewarding images but not rewarding actions. We used a RL model to quantify learning and choice consistency and found that, in stimulus-based RL, the VS lesion monkeys were more influenced by negative feedback and had lower choice consistency than controls. Using a Bayesian model to parse the groups' learning strategies, we also found that VS lesion monkeys defaulted to an action-based choice strategy. Therefore, the VS is involved specifically in learning the value of stimuli, not actions.SIGNIFICANCE STATEMENT Reinforcement learning models of the ventral striatum (VS) often assume that it maintains an estimate of state value. This suggests that it plays a general role in learning whether rewards are assigned based on a chosen action or stimulus. In the present experiment, we examined the effects of VS lesions on monkeys' ability to learn that choosing a particular action or stimulus was more likely to lead to reward. We found that VS lesions caused a specific deficit in the monkeys' ability to discriminate between images with different values, whereas their ability to discriminate between actions with different values remained intact. Our results therefore suggest that the VS plays a specific role in learning to select rewarded stimuli.