The ventral striato-pallidal pathway mediates the effect of predictive learning on choice between goal-directed actions

A ventral pallidal-thalamocortical circuit mediates the cognitive control of instrumental action

Predictive learning can engage a selective form of cognitive control that biases choice between actions based on information about future outcomes that the learning provides. This influence has been hypothesized to depend on a feedback circuit in the brain through which the basal ganglia modulate activity in the prefrontal cortex; however, direct evidence for this functional circuit has proven elusive. Here, using an animal model of cognitive control, we found that the influence of predictive learning on decision making is mediated by an inhibitory feedback circuit linking the medial ventral pallidum and the mediodorsal thalamus, the activation of which causes disinhibition of the orbitofrontal cortex via reduced activation of inhibitory parvalbumin interneurons during choice. Thus, we found that, for this function, the mediodorsal thalamus serves as a pallidal-cortical relay through which predictive learning controls action selection, which has important implications for understanding cognitive control and its vicissitudes in various psychiatric disorders and addiction.

Ventral Pallidum and Amygdala Cooperate to Restrain Reward Approach under Threat

Foraging decisions involve assessing potential risks and prioritizing food sources, which can be challenging when confronted with changing and conflicting circumstances. A crucial aspect of this decision-making process is the ability to actively overcome defensive reactions to threats and focus on achieving specific goals. The ventral pallidum (VP) and basolateral amygdala (BLA) are two brain regions that play key roles in regulating behavior motivated by either rewards or threats. However, it is unclear whether these regions are necessary in decision-making processes involving competing motivational drives during conflict. Our aim was to investigate the requirements of the VP and BLA for foraging choices in conflicts involving overcoming defensive responses. Here, we used a novel foraging task and pharmacological techniques to inactivate either the VP or BLA or to disconnect these brain regions before conducting a conflict test in male rats. Our findings showed that BLA is necessary for making risky choices during conflicts, whereas VP is necessary for invigorating the drive to obtain food, regardless of the presence of conflict. Importantly, our research revealed that the connection between VP and BLA is critical in controlling risky food-seeking choices during conflict situations. This study provides a new perspective on the collaborative function of VP and BLA in driving behavior, aimed at achieving goals in the face of dangers.

Inactivation of phosphodiesterase-4B gene in rat nucleus accumbens shell by CRISPR/Cas9 or positive allosteric modulation of the protein affects the motivation to chronically self-administer nicotine

Large scale human genome wide association studies (GWAS) have identified a growing pool of genes associated with cigarette smoking. One of the most prominent, phosphodiesterase-4B (PDE4B), has been associated with multiple smoking phenotypes. Although PDE4B modulates the half-life of neuronal cAMP, its precise role in smoking behaviors is unknown. To address this knowledge gap, we used a reverse translational approach. We inactivated PDE4B in bilateral medial nucleus accumbens shell (NAcs) neurons by injecting AAV containing a specific gRNA in female transgenic Cas9+ Long Evans rats. These rats then were given 23-h chronic access to nicotine intravenous self-administration (IVSA) under a schedule of increasing fixed ratios (FR). With the increased effort required at FR7, nicotine SA (i.e. active presses and drug infusions) declined significantly in controls, whereas it was maintained in the mutagenized group. A progressive ratio (PR) study also showed significantly greater cumulative nicotine infusions in the PDE4B-edited group. Hence, we hypothesized that enhanced PDE4B protein activity would reduce nicotine IVSA. A positive allosteric modulator, 2-(3-(4-chloro-3-fluorophenyl)-5-ethyl-1H-1,2,4-triazol-1-yl)-N-(3,5-dichlorobenzyl)acetamide (MR-L2), was microinfused into NAcs bilaterally at FR3 or FR5; in both cohorts, MR-L2 acutely reduced nicotine IVSA. In summary, these studies show that the activity of PDE4B regulates the capacity of NAcs to maintain nicotine IVSA in face of the cost of increasing work. This finding and the results of the PR study indicate that PDE4B affects the motivation to obtain nicotine. These reverse translational studies in rats provide insight into the motivational effects of NAcs PDE4B that advance our understanding of the smoking behaviors mapped in human GWAS.

Inactivation of phosphodiesterase-4B gene in rat nucleus accumbens shell by CRISPR/Cas9 modulates the motivation to chronically self-administer nicotine

Large scale human genome wide association studies (GWAS) have identified a growing pool of genes associated with cigarette smoking. One of the most prominent, phosphodiesterase-4B (PDE4B), has been associated with multiple smoking phenotypes. Although PDE4B modulates the half-life of neuronal cAMP, its precise role in smoking behaviors is unknown. To address this knowledge gap, we used a reverse translational approach. We inactivated PDE4B in bilateral medial nucleus accumbens shell (NAcs) neurons by injecting AAV containing a specific gRNA in female transgenic Cas9+ Long Evans rats. These rats then were given 23-hour chronic access to nicotine intravenous self-administration (IVSA) under a schedule of increasing fixed ratios (FR). With the increased effort required at FR7, nicotine SA (i.e. active presses and drug infusions) declined significantly in controls, whereas it was maintained in the mutagenized group. A progressive ratio (PR) study also showed significantly greater cumulative nicotine infusions in the mutant group. Hence, we hypothesized that enhanced PDE4B protein activity would reduce nicotine IVSA. A positive allosteric modulator,2-(3-(4-chloro-3-fluorophenyl)-5-ethyl-1H-1,2,4-triazol-1-yl)-N-(3,5-dichlorobenzyl)acetamide (MR-L2), was microinfused into NAcs bilaterally at FR3 or FR5; in both cohorts, MR-L2 acutely reduced nicotine IVSA. In summary, these studies show that the activity of PDE4B regulates the capacity of NAcs to maintain nicotine IVSA in face of the cost of increasing work. This finding and the results of the PR study indicate that PDE4B affects the motivation to obtain nicotine. These reverse translational studies in rats provide insight into the motivational effects of NAcs PDE4B that advance our understanding of the smoking behaviors mapped in human GWAS.

Ventral pallidal regulation of motivated behaviors and reinforcement

The interconnected nuclei of the ventral basal ganglia have long been identified as key regulators of motivated behavior, and dysfunction of this circuit is strongly implicated in mood and substance use disorders. The ventral pallidum (VP) is a central node of the ventral basal ganglia, and recent studies have revealed complex VP cellular heterogeneity and cell- and circuit-specific regulation of reward, aversion, motivation, and drug-seeking behaviors. Although the VP is canonically considered a relay and output structure for this circuit, emerging data indicate that the VP is a central hub in an extensive network for reward processing and the regulation of motivation that extends beyond classically defined basal ganglia borders. VP neurons respond temporally faster and show more advanced reward coding and prediction error processing than neurons in the upstream nucleus accumbens, and regulate the activity of the ventral mesencephalon dopamine system. This review will summarize recent findings in the literature and provide an update on the complex cellular heterogeneity and cell- and circuit-specific regulation of motivated behaviors and reinforcement by the VP with a specific focus on mood and substance use disorders. In addition, we will discuss mechanisms by which stress and drug exposure alter the functioning of the VP and produce susceptibility to neuropsychiatric disorders. Lastly, we will outline unanswered questions and identify future directions for studies necessary to further clarify the central role of VP neurons in the regulation of motivated behaviors. Significance: Research in the last decade has revealed a complex cell- and circuit-specific role for the VP in reward processing and the regulation of motivated behaviors. Novel insights obtained using cell- and circuit-specific interrogation strategies have led to a major shift in our understanding of this region. Here, we provide a comprehensive review of the VP in which we integrate novel findings with the existing literature and highlight the emerging role of the VP as a linchpin of the neural systems that regulate motivation, reward, and aversion. In addition, we discuss the dysfunction of the VP in animal models of neuropsychiatric disorders.

Outcome devaluation by specific satiety disrupts sensory-specific Pavlovian-to-instrumental transfer

Reward predictive cues can selectively motivate instrumental behaviors that predict the same rewarding outcomes, an effect known as specific Pavlovian-to-instrumental transfer (PIT). This selective effect is thought to be mediated by a representation of the sensory specific properties of an outcome, that has become associated with both the Pavlovian cue and the instrumental response during initial learning. Specific satiety is a common method of outcome devaluation that reduces an outcome's value but might also lead to the habituation of the outcome's sensory properties. Previous research has demonstrated that specific PIT is insensitive to changes in specific outcome value following taste aversion devaluation, as well as general satiety manipulations, and therefore specific satiety should not disrupt specific PIT by reducing outcome value. The present rodent experiments used a specific satiety devaluation procedure immediately prior to a specific PIT test to show that habituation of these outcome specific sensory representations can disrupt its efficacy as a stimulus and abolish the specific PIT effect. Experiment 1 employed a two-lever choice test to show that a non-devalued stimulus supports specific PIT, whereas a devalued stimulus abolished the specific PIT effect. Experiment 2 replicated this procedure while controlling for response competition by using a single-lever test to confirm that a devalued stimulus abolishes the specific PIT effect. These findings demonstrate that specific satiety can disrupt the ability of an outcome specific representation to support specific PIT. Given previous findings that specific PIT is insensitive to changes in outcome value by general satiety and taste aversion devaluation, this suggests that specific satiety devaluation might disrupt the use of sensory specific outcome representations to guide behavior via a mechanism that is independent of the outcome's current value.

Food cue reactivity: Neurobiological and behavioral underpinnings

The modern obesogenic environment contains an abundance of food cues (e.g., sight, smell of food) as well cues that are associated with food through learning and memory processes. Food cue exposure can lead to food seeking and excessive consumption in otherwise food-sated individuals, and a high level of food cue responsivity is a risk factor for overweight and obesity. Similar food cue responses are observed in experimental rodent models, and these models are therefore useful for mechanistically identifying the neural circuits mediating food cue responsivity. This review draws from both experimental rodent models and human data to characterize the behavioral and biological processes through which food-associated stimuli contribute to overeating and weight gain. Two rodent models are emphasized - cue-potentiated feeding and Pavlovian-instrumental transfer - that provide insight in the neural circuits and peptide systems underlying food cue responsivity. Data from humans are highlighted that reveal physiological, psychological, and neural mechanisms that connect food cue responsivity with overeating and weight gain. The collective literature identifies connections between heightened food cue responsivity and obesity in both rodents and humans, and identifies underlying brain regions (nucleus accumbens, amygdala, orbitofrontal cortex, hippocampus) and endocrine systems (ghrelin) that regulate food cue responsivity in both species. These species similarities are encouraging for the possibility of mechanistic rodent model research and further human research leading to novel treatments for excessive food cue responsivity in humans.

Neurologic Effects of Gadolinium Retention in the Brain after Gadolinium-based Contrast Agent Administration

Background Concerns over the neurotoxic potential of retained gadolinium in brain tissues after intravenous gadolinium-based contrast agent (GBCA) administration have led to pronounced worldwide use changes, yet the clinical sequelae of gadolinium retention remain undefined. Purpose To assess clinical and neurologic effects and potential neurotoxicity of gadolinium retention in rats after administration of various GBCAs. Materials and Methods From March 2017 through July 2018, 183 male Wistar rats received 20 intravenous injections of 2.5 mmol per kilogram of body weight (80 human equivalent doses) of various GBCAs (gadodiamide, gadobenate, gadopentetate, gadoxetate, gadobutrol, gadoterate, and gadoteridol) or saline over 4 weeks. Rats were evaluated 6 and 34 weeks after injection with five behavioral tests, and inductively coupled plasma mass spectrometry, transmission electron microscopy, and histopathology were performed on urine, serum, cerebrospinal fluid (CSF), basal ganglia, dentate nucleus, and kidney samples. Dunnett post hoc test and Wilcoxon rank sum test were used to compare differences between treatment groups. Results No evidence of differences in any behavioral test was observed between GBCA-exposed rats and control animals at either 6 or 34 weeks (P = .08 to P = .99). Gadolinium concentrations in both neuroanatomic locations were higher in linear GBCA-exposed rats than macrocyclic GBCA-exposed rats at 6 and 34 weeks (P < .001). Gadolinium clearance over time varied among GBCAs, with gadobutrol having the largest clearance (median: 62% for basal ganglia, 70% for dentate) and gadodiamide having no substantial clearance. At 34 weeks, gadolinium was largely cleared from the CSF and serum of gadodiamide-, gadobenate-, gadoterate-, and gadobutrol-exposed rats, especially for the macrocyclic agents (range: 70%-98% removal for CSF, 34%-94% removal for serum), and was nearly completely removed from urine (range: 96%-99% removal). Transmission electron microscopy was used to detect gadolinium foci in linear GBCA-exposed brain tissue, but no histopathologic differences were observed for any GBCA. Conclusion In this rat model, no clinical evidence of neurotoxicity was observed after exposure to linear and macrocyclic gadolinium-based contrast agents at supradiagnostic doses. © RSNA, 2022 Online supplemental material is available for this article.

Ventral arkypallidal neurons inhibit accumbal firing to promote reward consumption

The nucleus accumbens shell (NAcSh) and the ventral pallidum (VP) are critical for reward processing, although the question of how coordinated activity within these nuclei orchestrates reward valuation and consumption remains unclear. Inhibition of NAcSh firing is necessary for reward consumption, but the source of this inhibition remains unknown. Here, we report that a subpopulation of VP neurons, the ventral arkypallidal (vArky) neurons, project back to the NAcSh, where they inhibit NAcSh neurons in vivo in mice. Consistent with this pathway driving reward consumption via inhibition of the NAcSh, calcium activity of vArky neurons scaled with reward palatability (which was dissociable from reward seeking) and predicted the subsequent drinking behavior during a free-access paradigm. Activation of the VP-NAcSh pathway increased ongoing reward consumption while amplifying hedonic reactions to reward. These results establish a pivotal role for vArky neurons in the promotion of reward consumption through modulation of NAcSh firing in a value-dependent manner.

Instrumental and Pavlovian Mechanisms in Alcohol Use Disorder

Purpose of Review

Current theories of alcohol use disorders (AUD) highlight the importance of Pavlovian and instrumental learning processes mainly based on preclinical animal studies. Here, we summarize available evidence for alterations of those processes in human participants with AUD with a focus on habitual versus goal-directed instrumental learning, Pavlovian conditioning, and Pavlovian-to-instrumental transfer (PIT) paradigms.

Recent Findings

The balance between habitual and goal-directed control in AUD participants has been studied using outcome devaluation or sequential decision-making procedures, which have found some evidence of reduced goal-directed/model-based control, but little evidence for stronger habitual responding. The employed Pavlovian learning and PIT paradigms have shown considerable differences regarding experimental procedures, e.g., alcohol-related or conventional reinforcers or stimuli.

Summary

While studies of basic learning processes in human participants with AUD support a role of Pavlovian and instrumental learning mechanisms in the development and maintenance of drug addiction, current studies are characterized by large variability regarding methodology, sample characteristics, and results, and translation from animal paradigms to human research remains challenging. Longitudinal approaches with reliable and ecologically valid paradigms of Pavlovian and instrumental processes, including alcohol-related cues and outcomes, are warranted and should be combined with state-of-the-art imaging techniques, computational approaches, and ecological momentary assessment methods.

Motivational competition and the paraventricular thalamus

Although significant progress has been made in understanding the behavioral and brain mechanisms for motivational systems, much less is known about competition between motivational states or motivational conflict (e.g., approach - avoidance conflict). Despite being produced under diverse conditions, behavior during motivational competition has two signatures: bistability and metastability. These signatures reveal the operation of positive feedback mechanisms in behavioral selection. Different neuronal architectures can instantiate this selection to achieve bistability and metastability in behavior, but each relies on circuit-level inhibition to achieve rapid and stable selection between competing tendencies. Paraventricular thalamus (PVT) is identified as critical to this circuit level inhibition, resolving motivational competition via its extensive projections to local inhibitory networks in the ventral striatum and extended amygdala, enabling adaptive responding under motivational conflict.

Neural Networks With Motivation

Animals rely on internal motivational states to make decisions. The role of motivational salience in decision making is in early stages of mathematical understanding. Here, we propose a reinforcement learning framework that relies on neural networks to learn optimal ongoing behavior for dynamically changing motivation values. First, we show that neural networks implementing Q-learning with motivational salience can navigate in environment with dynamic rewards without adjustments in synaptic strengths when the needs of an agent shift. In this setting, our networks may display elements of addictive behaviors. Second, we use a similar framework in hierarchical manager-agent system to implement a reinforcement learning algorithm with motivation that both infers motivational states and behaves. Finally, we show that, when trained in the Pavlovian conditioning setting, the responses of the neurons in our model resemble previously published neuronal recordings in the ventral pallidum, a basal ganglia structure involved in motivated behaviors. We conclude that motivation allows Q-learning networks to quickly adapt their behavior to conditions when expected reward is modulated by agent's dynamic needs. Our approach addresses the algorithmic rationale of motivation and makes a step toward better interpretability of behavioral data via inference of motivational dynamics in the brain.

Complementary Roles for Ventral Pallidum Cell Types and Their Projections in Relapse

The ventral pallidum (VP) is a key node in the neural circuits controlling relapse to drug seeking. How this role relates to different VP cell types and their projections is poorly understood. Using male rats, we show how different forms of relapse to alcohol-seeking are assembled from VP cell types and their projections to lateral hypothalamus (LH) and ventral tegmental area (VTA). Using RNAScope in situ hybridization to characterize activity of different VP cell types during relapse to alcohol-seeking provoked by renewal (context-induced reinstatement), we found that VP Gad1 and parvalbumin (PV), but not vGlut2, neurons show relapse-associated changes in c-Fos expression. Next, we used retrograde tracing, chemogenetic, and electrophysiological approaches to study the roles of VPGad1 and VPPV neurons in relapse. We show that VPGad1 neurons contribute to contextual control over relapse (renewal), but not to relapse during reacquisition, via projections to LH, where they converge with ventral striatal inputs onto LHGad1 neurons. This convergence of striatopallidal inputs at the level of individual LHGad1 neurons may be critical to balancing propensity for relapse versus abstinence. In contrast, VPPV neurons contribute to relapse during both renewal and reacquisition via projections to VTA. These findings identify a double dissociation in the roles for different VP cell types and their projections in relapse. VPGad1 neurons control relapse during renewal via projections to LH. VPPV neurons control relapse during both renewal and reacquisition via projections to VTA. Targeting these different pathways may provide tailored interventions for different forms of relapse.SIGNIFICANCE STATEMENT Relapse to drug or reward seeking after a period of extinction or abstinence remains a key impediment to successful treatment. The ventral pallidum, located in the ventral basal ganglia, has long been recognized as an obligatory node in a 'final common pathway' for relapse. Yet how this role relates to the considerable VP cellular and circuit heterogeneity is not well understood. We studied the cellular and circuit architecture for VP in relapse control. We show that different forms of relapse have complementary VP cellular and circuit architectures, raising the possibility that targeting these different neural architectures may provide tailored interventions for different forms of relapse.

Interfacing behavioral and neural circuit models for habit formation

Habits are an important mechanism by which organisms can automate the control of behavior to alleviate cognitive demand. However, transitions to habitual control are risky because they lead to inflexible responding in the face of change. The question of how the brain controls transitions into habit is thus an intriguing one. How do we regulate when our repeated actions become automated? When is it advantageous or disadvantageous to release actions from cognitive control? Decades of research have identified a variety of methods for eliciting habitual responding in animal models. Progress has also been made to understand which brain areas and neural circuits control transitions into habit. Here, I discuss existing research on behavioral and neural circuit models for habit formation (with an emphasis on striatal circuits), and discuss strategies for combining information from different paradigms and levels of analysis to prompt further progress in the field.

Ventral pallidum encodes relative reward value earlier and more robustly than nucleus accumbens

The ventral striatopallidal system, a basal ganglia network thought to convert limbic information into behavioral action, includes the nucleus accumbens (NAc) and the ventral pallidum (VP), typically described as a major output of NAc. Here, to investigate how reward-related information is transformed across this circuit, we measure the activity of neurons in NAc and VP when rats receive two highly palatable but differentially preferred rewards, allowing us to track the reward-specific information contained within the neural activity of each region. In VP, we find a prominent preference-related signal that flexibly reports the relative value of reward outcomes across multiple conditions. This reward-specific firing in VP is present in a greater proportion of the population and arises sooner following reward delivery than in NAc. Our findings establish VP as a preeminent value signaler and challenge the existing model of information flow in the ventral basal ganglia.

Neural encoding of choice during a delayed response task in primate striatum and orbitofrontal cortex

Reward outcomes are available in many diverse situations and all involve choice. If there are multiple outcomes each rewarding, then decisions regarding relative value lead to choosing one over another. Important factors related to choice context should be encoded and utilized for this form of adaptive choosing. These factors can include the number of alternatives, the pacing of choice behavior and the possibility to reverse one's choice. An essential step in understanding if the context of choice is encoded is to directly compare choice with a context in which choice is absent. Neural activity in orbitofrontal cortex and striatum encodes potential value parameters related to reward quality and quantity as well as relative preference. We examined how neural activations in these brain regions are sensitive to choice situations and potentially involved in a prediction for the upcoming outcome selection. Neural activity was recorded and compared between a two-choice spatial delayed response task and an imperative 'one-option' task. Neural activity was obtained that extended from the instruction cue to the movement similar to previous work utilizing the identical imperative task. Orbitofrontal and striatal neural responses depended upon the decision about the choice of which reward to collect. Moreover, signals to predictive instruction cues that precede choice were selective for the choice situation. These neural responses could reflect chosen value with greater information on relative value of individual options as well as encode choice context itself embedded in the task as a part of the post-decision variable.

Dynamic Encoding of Incentive Salience in the Ventral Pallidum: Dependence on the Form of the Reward Cue

Some rats are especially prone to attribute incentive salience to a cue (conditioned stimulus, CS) paired with food reward (sign-trackers, STs), but the extent they do so varies as a function of the form of the CS. Other rats respond primarily to the predictive value of a cue (goal-trackers, GTs), regardless of its form. Sign-tracking is associated with greater cue-induced activation of mesolimbic structures than goal-tracking; however, it is unclear how the form of the CS itself influences activity in neural systems involved in incentive salience attribution. Thus, our goal was to determine how different cue modalities affect neural activity in the ventral pallidum (VP), which is known to encode incentive salience attribution, as rats performed a two-CS Pavlovian conditioned approach task in which both a lever-CS and a tone-CS predicted identical food reward. The lever-CS elicited sign-tracking in some rats (STs) and goal-tracking in others (GTs), whereas the tone-CS elicited only goal-tracking in all rats. The lever-CS elicited robust changes in neural activity (sustained tonic increases or decreases in firing) throughout the VP in STs, relative to GTs. These changes were not seen when STs were exposed to the tone-CS, and in GTs there were no differences in firing between the lever-CS and tone-CS. We conclude that neural activity throughout the VP encodes incentive signals and is especially responsive when a cue is of a form that promotes the attribution of incentive salience to it, especially in predisposed individuals.

Pharmacological Blockade of Adenosine A2A but Not A1 Receptors Enhances Goal-Directed Valuation in Satiety-Based Instrumental Behavior

The balance and smooth shift between flexible, goal-directed behaviors and repetitive, habitual actions are critical to optimal performance of behavioral tasks. The striatum plays an essential role in control of goal-directed versus habitual behaviors through a rich interplay of the numerous neurotransmitters and neuromodulators to modify the input, processing and output functions of the striatum. The adenosine receptors (namely A_2AR and A₁R), with their high expression pattern in the striatum and abilities to interact and integrate dopamine, glutamate and cannabinoid signals in the striatum, may represent novel therapeutic targets for modulating instrumental behavior. In this study, we examined the effects of pharmacological blockade of the A_2ARs and A₁Rs on goal-directed versus habitual behaviors in different information processing phases of instrumental learning using a satiety-based instrumental behavior procedure. We found that A_2AR antagonist acts at the coding, consolidation and expression phases of instrumental learning to modulate animals’ sensitivity to goal-directed valuation without modifying action-outcome contingency. However, pharmacological blockade and genetic knockout of A₁Rs did not affect acquisition or sensitivity to goal-valuation of instrumental behavior. These findings provide pharmacological evidence for a potential therapeutic strategy to control abnormal instrumental behaviors associated with drug addiction and obsessive-compulsive disorder by targeting the A_2AR.

Ventral pallidal encoding of reward-seeking behavior depends on the underlying associative structure

Despite its being historically conceptualized as a motor expression site, emerging evidence suggests the ventral pallidum (VP) plays a more active role in integrating information to generate motivation. Here, we investigated whether rat VP cue responses would encode and contribute similarly to the vigor of reward-seeking behaviors trained under Pavlovian versus instrumental contingencies, when these behavioral responses consist of superficially similar locomotor response patterns but may reflect distinct underlying decision-making processes. We find that cue-elicited activity in many VP neurons predicts the latency of instrumental reward seeking, but not of Pavlovian response latency. Further, disruption of VP signaling increases the latency of instrumental but not Pavlovian reward seeking. This suggests that VP encoding of and contributions to response vigor are specific to the ability of incentive cues to invigorate reward-seeking behaviors upon which reward delivery is contingent.

Sounds or other cues associated with receiving a reward can have a powerful effect on an individual’s behavior or emotions. For example, the sound of an ice cream truck might cause salivation and motivate an individual to stand in a long line. Cues may prompt specific actions necessary to receive a reward, for example, approaching the ice cream truck and paying to get an ice cream. This is called instrumental conditioning. Some cues predict reward delivery, without requiring a specific action. This is called Pavlovian conditioning. Pavlovian cues can still prompt actions, such as approaching the truck, even though the action is not required. But exactly what happens in the brain to generate these actions during the two types of learning, is unclear.

Learning more about these reward-driven brain mechanisms might help scientists to develop better treatments for people with addiction or other conditions that involve compulsive reward-seeking behavior. Currently, scientists do not know enough about how the brain triggers this kind of behavior or how these processes lead to relapse in individuals who have been abstinent. Basic studies on the brain mechanisms that trigger reward-seeking behavior are needed.

Now, Richard et al. show that a greater activity in neurons, or brain cells, in a part of the brain called the ventral pallidum predicts a faster response to a reward cue. In the experiments, some rats were trained to approach a certain location when they heard a particular sound in order to receive sugar water, a form of instrumental conditioning. Another group of rats underwent Pavlovian training and learned to expect sugar water every time they heard sound even if they did nothing. Both groups learned to approach the sugar water location when they heard the cue, despite the different training requirements.

Richard et al. measured the activity of neurons in the ventral pallidum when the rats in the two groups heard the reward-associated sound. The experiments showed that the amount of activity in the brain cells in this area predicted whether a rat would approach the sugar-water delivery area and how quickly they would approach the reward after hearing the cue. The predictions were most reliable for rats that had to do something to get the sugar water. When Richard et al. reduced the activity in these cells they found the rats took longer to approach the reward source, but only when this action was required to receive sugar water. The experiments show that the ventral pallidum may provide the motivation to undertake reward-seeking behavior.

Paradoxical accentuation of motivation following accumbens-pallidum disconnection

The nucleus accumbens (NAc) and ventral pallidum (VP) are reciprocally connected, and activity within this circuit is thought to promote reward learning. Inconsistent with this notion, we find that disconnecting NAc medial shell and VP greatly enhances the attribution of value to a cue that is paired with reward. This result suggests that medial NAc shell and VP are both needed for attributing value to cues yet can also oppose one-another's functional contribution.

The Nucleus Accumbens: Mechanisms of Addiction across Drug Classes Reflect the Importance of Glutamate Homeostasis

The nucleus accumbens is a major input structure of the basal ganglia and integrates information from cortical and limbic structures to mediate goal-directed behaviors. Chronic exposure to several classes of drugs of abuse disrupts plasticity in this region, allowing drug-associated cues to engender a pathologic motivation for drug seeking. A number of alterations in glutamatergic transmission occur within the nucleus accumbens after withdrawal from chronic drug exposure. These drug-induced neuroadaptations serve as the molecular basis for relapse vulnerability. In this review, we focus on the role that glutamate signal transduction in the nucleus accumbens plays in addiction-related behaviors. First, we explore the nucleus accumbens, including the cell types and neuronal populations present as well as afferent and efferent connections. Next we discuss rodent models of addiction and assess the viability of these models for testing candidate pharmacotherapies for the prevention of relapse. Then we provide a review of the literature describing how synaptic plasticity in the accumbens is altered after exposure to drugs of abuse and withdrawal and also how pharmacological manipulation of glutamate systems in the accumbens can inhibit drug seeking in the laboratory setting. Finally, we examine results from clinical trials in which pharmacotherapies designed to manipulate glutamate systems have been effective in treating relapse in human patients. Further elucidation of how drugs of abuse alter glutamatergic plasticity within the accumbens will be necessary for the development of new therapeutics for the treatment of addiction across all classes of addictive substances.

Ventral Pallidum Output Pathways in Context-Induced Reinstatement of Alcohol Seeking

Ventral pallidum (VP) is a well-established locus for the reinforcing effects of drugs of abuse and reinstatement of drug seeking. However, VP neurons are at the origin of multiple output pathways, with strong projections to ventral tegmental area (VTA), subthalamic nucleus (STN), lateral hypothalamus, among others, and the roles of these VP output pathways in reinstatement of drug seeking remain poorly understood. Here we addressed these issues using a combination of neuroanatomical tracing and chemogenetic approaches. First, using dual-retrograde tracing, we show that VP neurons projecting to either VTA or STN are recruited during context-induced reinstatement of extinguished alcohol seeking in rats. Then, using chemogenetics, we show modulation of context-induced reinstatement and reacquisition of alcohol seeking via designer receptors exclusively activated by designer drugs excitation or inhibition of the VP. To determine the causal roles of VP → VTA and VP → STN pathways in context-induced reinstatement and reacquisition we used a chemogenetic disconnection approach and show that silencing either the VP → VTA or VP → STN pathways is sufficient to reduce both reinstatement and reacquisition of alcohol seeking. Moreover, these disconnections also each reduced responding and motivation during a progressive ratio test but had no effect on locomotor activity. Together, these results show that multiple ventral pallidal output pathways contribute to relapse to alcohol seeking.

SIGNIFICANCE STATEMENT

Ventral pallidum (VP) serves important roles in reward and motivation and is a critical node in the neural circuitry for reinstatement of drug seeking. Despite being a common locus for different forms of reinstatement, fundamental aspects of neural circuitry for these VP contributions to reinstatement of drug seeking remain unknown. Here we used a combination of neuroanatomical tracing and chemogenetic approaches to map the VP output pathways for context-induced reinstatement and reacquisition of alcohol seeking. We show that VP output pathways to the subthalamic nucleus and also to the ventral tegmental area are necessary for these forms of reinstatement.

How contexts promote and prevent relapse to drug seeking

The contexts where drugs are self-administered play an important role in regulating persistent drug taking and in relapse to such taking after periods of abstinence. Here, we review the behavioral and brain mechanisms enabling contexts to promote and prevent relapse to drug seeking. We review the key brain structures, their neuropharmacology and their connectivity. We discuss the similarities and differences between the mechanisms for context-induced reinstatement of drug seeking vs. other forms of relapse to drug seeking in animal models and we highlight the numerous deficits in our understanding. We emphasize that current understanding, although significant, defies explanations in terms of models at the level of brain structures and their connectivity. Rather, we show that there is significant functional compartmentalization and segregation within these structures during reinstatement and extinction of drug seeking that parallels their anatomical segregation into circuits and channels. A key challenge is to recognize this complexity, understand how these circuits and channels are organized, as well as understand how different modes of activity of ensembles of neurons within them promote abstinence or relapse to drug seeking.

Chemogenetic manipulation of ventral pallidal neurons impairs acquisition of sign-tracking in rats

Cues associated with rewarding events acquire value themselves as a result of the incentive value of the reward being transferred to the cue. Consequently, presentation of a reward-paired cue can trigger reward-seeking behaviours towards the cue itself (i.e. sign-tracking). The ventral pallidum (VP) has been demonstrated to be involved in a number of motivated behaviours, both conditioned and unconditioned. However, its contribution to the acquisition of incentive value is unknown. Using a discriminative autoshaping procedure with levers, the effects of disrupting VP activity in rats on the emergence of sign-tracking was investigated using chemogenetics, i.e. Designer Receptors Exclusively Activated by Designer Drugs (DREADDs). Transient disruption of VP neurons [activation of the inhibitory hM4D(Gi) DREADD through systemic injections of clozapine N-oxide (CNO) prior to each autoshaping session] impaired acquisition of sign-tracking (lever press rate) without having any effect on approach to the site of reward delivery (i.e. goal-tracking) or on the expression of sign-tracking after it was acquired. In addition, electrophysiological recordings were conducted in freely behaving rats following VP DREADD activation. The majority of VP units that were responsive to CNO injections exhibited rapid inhibition relative to baseline, a subset of CNO-responsive units showed delayed excitation, and a smaller subset displayed a mixed response of inhibition and excitation following CNO injections. It is argued that disruption of VP during autoshaping specifically disrupted the transfer of incentive value that was attributed to the lever cue, suggesting a surprisingly fundamental role for the VP in acquiring, compared with expressing, Pavlovian incentive values.

Neural Activity in the Ventral Pallidum Encodes Variation in the Incentive Value of a Reward Cue

There is considerable individual variation in the extent to which reward cues are attributed with incentive salience. For example, a food-predictive conditioned stimulus (CS; an illuminated lever) becomes attractive, eliciting approach toward it only in some rats ("sign trackers," STs), whereas others ("goal trackers," GTs) approach the food cup during the CS period. The purpose of this study was to determine how individual differences in Pavlovian approach responses are represented in neural firing patterns in the major output structure of the mesolimbic system, the ventral pallidum (VP). Single-unit in vivo electrophysiology was used to record neural activity in the caudal VP during the performance of ST and GT conditioned responses. All rats showed neural responses to both cue onset and reward delivery but, during the CS period, STs showed greater neural activity than GTs both in terms of the percentage of responsive neurons and the magnitude of the change in neural activity. Furthermore, neural activity was positively correlated with the degree of attraction to the cue. Given that the CS had equal predictive value in STs and GTs, we conclude that neural activity in the VP largely reflects the degree to which the CS was attributed with incentive salience.

SIGNIFICANCE STATEMENT

Cues associated with reward can acquire motivational properties (i.e., incentive salience) that cause them to have a powerful influence on desire and motivated behavior. There are individual differences in sensitivity to reward-paired cues, with some individuals attaching greater motivational value to cues than others. Here, we investigated the neural activity associated with these individual differences in incentive salience. We found that cue-evoked neural firing in the ventral pallidum (VP) reflected the strength of incentive motivation, with the greatest neural responses occurring in individuals that demonstrated the strongest attraction to the cue. This suggests that the VP plays an important role in the process by which cues gain control over motivation and behavior.

Ventral Pallidum Neurons Encode Incentive Value and Promote Cue-Elicited Instrumental Actions

The ventral pallidum (VP) is posited to contribute to reward seeking by conveying upstream signals from the nucleus accumbens (NAc). Yet, very little is known about how VP neuron responses contribute to behavioral responses to incentive cues. Here, we recorded activity of VP neurons in a cue-driven reward-seeking task previously shown to require neural activity in the NAc. We find that VP neurons encode both learned cue value and subsequent reward seeking and that activity in VP neurons is required for robust cue-elicited reward seeking. Surprisingly, the onset of VP neuron responses occurs at a shorter latency than cue-elicited responses in NAc neurons. This suggests that this VP encoding is not a passive response to signals generated in the NAc and that VP neurons integrate sensory and motivation-related information received directly from other mesocorticolimbic inputs.

Changes in D1 but not D2 dopamine or mu-opioid receptor expression in limbic and motor structures after lateral hypothalamus electrical self-stimulation: A quantitative autoradiographic study

Intracranial self-stimulation (ICSS) of the lateral hypothalamus (LH) is involved in the activation of neuroanatomical systems that are also associated with the processing of natural and other artificial rewarding stimuli. Specific components of this behavior (hedonic impact, learning, and motor behavior) may involve changes in different neurotransmitters, such as dopamine and opioids. In this study, quantitative autoradiography was used to examine changes in mu-opioid and D1/D2-dopamine receptor expression in various anatomical regions related to the motor and mesolimbic reward systems after intracranial self-stimulation of the LH. Results of the behavioral procedure and subsequent radiochemical assays show selective changes in D1 but not D2 or mu receptors in Accumbens-Shell, Ventral Pallidum, Caudate-Putamen, and Medial Globus Pallidus. These findings are discussed in relation to the different psychobiological components of the appetitive motivational system, identifying some dissociation among them, particularly with respect to the involvement of the D1-dopamine subsystem (but not D2 or mu receptors) in goal-directed behaviors.

The role of opioid processes in reward and decision-making

Contemporary theories emphasize the involvement of the endogenous opioid system in assigning hedonic values to rewards. Although earlier research supports this view, recent findings suggest that opioids play a larger and more complex role in reward processes than these theories suggest. For example, opioid activity in the basolateral amygdala is required for encoding incentive learning, a process by which the value of goal-directed actions is updated. Outside the amygdala, opioid receptors in the ventral striatum have been found to promote choice between different courses of action. Specifically, μ opioid receptors in the nucleus accumbens core and δ opioid receptors in the nucleus accumbens shell have been reported to mediate distinct aspects of incentive motivation; the core regulating the effect of experienced reward and the shell of predicted reward on choice. In both cases, the involvement of opioid receptors was restricted to the time of choice, although changes in their expression pattern could be observed prior to that point. This time-restricted involvement of opioid receptor-related processes is consistent with the view that opioids in the nucleus accumbens are central components of the limbic-motor interface, integrating reward-related information with instrumental learning to guide decision-making, particularly the selection and execution of goal-directed actions.

LINKED ARTICLES

This article is part of a themed section on Opioids: New Pathways to Functional Selectivity. To view the other articles in this section visit http://dx.doi.org/10.1111/bph.2015.172.issue-2.

The ventral pallidum: Subregion-specific functional anatomy and roles in motivated behaviors

The ventral pallidum (VP) plays a critical role in the processing and execution of motivated behaviors. Yet this brain region is often overlooked in published discussions of the neurobiology of mental health (e.g., addiction, depression). This contributes to a gap in understanding the neurobiological mechanisms of psychiatric disorders. This review is presented to help bridge the gap by providing a resource for current knowledge of VP anatomy, projection patterns and subregional circuits, and how this organization relates to the function of VP neurons and ultimately behavior. For example, ventromedial (VPvm) and dorsolateral (VPdl) VP subregions receive projections from nucleus accumbens shell and core, respectively. Inhibitory GABAergic neurons of the VPvm project to mediodorsal thalamus, lateral hypothalamus, and ventral tegmental area, and this VP subregion helps discriminate the appropriate conditions to acquire natural rewards or drugs of abuse, consume preferred foods, and perform working memory tasks. GABAergic neurons of the VPdl project to subthalamic nucleus and substantia nigra pars reticulata, and this VP subregion is modulated by, and is necessary for, drug-seeking behavior. Additional circuits arise from nonGABAergic neuronal phenotypes that are likely to excite rather than inhibit their targets. These subregional and neuronal phenotypic circuits place the VP in a unique position to process motivationally relevant stimuli and coherent adaptive behaviors.

Ventral pallidal projections to mediodorsal thalamus and ventral tegmental area play distinct roles in outcome-specific Pavlovian-instrumental transfer

Outcome-specific Pavlovian-instrumental transfer (PIT) demonstrates the way that reward-related cues influence choice between instrumental actions. The nucleus accumbens shell (NAc-S) contributes critically to this effect, particularly through its output to the rostral medial ventral pallidum (VP-m). Using rats, we investigated in two experiments the role in the PIT effect of the two major outputs of this VP-m region innervated by the NAc-S, the mediodorsal thalamus (MD) and the ventral tegmental area (VTA). First, two retrograde tracers were injected into the MD and VTA to compare the neuronal activity of the two populations of projection neurons in the VP-m during PIT relative to controls. Second, the functional role of the connection between the VP-m and the MD or VTA was assessed using asymmetrical pharmacological manipulations before a PIT test. It was found that, whereas neurons in the VP-m projecting to the MD showed significantly more neuronal activation during PIT than those projecting to the VTA, neuronal activation of these latter neurons correlated with the size of the PIT effect. Disconnection of the two pathways during PIT also revealed different deficits in performance: disrupting the VP-m to MD pathway removed the response biasing effects of reward-related cues, whereas disrupting the VP-m to VTA pathway preserved the response bias but altered the overall rate of responding. The current results therefore suggest that the VP-m exerts distinct effects on the VTA and MD and that these latter structures mediate the motivational and cognitive components of specific PIT, respectively.

Learning and Motivational Processes Contributing to Pavlovian-Instrumental Transfer and Their Neural Bases: Dopamine and Beyond

Pavlovian stimuli exert a range of effects on behavior from simple conditioned reflexes, such as salivation, to altering the vigor and direction of instrumental actions. It is currently accepted that these distinct behavioral effects stem from two sources (i) the various associative connections between predictive stimuli and the component features of the events that these stimuli predict and (ii) the distinct motivational and cognitive functions served by cues, particularly their arousing and informational effects on the selection and performance of specific actions. Here, we describe studies that have assessed these latter phenomena using a paradigm that has come to be called Pavlovian-instrumental transfer. We focus first on behavioral experiments that have described distinct sources of stimulus control derived from the general affective and outcome-specific predictions of conditioned stimuli, referred to as general transfer and specific transfer, respectively. Subsequently, we describe research efforts attempting to establish the neural bases of these transfer effects, largely in the afferent and efferent connections of the nucleus accumbens (NAc) core and shell. Finally, we examine the role of predictive cues in examples of aberrant stimulus control associated with psychiatric disorders and addiction.

The place of dopamine in the cortico-basal ganglia circuit

The midbrain dopamine (DA) neurons play a central role in developing appropriate goal-directed behaviors, including the motivation and cognition to develop appropriate actions to obtain a specific outcome. Indeed, subpopulations of DA neurons have been associated with these different functions: the mesolimbic, mesocortical, and nigrostriatal pathways. The mesolimbic and nigrostriatal pathways are an integral part of the basal ganglia through its reciprocal connections to the ventral and dorsal striatum respectively. This chapter reviews the connections of the midbrain DA cells and their role in integrating information across limbic, cognitive and motor functions. Emphasis is placed on the interface between these functional domains within the striatum through corticostriatal connections, through the striato-nigro-striatal connection, and through the lateral habenula projection to the midbrain.

δ-opioid and dopaminergic processes in accumbens shell modulate the cholinergic control of predictive learning and choice

Decision-making depends on the ability to extract predictive information from the environment to guide future actions. Outcome-specific Pavlovian-instrumental transfer (PIT) provides an animal model of this process in which a stimulus predicting a particular outcome biases choice toward actions earning that outcome. Recent evidence suggests that cellular adaptations of δ-opioid receptors (DORs) on cholinergic interneurons (CINs) in the nucleus accumbens shell (NAc-S) are necessary for PIT. Here we found that modulation of DORs in CINs critically influences D1-receptor (D1R)-expressing projection neurons in the NAc-S to promote PIT. First, we assessed PIT-induced changes in signaling processes in dopamine D1- and D2-receptor-expressing neurons using drd2-eGFP mice, and found that PIT-related signaling was restricted to non-D2R-eGFP-expressing neurons, suggesting major involvement of D1R-neurons. Next we confirmed the role of D1Rs pharmacologically: the D1R antagonist SCH-23390, but not the D2R antagonist raclopride, infused into the NAc-S abolished PIT in rats, an effect that depended on DOR activity. Moreover, asymmetrical infusion of SCH-23390 and the DOR antagonist naltrindole into the NAc-S also abolished PIT. DOR agonists were found to sensitize the firing responses of CINs in brain slices prepared immediately after the PIT test. We confirmed the opioid-acetylcholinergic influence over D1R-neurons by selectively blocking muscarinic M4 receptors in the NAc-S, which tightly regulate the activity of D1Rs, a treatment that rescued the deficit in PIT induced by naltrindole. We describe a model of NAc-S function in which DORs modulate CINs to influence both D1R-neurons and stimulus-guided choice between goal-directed actions.

Designer receptors show role for ventral pallidum input to ventral tegmental area in cocaine seeking

Ventral pallidum (VP) is centrally positioned within mesocorticolimbic reward circuits, and its dense projection to ventral tegmental area (VTA) regulates neuronal activity there. However, VP is a heterogeneous structure, and how this complexity affects its role within wider reward circuits is unclear. Here we demonstrate that projections to VTA from rostral (RVP), but not caudal VP (CVP) are robustly Fos-activated during cue-induced reinstatement of cocaine seeking—a rat model of relapse in addiction. Moreover, designer receptor-mediated transient inactivation of RVP neurons, their terminals in VTA, or functional connectivity between RVP and VTA dopamine neurons blocks the ability of drug-associated cues (but not a cocaine prime) to reinstate cocaine seeking. In contrast, CVP neuronal inhibition instead blocked cocaine-primed, but not cue-induced reinstatement. This novel double dissociation in VP sub-regional roles in drug seeking is likely important for understanding mesocorticolimbic circuits underlying reward seeking and addiction.