δ-Opioid receptors in the accumbens shell mediate the influence of both excitatory and inhibitory predictions on choice

Dopamine errors drive excitatory and inhibitory components of backward conditioning in an outcome-specific manner

For over two decades, phasic activity in midbrain dopamine neurons was considered synonymous with the prediction error in temporal-difference reinforcement learning.^1-4 Central to this proposal is the notion that reward-predictive stimuli become endowed with the scalar value of predicted rewards. When these cues are subsequently encountered, their predictive value is compared to the value of the actual reward received, allowing for the calculation of prediction errors.^5,6 Phasic firing of dopamine neurons was proposed to reflect this computation,^1,2 facilitating the backpropagation of value from the predicted reward to the reward-predictive stimulus, thus reducing future prediction errors. There are two critical assumptions of this proposal: (1) that dopamine errors can only facilitate learning about scalar value and not more complex features of predicted rewards, and (2) that the dopamine signal can only be involved in anticipatory cue-reward learning in which cues or actions precede rewards. Recent work^7-15 has challenged the first assumption, demonstrating that phasic dopamine signals across species are involved in learning about more complex features of the predicted outcomes, in a manner that transcends this value computation. Here, we tested the validity of the second assumption. Specifically, we examined whether phasic midbrain dopamine activity would be necessary for backward conditioning-when a neutral cue reliably follows a rewarding outcome.^16-20 Using a specific Pavlovian-to-instrumental transfer (PIT) procedure,^21-23 we show rats learn both excitatory and inhibitory components of a backward association, and that this association entails knowledge of the specific identity of the reward and cue. We demonstrate that brief optogenetic inhibition of VTA_DA neurons timed to the transition between the reward and cue reduces both of these components of backward conditioning. These findings suggest VTA_DA neurons are capable of facilitating associations between contiguously occurring events, regardless of the content of those events. We conclude that these data may be in line with suggestions that the VTA_DA error acts as a universal teaching signal. This may provide insight into why dopamine function has been implicated in myriad psychological disorders that are characterized by very distinct reinforcement-learning deficits.

How predictive learning influences choice: Evidence for a GPCR-based memory process necessary for Pavlovian-instrumental transfer

Predictive learning endows stimuli with the capacity to signal both the sensory-specific and general motivational properties of their associated rewards or outcomes. These two signals can be distinguished behaviorally by their influence on the selection and performance of instrumental actions, respectively. This review focuses on how sensory-specific predictive learning guides choice between actions that earn otherwise equally desirable outcomes. We describe evidence that outcome-specific predictive learning is encoded in the basolateral amygdala and drives the accumulation of delta-opioid receptors on the surface of cholinergic interneurons located in the nucleus accumbens shell. This accumulation constitutes a novel form of cellular memory, not for outcome-specific predictive learning per se but for the selection of, and choice between, future instrumental actions. We describe recent evidence regarding the cascade of events necessary for the formation and expression of this cellular memory and point to open questions for future research into this process. Beyond these mechanistic considerations, the discovery of this new form of memory is consistent with recent evidence suggesting that intracellular rather than synaptic changes can mediate learning-related plasticity to modify brain circuitry to prepare for future significant events.

Motivational competition and the paraventricular thalamus

Although significant progress has been made in understanding the behavioral and brain mechanisms for motivational systems, much less is known about competition between motivational states or motivational conflict (e.g., approach - avoidance conflict). Despite being produced under diverse conditions, behavior during motivational competition has two signatures: bistability and metastability. These signatures reveal the operation of positive feedback mechanisms in behavioral selection. Different neuronal architectures can instantiate this selection to achieve bistability and metastability in behavior, but each relies on circuit-level inhibition to achieve rapid and stable selection between competing tendencies. Paraventricular thalamus (PVT) is identified as critical to this circuit level inhibition, resolving motivational competition via its extensive projections to local inhibitory networks in the ventral striatum and extended amygdala, enabling adaptive responding under motivational conflict.

The effect of conditioned inhibitors and preexposed cues on the outcome-specific Pavlovian-to-instrumental transfer effect in humans

Using a human Pavlovian-to-instrumental transfer (PIT) task, Alarcón and Bonardi showed that the selective elevation of instrumental responding produced by excitatory transfer cues was reduced when these cues were presented with a conditioned inhibitor (CI), relative to a control cue that was simply preexposed. However, previous research has shown that preexposed cues might also acquire inhibitor-like properties. This study aimed to contrast the inhibitory properties of CIs and preexposed cues, using novel stimuli as controls, in summation and PIT tests. Participants were trained to perform two actions, each reinforced with a distinct outcome (O₁ or O₂). Two images were trained as CIs, each signalling the absence of one of the outcomes, by presenting them with a cue that was otherwise followed by that outcome (e.g., A→O₁, AI→no O₁). In contrast, the preexposed cues were simply presented in the absence of the outcomes. In the summation test, participants rated the likelihood of the outcomes in the presence of two independently trained excitatory cues, each presented with a CI, a preexposed cue, or a novel stimulus. Similarly, in the PIT test, participants performed both actions in the presence and absence of these compounds. In the summation test, the CIs and the preexposed cues reduced participants' expectations of the outcomes more than the novel stimuli. However, in the PIT test, only the CIs reduced the selective elevation of responding produced by the transfer cues. These results might reflect distinct properties of stimuli trained as CIs and those simply preexposed.

Delta opioid receptor agonists are effective for multiple types of headache disorders

Headaches are highly disabling and are among the most common neurological disorders worldwide. Despite the high prevalence of headache, therapeutic options are limited. We recently identified the delta opioid receptor (DOR) as an emerging therapeutic target for migraine. In this study, we examined the effectiveness of a hallmark DOR agonist, SNC80, in disease models reflecting diverse headache disorders including: chronic migraine, post-traumatic headache (PTH), medication overuse headache by triptans (MOH), and opioid-induced hyperalgesia (OIH). To model chronic migraine C57BL/6J mice received chronic intermittent treatment with the known human migraine trigger, nitroglycerin. PTH was modeled by combining the closed head weight drop model with the nitroglycerin model of chronic migraine. For MOH and OIH, mice were chronically treated with sumatriptan or morphine, respectively. The development of periorbital and peripheral allodynia was observed in all four models; and SNC80 significantly inhibited allodynia in all cases. In addition, we also determined if chronic daily treatment with SNC80 would induce MOH/OIH, and we observed limited hyperalgesia relative to sumatriptan or morphine. Together, our results indicate that DOR agonists could be effective in multiple headache disorders, despite their distinct etiology, thus presenting a novel therapeutic target for headache.

Associative mechanisms involved in specific Pavlovian-to-instrumental transfer in human learning tasks

Four experiments compared the effect of forward and backward conditioning procedures on the ability of conditioned stimuli (CS) to elevate instrumental responding in a Pavlovian-to-instrumental transfer (PIT) task. Two responses were each trained with one distinct outcome (R1->O1, R2->O2), either concurrently (Experiment 1) or separately (Experiments 2, 3 and 4). Then, in Experiments 1 and 2, four CSs were either followed or preceded by one outcome (A->O1, B->O2, O1->C, O2->D). In Experiment 3, each CS was preceded and followed by an outcome: for one group of participants, both outcomes were identical (e.g., O1->A->O1, O2->B->O2), but for the other, they were different (e.g., O1->A->O2, O2->B->O1). In Experiment 4, two CSs were preceded and followed by identical outcomes, and two CSs by different outcomes. In the PIT tests, participants performed R1 and R2 in the presence and absence of the CSs. In Experiments 1 and 2, only the CSs followed by outcomes in Pavlovian training elevated responding. In Experiments 3 and 4, all the CSs elevated responding but based on the outcome that followed them in training. These results support the stimulus-outcome-response (S-O-R) mechanism of specific PIT, according to which CSs elevate responding via activation of its associated outcome representation.

Food Addiction and Binge Eating: Lessons Learned from Animal Models

The feeding process is required for basic life, influenced by environment cues and tightly regulated according to demands of the internal milieu by regulatory brain circuits. Although eating behaviour cannot be considered "addictive" under normal circumstances, people can become "addicted" to this behaviour, similarly to how some people are addicted to drugs. The symptoms, cravings and causes of "eating addiction" are remarkably similar to those experienced by drug addicts, and both drug-seeking behaviour as eating addiction share the same neural pathways. However, while the drug addiction process has been highly characterised, eating addiction is a nascent field. In fact, there is still a great controversy over the concept of "food addiction". This review aims to summarize the most relevant animal models of "eating addictive behaviour", emphasising binge eating disorder, that could help us to understand the neurobiological mechanisms hidden under this behaviour, and to improve the psychotherapy and pharmacological treatment in patients suffering from these pathologies.

The Lateral Habenula and Its Input to the Rostromedial Tegmental Nucleus Mediates Outcome-Specific Conditioned Inhibition

Animals can readily learn that stimuli predict the absence of specific appetitive outcomes; however, the neural substrates underlying such outcome-specific conditioned inhibition remain largely unexplored. Here, using female and male rats as subjects, we examined the involvement of the lateral habenula (LHb) and of its inputs onto the rostromedial tegmental nucleus (RMTg) in inhibitory learning. In these experiments, we used backward conditioning and contingency reversal to establish outcome-specific conditioned inhibitors for two distinct appetitive outcomes. Then, using the Pavlovian-instrumental transfer paradigm, we assessed the effects of manipulations of the LHb and the LHb-RMTg pathway on that inhibitory encoding. In control animals, we found that an outcome-specific conditioned inhibitor biased choice away from actions delivering that outcome and toward actions earning other outcomes. Importantly, this bias was abolished by both electrolytic lesions of the LHb and selective ablation of LHb neurons using Cre-dependent Caspase3 expression in Cre-expressing neurons projecting to the RMTg. This deficit was specific to conditioned inhibition; an excitatory predictor of a specific outcome-biased choice toward actions delivering the same outcome to a similar degree whether the LHb or the LHb-RMTg network was intact or not. LHb lesions also disrupted the ability of animals to inhibit previously encoded stimulus-outcome contingencies after their reversal, pointing to a critical role of the LHb and of its inputs onto the RMTg in outcome-specific conditioned inhibition in appetitive settings. These findings are consistent with the developing view that the LHb promotes a negative reward prediction error in Pavlovian conditioning.SIGNIFICANCE STATEMENT Stimuli that positively or negatively predict rewarding outcomes influence choice between actions that deliver those outcomes. Previous studies have found that a positive predictor of a specific outcome biases choice toward actions delivering that outcome. In contrast, a negative predictor of an outcome biases choice away from actions earning that outcome and toward other actions. Here we reveal that the lateral habenula is critical for negative predictors, but not positive predictors, to affect choice. Furthermore, these effects were found to require activation of lateral habenula inputs to the rostromedial tegmental nucleus. These results are consistent with the view that the lateral habenula establishes inhibitory relationships between stimuli and food outcomes and computes a negative prediction error in Pavlovian conditioning.

Molecular Pharmacology of δ-Opioid Receptors

Opioids are among the most effective analgesics available and are the first choice in the treatment of acute severe pain. However, partial efficacy, a tendency to produce tolerance, and a host of ill-tolerated side effects make clinically available opioids less effective in the management of chronic pain syndromes. Given that most therapeutic opioids produce their actions via µ-opioid receptors (MOPrs), other targets are constantly being explored, among which δ-opioid receptors (DOPrs) are being increasingly considered as promising alternatives. This review addresses DOPrs from the perspective of cellular and molecular determinants of their pharmacological diversity. Thus, DOPr ligands are examined in terms of structural and functional variety, DOPrs' capacity to engage a multiplicity of canonical and noncanonical G protein-dependent responses is surveyed, and evidence supporting ligand-specific signaling and regulation is analyzed. Pharmacological DOPr subtypes are examined in light of the ability of DOPr to organize into multimeric arrays and to adopt multiple active conformations as well as differences in ligand kinetics. Current knowledge on DOPr targeting to the membrane is examined as a means of understanding how these receptors are especially active in chronic pain management. Insight into cellular and molecular mechanisms of pharmacological diversity should guide the rational design of more effective, longer-lasting, and better-tolerated opioid analgesics for chronic pain management.

Endogenous Opiates and Behavior: 2015

This paper is the thirty-eighth consecutive installment of the annual review of research concerning the endogenous opioid system. It summarizes papers published during 2015 that studied the behavioral effects of molecular, pharmacological and genetic manipulation of opioid peptides, opioid receptors, opioid agonists and opioid antagonists. The particular topics that continue to be covered include the molecular-biochemical effects and neurochemical localization studies of endogenous opioids and their receptors related to behavior, and the roles of these opioid peptides and receptors in pain and analgesia, stress and social status, tolerance and dependence, learning and memory, eating and drinking, drug abuse and alcohol, sexual activity and hormones, pregnancy, development and endocrinology, mental illness and mood, seizures and neurologic disorders, electrical-related activity and neurophysiology, general activity and locomotion, gastrointestinal, renal and hepatic functions, cardiovascular responses, respiration and thermoregulation, and immunological responses.

Extinction Generates Outcome-Specific Conditioned Inhibition

Extinction involves altering a previously established predictive relationship between a cue and its outcome by repeatedly presenting that cue alone. Although it is widely accepted that extinction generates some form of inhibitory learning [1-4], direct evidence for this claim has been lacking, and the nature of the associative changes induced by extinction have, therefore, remained a matter of debate [5-8]. In the current experiments, we used a novel behavioral approach that we recently developed and that provides a direct measure of conditioned inhibition [9] to compare the influence of extinguished and non-extinguished cues on choice between goal-directed actions. Using this approach, we provide direct evidence that extinction generates outcome-specific conditioned inhibition. Furthermore, we demonstrate that this inhibitory learning is controlled by the infralimbic cortex (IL); inactivation of the IL using M4 DREADDs abolished outcome-specific inhibition and rendered the cue excitatory. Importantly, we found that context modulated this inhibition. Outside its extinction context, the cue was excitatory and functioned as a specific predictor of its previously associated outcome, biasing choice toward actions earning the same outcome. In its extinction context, however, the cue acted as a specific inhibitor and biased choice toward actions earning different outcomes. Context modulation of these excitatory and inhibitory memories was mediated by the dorsal hippocampus (HPC), suggesting that the HPC and IL act in concert to control the influence of conditioned inhibitors on choice. These findings demonstrate for the first time that extinction turns a cue into a net inhibitor that can influence choice via counterfactual action-outcome associations.

Ventral pallidal projections to mediodorsal thalamus and ventral tegmental area play distinct roles in outcome-specific Pavlovian-instrumental transfer

Outcome-specific Pavlovian-instrumental transfer (PIT) demonstrates the way that reward-related cues influence choice between instrumental actions. The nucleus accumbens shell (NAc-S) contributes critically to this effect, particularly through its output to the rostral medial ventral pallidum (VP-m). Using rats, we investigated in two experiments the role in the PIT effect of the two major outputs of this VP-m region innervated by the NAc-S, the mediodorsal thalamus (MD) and the ventral tegmental area (VTA). First, two retrograde tracers were injected into the MD and VTA to compare the neuronal activity of the two populations of projection neurons in the VP-m during PIT relative to controls. Second, the functional role of the connection between the VP-m and the MD or VTA was assessed using asymmetrical pharmacological manipulations before a PIT test. It was found that, whereas neurons in the VP-m projecting to the MD showed significantly more neuronal activation during PIT than those projecting to the VTA, neuronal activation of these latter neurons correlated with the size of the PIT effect. Disconnection of the two pathways during PIT also revealed different deficits in performance: disrupting the VP-m to MD pathway removed the response biasing effects of reward-related cues, whereas disrupting the VP-m to VTA pathway preserved the response bias but altered the overall rate of responding. The current results therefore suggest that the VP-m exerts distinct effects on the VTA and MD and that these latter structures mediate the motivational and cognitive components of specific PIT, respectively.

Fluorescent knock-in mice to decipher the physiopathological role of G protein-coupled receptors

G protein-coupled receptors (GPCRs) modulate most physiological functions but are also critically involved in numerous pathological states. Approximately a third of marketed drugs target GPCRs, which places this family of receptors in the main arena of pharmacological pre-clinical and clinical research. The complexity of GPCR function demands comprehensive appraisal in native environment to collect in-depth knowledge of receptor physiopathological roles and assess the potential of therapeutic molecules. Identifying neurons expressing endogenous GPCRs is therefore essential to locate them within functional circuits whereas GPCR visualization with subcellular resolution is required to get insight into agonist-induced trafficking. Both remain frequently poorly investigated because direct visualization of endogenous receptors is often hampered by the lack of appropriate tools. Also, monitoring intracellular trafficking requires real-time visualization to gather in-depth knowledge. In this context, knock-in mice expressing a fluorescent protein or a fluorescent version of a GPCR under the control of the endogenous promoter not only help to decipher neuroanatomical circuits but also enable real-time monitoring with subcellular resolution thus providing invaluable information on their trafficking in response to a physiological or a pharmacological challenge. This review will present the animal models and discuss their contribution to the understanding of the physiopathological role of GPCRs. We will also address the drawbacks associated with this methodological approach and browse future directions.

Learning and Motivational Processes Contributing to Pavlovian-Instrumental Transfer and Their Neural Bases: Dopamine and Beyond

Pavlovian stimuli exert a range of effects on behavior from simple conditioned reflexes, such as salivation, to altering the vigor and direction of instrumental actions. It is currently accepted that these distinct behavioral effects stem from two sources (i) the various associative connections between predictive stimuli and the component features of the events that these stimuli predict and (ii) the distinct motivational and cognitive functions served by cues, particularly their arousing and informational effects on the selection and performance of specific actions. Here, we describe studies that have assessed these latter phenomena using a paradigm that has come to be called Pavlovian-instrumental transfer. We focus first on behavioral experiments that have described distinct sources of stimulus control derived from the general affective and outcome-specific predictions of conditioned stimuli, referred to as general transfer and specific transfer, respectively. Subsequently, we describe research efforts attempting to establish the neural bases of these transfer effects, largely in the afferent and efferent connections of the nucleus accumbens (NAc) core and shell. Finally, we examine the role of predictive cues in examples of aberrant stimulus control associated with psychiatric disorders and addiction.