2
|
Navarro VM, Dwyer DM, Honey RC. Prediction error in models of adaptive behavior. Curr Biol 2023; 33:4238-4243.e3. [PMID: 37708886 DOI: 10.1016/j.cub.2023.08.043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Revised: 07/19/2023] [Accepted: 08/15/2023] [Indexed: 09/16/2023]
Abstract
Pavlovian conditioning is evident in every species in which it has been assessed, and there is a consensus about its interpretation across behavioral,1,2 brain,3,4,5,6 and computational analyses7,8,9,10,11: conditioned behavior reflects the formation of a directional associative link from the memory of one stimulus (e.g., a visual stimulus) to another (e.g., food), with learning stopping when there is no error between the prediction generated by the visual stimulus and what happens next (e.g., food). This consensus fails to anticipate the results that we report here. In our experiments with rats, we find that arranging predictive (visual stimulus→food) and nonpredictive (food→visual stimulus) relationships produces marked and sustained changes in conditioned behaviors when the visual stimulus is presented alone. Moreover, the type of relationship affects (1) the distribution of conditioned behaviors related to the properties of both food (called goal-tracking) and the visual stimulus (called sign-tracking) and (2) when in the visual stimulus, these two behaviors are evident. These results represent an impetus for a fundamental shift in how Pavlovian conditioning is interpreted: animals learn about the relationship between two stimuli irrespective of the order in which they are presented, but they exhibit this knowledge in different ways. This interpretation and our new results are captured by a recent model of Pavlovian conditioning,12,13 HeiDI, and both are consistent with the need for animals to represent the fact that the impact of a cause (e.g., the ingestion of nutrients or the bite of a predator) can be felt before or after the cause has been perceived.
Collapse
Affiliation(s)
- Victor M Navarro
- School of Psychology, Cardiff University, 70 Park Place, CF10 3AT Cardiff, UK
| | - Dominic M Dwyer
- School of Psychology, Cardiff University, 70 Park Place, CF10 3AT Cardiff, UK
| | - Robert C Honey
- School of Psychology, Cardiff University, 70 Park Place, CF10 3AT Cardiff, UK.
| |
Collapse
|
3
|
Wang W, Xie X, Zhuang X, Huang Y, Tan T, Gangal H, Huang Z, Purvines W, Wang X, Stefanov A, Chen R, Rodriggs L, Chaiprasert A, Yu E, Vierkant V, Hook M, Huang Y, Darcq E, Wang J. Striatal μ-opioid receptor activation triggers direct-pathway GABAergic plasticity and induces negative affect. Cell Rep 2023; 42:112089. [PMID: 36796365 PMCID: PMC10404641 DOI: 10.1016/j.celrep.2023.112089] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2022] [Revised: 12/27/2022] [Accepted: 01/26/2023] [Indexed: 02/17/2023] Open
Abstract
Withdrawal from chronic opioid use often causes hypodopaminergic states and negative affect, which may drive relapse. Direct-pathway medium spiny neurons (dMSNs) in the striatal patch compartment contain μ-opioid receptors (MORs). It remains unclear how chronic opioid exposure and withdrawal impact these MOR-expressing dMSNs and their outputs. Here, we report that MOR activation acutely suppressed GABAergic striatopallidal transmission in habenula-projecting globus pallidus neurons. Notably, withdrawal from repeated morphine or fentanyl administration potentiated this GABAergic transmission. Furthermore, intravenous fentanyl self-administration enhanced GABAergic striatonigral transmission and reduced midbrain dopaminergic activity. Fentanyl-activated striatal neurons mediated contextual memory retrieval required for conditioned place preference tests. Importantly, chemogenetic inhibition of striatal MOR+ neurons rescued fentanyl withdrawal-induced physical symptoms and anxiety-like behaviors. These data suggest that chronic opioid use triggers GABAergic striatopallidal and striatonigral plasticity to induce a hypodopaminergic state, which may promote negative emotions and relapse.
Collapse
Affiliation(s)
- Wei Wang
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA; Interdisciplinary Faculty of Toxicology, College of Veterinary Medicine and Biomedical Sciences, Texas A&M University, College Station, TX 77843, USA
| | - Xueyi Xie
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA
| | - Xiaowen Zhuang
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA
| | - Yufei Huang
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA; Institute for Neuroscience, Texas A&M University, College Station, TX 77843, USA
| | - Tao Tan
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA
| | - Himanshu Gangal
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA; Institute for Neuroscience, Texas A&M University, College Station, TX 77843, USA
| | - Zhenbo Huang
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA
| | - William Purvines
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA; Institute for Neuroscience, Texas A&M University, College Station, TX 77843, USA
| | - Xuehua Wang
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA
| | - Alexander Stefanov
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA; Institute for Neuroscience, Texas A&M University, College Station, TX 77843, USA
| | - Ruifeng Chen
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA; Interdisciplinary Faculty of Toxicology, College of Veterinary Medicine and Biomedical Sciences, Texas A&M University, College Station, TX 77843, USA
| | - Lucas Rodriggs
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA
| | - Anita Chaiprasert
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA
| | - Emily Yu
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA
| | - Valerie Vierkant
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA
| | - Michelle Hook
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA; Institute for Neuroscience, Texas A&M University, College Station, TX 77843, USA
| | - Yun Huang
- Institute of Biosciences and Technology, Department of Translational Medical Sciences, College of Medicine, Texas A&M University, Houston, TX 77030, USA
| | - Emmanuel Darcq
- Department of Psychiatry, University of Strasbourg, INSERM U1114, 67084 Strasbourg Cedex, France
| | - Jun Wang
- Department of Neuroscience and Experimental Therapeutics, College of Medicine, Texas A&M University Health Science Center, Bryan, TX 77807, USA; Interdisciplinary Faculty of Toxicology, College of Veterinary Medicine and Biomedical Sciences, Texas A&M University, College Station, TX 77843, USA; Institute for Neuroscience, Texas A&M University, College Station, TX 77843, USA; Institute of Biosciences and Technology, Department of Translational Medical Sciences, College of Medicine, Texas A&M University, Houston, TX 77030, USA.
| |
Collapse
|
4
|
Amo R, Matias S, Yamanaka A, Tanaka KF, Uchida N, Watabe-Uchida M. A gradual temporal shift of dopamine responses mirrors the progression of temporal difference error in machine learning. Nat Neurosci 2022; 25:1082-1092. [PMID: 35798979 PMCID: PMC9624460 DOI: 10.1038/s41593-022-01109-2] [Citation(s) in RCA: 28] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2020] [Accepted: 05/24/2022] [Indexed: 02/03/2023]
Abstract
A large body of evidence has indicated that the phasic responses of midbrain dopamine neurons show a remarkable similarity to a type of teaching signal (temporal difference (TD) error) used in machine learning. However, previous studies failed to observe a key prediction of this algorithm: that when an agent associates a cue and a reward that are separated in time, the timing of dopamine signals should gradually move backward in time from the time of the reward to the time of the cue over multiple trials. Here we demonstrate that such a gradual shift occurs both at the level of dopaminergic cellular activity and dopamine release in the ventral striatum in mice. Our results establish a long-sought link between dopaminergic activity and the TD learning algorithm, providing fundamental insights into how the brain associates cues and rewards that are separated in time.
Collapse
Affiliation(s)
- Ryunosuke Amo
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Sara Matias
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Akihiro Yamanaka
- Department of Neuroscience II, Research Institute of Environmental Medicine, Nagoya University, Nagoya, Japan
| | - Kenji F Tanaka
- Division of Brain Sciences, Institute for Advanced Medical Research, Keio University School of Medicine, Tokyo, Japan
| | - Naoshige Uchida
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Mitsuko Watabe-Uchida
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA, USA.
| |
Collapse
|
7
|
Biological constraints on neural network models of cognitive function. Nat Rev Neurosci 2021; 22:488-502. [PMID: 34183826 PMCID: PMC7612527 DOI: 10.1038/s41583-021-00473-5] [Citation(s) in RCA: 36] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/17/2021] [Indexed: 02/06/2023]
Abstract
Neural network models are potential tools for improving our understanding of complex brain functions. To address this goal, these models need to be neurobiologically realistic. However, although neural networks have advanced dramatically in recent years and even achieve human-like performance on complex perceptual and cognitive tasks, their similarity to aspects of brain anatomy and physiology is imperfect. Here, we discuss different types of neural models, including localist, auto-associative, hetero-associative, deep and whole-brain networks, and identify aspects under which their biological plausibility can be improved. These aspects range from the choice of model neurons and of mechanisms of synaptic plasticity and learning to implementation of inhibition and control, along with neuroanatomical properties including areal structure and local and long-range connectivity. We highlight recent advances in developing biologically grounded cognitive theories and in mechanistically explaining, on the basis of these brain-constrained neural models, hitherto unaddressed issues regarding the nature, localization and ontogenetic and phylogenetic development of higher brain functions. In closing, we point to possible future clinical applications of brain-constrained modelling.
Collapse
|
9
|
Mollick JA, Chang LJ, Krishnan A, Hazy TE, Krueger KA, Frank GKW, Wager TD, O'Reilly RC. The Neural Correlates of Cued Reward Omission. Front Hum Neurosci 2021; 15:615313. [PMID: 33679345 PMCID: PMC7928384 DOI: 10.3389/fnhum.2021.615313] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2020] [Accepted: 01/19/2021] [Indexed: 11/13/2022] Open
Abstract
Compared to our understanding of positive prediction error signals occurring due to unexpected reward outcomes, less is known about the neural circuitry in humans that drives negative prediction errors during omission of expected rewards. While classical learning theories such as Rescorla-Wagner or temporal difference learning suggest that both types of prediction errors result from a simple subtraction, there has been recent evidence suggesting that different brain regions provide input to dopamine neurons which contributes to specific components of this prediction error computation. Here, we focus on the brain regions responding to negative prediction error signals, which has been well-established in animal studies to involve a distinct pathway through the lateral habenula. We examine the activity of this pathway in humans, using a conditioned inhibition paradigm with high-resolution functional MRI. First, participants learned to associate a sensory stimulus with reward delivery. Then, reward delivery was omitted whenever this stimulus was presented simultaneously with a different sensory stimulus, the conditioned inhibitor (CI). Both reward presentation and the reward-predictive cue activated midbrain dopamine regions, insula and orbitofrontal cortex. While we found significant activity at an uncorrected threshold for the CI in the habenula, consistent with our predictions, it did not survive correction for multiple comparisons and awaits further replication. Additionally, the pallidum and putamen regions of the basal ganglia showed modulations of activity for the inhibitor that did not survive the corrected threshold.
Collapse
Affiliation(s)
- Jessica A Mollick
- Department of Psychiatry, Yale University, New Haven, CT, United States
| | - Luke J Chang
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, United States
| | - Anjali Krishnan
- Department of Psychology, Brooklyn College, City University of New York, Brooklyn, NY, United States
| | | | | | - Guido K W Frank
- UCSD Eating Disorder Center for Treatment and Research, University of California, San Diego, San Diego, CA, United States
| | - Tor D Wager
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, United States
| | - Randall C O'Reilly
- Department of Psychology and Computer Science Center for Neuroscience, University of California, Davis, Davis, CA, United States
| |
Collapse
|