1
|
Engel L, Wolff AR, Blake M, Collins VL, Sinha S, Saunders BT. Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning. Curr Biol 2024; 34:3086-3101.e4. [PMID: 38925117 PMCID: PMC11279555 DOI: 10.1016/j.cub.2024.05.069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2024] [Revised: 04/25/2024] [Accepted: 05/29/2024] [Indexed: 06/28/2024]
Abstract
Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward rewards. Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) are crucial for this process, via engagement of a reciprocally connected network with their striatal targets. Critically, it remains unknown how dopamine neuron activity itself engages dopamine signals throughout the striatum, across learning. Here, we investigated how optogenetic Pavlovian cue conditioning of VTA or SNc dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. Although VTA stimulation-evoked robust dopamine release in NAc core, shell, and DMS, predictive cues preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS. Despite the development of vigorous movement late in training, conditioned dopamine signals did not emerge in the DLS, even during Pavlovian conditioning with SNc dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show a broad dissociation in the fundamental prediction and reward-related information generated by VTA and SNc dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale adaptations across the striatal network emerge during learning to coordinate behavior.
Collapse
Affiliation(s)
- Liv Engel
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA; Medical Discovery Team on Addiction, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA
| | - Amy R Wolff
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA; Medical Discovery Team on Addiction, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA
| | - Madelyn Blake
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA
| | - Val L Collins
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA; Medical Discovery Team on Addiction, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA
| | - Sonal Sinha
- Krieger School of Arts & Sciences, Johns Hopkins University, 3400 N. Charles St, Baltimore, MD 21218, USA
| | - Benjamin T Saunders
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA; Medical Discovery Team on Addiction, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA.
| |
Collapse
|
2
|
Blum K, Bowirrat A, Sunder K, Thanos PK, Hanna C, Gold MS, Dennen CA, Elman I, Murphy KT, Makale MT. Dopamine Dysregulation in Reward and Autism Spectrum Disorder. Brain Sci 2024; 14:733. [PMID: 39061473 PMCID: PMC11274922 DOI: 10.3390/brainsci14070733] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2024] [Revised: 07/09/2024] [Accepted: 07/12/2024] [Indexed: 07/28/2024] Open
Abstract
Autism spectrum disorder (ASD) is primarily characterized by core deficits in social skills, communication, and cognition and by repetitive stereotyped behaviors. These manifestations are variable between individuals, and ASD pathogenesis is complex, with over a thousand implicated genes, many epigenetic factors, and multiple environmental influences. The mesolimbic dopamine (DA) mediated brain reward system is held to play a key role, but the rapidly expanding literature reveals intricate, nuanced signaling involving a wide array of mesolimbic loci, neurotransmitters and receptor subtypes, and neuronal variants. How altered DA signaling may constitute a downstream convergence of the manifold causal origins of ASD is not well understood. A clear working framework of ASD pathogenesis may help delineate common stages and potential diagnostic and interventional opportunities. Hence, we summarize the known natural history of ASD in the context of emerging data and perspectives to update ASD reward signaling. Then, against this backdrop, we proffer a provisional framework that organizes ASD pathogenesis into successive levels, including (1) genetic and epigenetic changes, (2) disrupted mesolimbic reward signaling pathways, (3) dysregulated neurotransmitter/DA signaling, and finally, (4) altered neurocognitive and social behavior and possible antagonist/agonist based ASD interventions. This subdivision of ASD into a logical progression of potentially addressable parts may help facilitate the rational formulation of diagnostics and targeted treatments.
Collapse
Affiliation(s)
- Kenneth Blum
- Division of Addiction Research & Education, Center for Exercise Sports, Mental Health, Western University of Health Sciences, Pomona, CA 91766, USA
- Sunder Foundation, Palm Springs, CA 92264, USA
- Division of Personalized Neuromodulations, PeakLogic, LLC, Del Mar, CA 92130, USA
| | - Abdalla Bowirrat
- Department of Molecular Biology, Adelson School of Medicine, Ariel University, Ariel 40700, Israel
| | | | - Panayotis K. Thanos
- Department of Pharmacology and Toxicology, State University of New York, SUNY, Buffalo, NY 14215, USA
| | - Colin Hanna
- Department of Pharmacology and Toxicology, State University of New York, SUNY, Buffalo, NY 14215, USA
| | - Mark S. Gold
- Department of Psychiatry, Washington University School of Medicine, St. Louis, MO 63110, USA
| | - Catherine A. Dennen
- Department of Family Medicine, Jefferson Health Northeast, Philadelphia, PA 19145, USA
| | - Igor Elman
- Department of Psychiatry, Harvard University School of Medicine, Cambridge, MA 02215, USA
| | - Kevin T. Murphy
- Division of Personalized Neuromodulations, PeakLogic, LLC, Del Mar, CA 92130, USA
| | - Milan T. Makale
- Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA 92093, USA
| |
Collapse
|
3
|
Millard SJ, Hoang IB, Sherwood S, Taira M, Reyes V, Greer Z, O'Connor SL, Wassum KM, James MH, Barker DJ, Sharpe MJ. Cognitive representations of intracranial self-stimulation of midbrain dopamine neurons depend on stimulation frequency. Nat Neurosci 2024; 27:1253-1259. [PMID: 38741021 PMCID: PMC11239488 DOI: 10.1038/s41593-024-01643-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Accepted: 04/05/2024] [Indexed: 05/16/2024]
Abstract
Dopamine neurons in the ventral tegmental area support intracranial self-stimulation (ICSS), yet the cognitive representations underlying this phenomenon remain unclear. Here, 20-Hz stimulation of dopamine neurons, which approximates a physiologically relevant prediction error, was not sufficient to support ICSS beyond a continuously reinforced schedule and did not endow cues with a general or specific value. However, 50-Hz stimulation of dopamine neurons was sufficient to drive robust ICSS and was represented as a specific reward to motivate behavior. The frequency dependence of this effect is due to the rate (not the number) of action potentials produced by dopamine neurons, which differently modulates dopamine release downstream.
Collapse
Affiliation(s)
- Samuel J Millard
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Ivy B Hoang
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Savannah Sherwood
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Masakazu Taira
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
- Department of Psychology, University of Sydney, Camperdown, New South Wales, Australia
| | - Vanessa Reyes
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Zara Greer
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Shayna L O'Connor
- Department of Psychiatry, Robert Wood Johnson Medical School, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
- Brain Health Institute, Rutgers Biomedical Health Sciences, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
- Department of Psychology, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
| | - Kate M Wassum
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Morgan H James
- Department of Psychiatry, Robert Wood Johnson Medical School, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
- Brain Health Institute, Rutgers Biomedical Health Sciences, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
| | - David J Barker
- Brain Health Institute, Rutgers Biomedical Health Sciences, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
- Department of Psychology, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
| | - Melissa J Sharpe
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA.
- Department of Psychology, University of Sydney, Camperdown, New South Wales, Australia.
| |
Collapse
|
4
|
Song MR, Lee SW. Rethinking dopamine-guided action sequence learning. Eur J Neurosci 2024; 60:3447-3465. [PMID: 38798086 DOI: 10.1111/ejn.16426] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Revised: 04/21/2024] [Accepted: 05/08/2024] [Indexed: 05/29/2024]
Abstract
As opposed to those requiring a single action for reward acquisition, tasks necessitating action sequences demand that animals learn action elements and their sequential order and sustain the behaviour until the sequence is completed. With repeated learning, animals not only exhibit precise execution of these sequences but also demonstrate enhanced smoothness and efficiency. Previous research has demonstrated that midbrain dopamine and its major projection target, the striatum, play crucial roles in these processes. Recent studies have shown that dopamine from the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA) serve distinct functions in action sequence learning. The distinct contributions of dopamine also depend on the striatal subregions, namely the ventral, dorsomedial and dorsolateral striatum. Here, we have reviewed recent findings on the role of striatal dopamine in action sequence learning, with a focus on recent rodent studies.
Collapse
Affiliation(s)
- Minryung R Song
- Department of Brain and Cognitive Sciences, KAIST, Daejeon, South Korea
| | - Sang Wan Lee
- Department of Brain and Cognitive Sciences, KAIST, Daejeon, South Korea
- Kim Jaechul Graduate School of AI, KAIST, Daejeon, South Korea
- KI for Health Science and Technology, KAIST, Daejeon, South Korea
- Center for Neuroscience-inspired AI, KAIST, Daejeon, South Korea
| |
Collapse
|
5
|
Faget L, Oriol L, Lee WC, Zell V, Sargent C, Flores A, Hollon NG, Ramanathan D, Hnasko TS. Ventral pallidum GABA and glutamate neurons drive approach and avoidance through distinct modulation of VTA cell types. Nat Commun 2024; 15:4233. [PMID: 38762463 PMCID: PMC11102457 DOI: 10.1038/s41467-024-48340-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 04/26/2024] [Indexed: 05/20/2024] Open
Abstract
The ventral pallidum (VP) contains GABA and glutamate neurons projecting to ventral tegmental area (VTA) whose stimulation drives approach and avoidance, respectively. Yet little is known about the mechanisms by which VP cell types shape VTA activity and drive behavior. Here, we found that both VP GABA and glutamate neurons were activated during approach to reward or by delivery of an aversive stimulus. Stimulation of VP GABA neurons inhibited VTA GABA, but activated dopamine and glutamate neurons. Remarkably, stimulation-evoked activation was behavior-contingent such that VTA recruitment was inhibited when evoked by the subject's own action. Conversely, VP glutamate neurons activated VTA GABA, as well as dopamine and glutamate neurons, despite driving aversion. However, VP glutamate neurons evoked dopamine in aversion-associated ventromedial nucleus accumbens (NAc), but reduced dopamine release in reward-associated dorsomedial NAc. These findings show how heterogeneous VP projections to VTA can be engaged to shape approach and avoidance behaviors.
Collapse
Affiliation(s)
- Lauren Faget
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA.
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA, USA.
| | - Lucie Oriol
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
| | - Wen-Chun Lee
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
| | - Vivien Zell
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
| | - Cody Sargent
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
| | - Andrew Flores
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA, USA
| | - Nick G Hollon
- Department of Psychiatry, University of California, San Diego, La Jolla, CA, USA
| | - Dhakshin Ramanathan
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA, USA
- Department of Psychiatry, University of California, San Diego, La Jolla, CA, USA
| | - Thomas S Hnasko
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA.
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA, USA.
| |
Collapse
|
6
|
Hart G, Burton TJ, Balleine BW. What Role Does Striatal Dopamine Play in Goal-directed Action? Neuroscience 2024; 546:20-32. [PMID: 38521480 DOI: 10.1016/j.neuroscience.2024.03.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Revised: 02/15/2024] [Accepted: 03/18/2024] [Indexed: 03/25/2024]
Abstract
Evidence suggests that dopamine activity provides a US-related prediction error for Pavlovian conditioning and the reinforcement signal supporting the acquisition of habits. However, its role in goal-directed action is less clear. There are currently few studies that have assessed dopamine release as animals acquire and perform self-paced instrumental actions. Here we briefly review the literature documenting the psychological, behavioral and neural bases of goal-directed actions in rats and mice, before turning to describe recent studies investigating the role of dopamine in instrumental learning and performance. Plasticity in dorsomedial striatum, a central node in the network supporting goal-directed action, clearly requires dopamine release, the timing of which, relative to cortical and thalamic inputs, determines the degree and form of that plasticity. Beyond this, bilateral release appears to reflect reward prediction errors as animals experience the consequences of an action. Such signals feedforward to update the value of the specific action associated with that outcome during subsequent performance, with dopamine release at the time of action reflecting the updated predicted action value. More recently, evidence has also emerged for a hemispherically lateralised signal associated with the action; dopamine release is greater in the hemisphere contralateral to the spatial target of the action. This effect emerges over the course of acquisition and appears to reflect the strength of the action-outcome association. Thus, during goal-directed action, dopamine release signals the action, the outcome and their association to shape the learning and performance processes necessary to support this form of behavioral control.
Collapse
Affiliation(s)
- Genevra Hart
- Decision Neuroscience Lab, UNSW Sydney, Australia
| | | | | |
Collapse
|
7
|
Hathaway BA, Li A, Brodie HG, Silveira MM, Tremblay M, Seo YS, Winstanley CA. Dopamine activity in the nigrostriatal pathway alters cue-induced risky choice patterns in female rats. Eur J Neurosci 2024; 59:1621-1637. [PMID: 38369911 DOI: 10.1111/ejn.16287] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 01/18/2024] [Accepted: 02/03/2024] [Indexed: 02/20/2024]
Abstract
Deficits in cost/benefit decision making is a critical risk factor for gambling disorder. Reward-paired cues may play an important role, as these stimuli can enhance risk preference in rats. Despite extensive research implicating the dorsal striatum in the compulsive aspects of addiction, the role of nigrostriatal dopaminergic activity in cue-induced risk preference remains unclear, particularly in females. Accordingly, we examined the effects of manipulating the dopaminergic nigrostriatal pathway on cue-induced risky choice in female rats. TH:Cre rats were trained on the cued version of the rat Gambling Task. This task was designed such that maximal reward is attained by avoiding the high-risk, high-reward options and instead favouring the options associated with lower per-trial gains, as they feature less frequent and shorter time-out penalties. Adding reward-paired audiovisual cues to the task leads to greater risky choice on average. To assess the role of the nigrostriatal pathway, a viral vector carrying either Cre-dependent inhibitory or excitatory DREADD was infused into the substantia nigra. Rats then received clozapine-N-oxide either during task acquisition or after a stable performance baseline was reached. Inhibition of this pathway accelerated the development of risk preference in early sessions and increased risky choice during performance, but long-term inhibition actually improved decision making. Activation of this pathway had minimal effects. These results provide evidence for the involvement of the dopaminergic nigrostriatal pathway in cue-induced risk preference in females, therefore shedding light on its role in cost/benefit decision-making deficits and expanding our knowledge of the female dopaminergic system.
Collapse
Affiliation(s)
- Brett A Hathaway
- Department of Psychology, Djavad Mowafaghian Centre for Brain Health, University of British Columbia, Vancouver, Canada
| | - Andrew Li
- Department of Psychology, Djavad Mowafaghian Centre for Brain Health, University of British Columbia, Vancouver, Canada
| | - Hannah G Brodie
- Department of Psychology, Djavad Mowafaghian Centre for Brain Health, University of British Columbia, Vancouver, Canada
| | - Mason M Silveira
- Department of Psychology, Djavad Mowafaghian Centre for Brain Health, University of British Columbia, Vancouver, Canada
| | - Melanie Tremblay
- Department of Psychology, Djavad Mowafaghian Centre for Brain Health, University of British Columbia, Vancouver, Canada
| | - Yeon Soo Seo
- Department of Psychology, Djavad Mowafaghian Centre for Brain Health, University of British Columbia, Vancouver, Canada
| | - Catharine A Winstanley
- Department of Psychology, Djavad Mowafaghian Centre for Brain Health, University of British Columbia, Vancouver, Canada
| |
Collapse
|
8
|
Engel L, Wolff AR, Blake M, Collins VL, Sinha S, Saunders BT. Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.07.01.547331. [PMID: 38585717 PMCID: PMC10996462 DOI: 10.1101/2023.07.01.547331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/09/2024]
Abstract
Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward acquisition of rewards. Dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SNC) are crucial for this process. Dopamine neurons are embedded in a reciprocally connected network with their striatal targets, the functional organization of which remains poorly understood. Here, we investigated how learning during optogenetic Pavlovian cue conditioning of VTA or SNC dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent dopamine biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. While VTA stimulation evoked robust dopamine release in NAc core, shell, and DMS, cues predictive of this activation preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Corresponding negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS, and not the shell. Despite development of vigorous movement late in training, conditioned dopamine signals did not similarly emerge in the DLS, even during Pavlovian conditioning with SNC dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show broad dissociation in the fundamental prediction and reward-related information generated by different dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale plasticity across the striatal network emerges during Pavlovian learning to coordinate behavior.
Collapse
Affiliation(s)
- Liv Engel
- Department of Neuroscience, University of Minnesota
- Medical Discovery Team on Addiction, University of Minnesota
- Current Address: Department of Psychology, University of Toronto
| | - Amy R Wolff
- Department of Neuroscience, University of Minnesota
- Medical Discovery Team on Addiction, University of Minnesota
| | - Madelyn Blake
- Department of Neuroscience, University of Minnesota
- Medical Discovery Team on Addiction, University of Minnesota
| | - Val L Collins
- Department of Neuroscience, University of Minnesota
- Medical Discovery Team on Addiction, University of Minnesota
| | | | - Benjamin T Saunders
- Department of Neuroscience, University of Minnesota
- Medical Discovery Team on Addiction, University of Minnesota
| |
Collapse
|
9
|
Carter F, Cossette MP, Trujillo-Pisanty I, Pallikaras V, Breton YA, Conover K, Caplan J, Solis P, Voisard J, Yaksich A, Shizgal P. Does phasic dopamine release cause policy updates? Eur J Neurosci 2024; 59:1260-1277. [PMID: 38039083 DOI: 10.1111/ejn.16199] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2023] [Revised: 10/21/2023] [Accepted: 11/01/2023] [Indexed: 12/03/2023]
Abstract
Phasic dopamine activity is believed to both encode reward-prediction errors (RPEs) and to cause the adaptations that these errors engender. If so, a rat working for optogenetic stimulation of dopamine neurons will repeatedly update its policy and/or action values, thus iteratively increasing its work rate. Here, we challenge this view by demonstrating stable, non-maximal work rates in the face of repeated optogenetic stimulation of midbrain dopamine neurons. Furthermore, we show that rats learn to discriminate between world states distinguished only by their history of dopamine activation. Comparison of these results to reinforcement learning simulations suggests that the induced dopamine transients acted more as rewards than RPEs. However, pursuit of dopaminergic stimulation drifted upwards over a time scale of days and weeks, despite its stability within trials. To reconcile the results with prior findings, we consider multiple roles for dopamine signalling.
Collapse
Affiliation(s)
- Francis Carter
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
- Montreal Institute for Learning Algorithms, Université de Montréal, Montreal, Quebec, Canada
| | | | - Ivan Trujillo-Pisanty
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
- Department of Psychology, Langara College, Vancouver, British Columbia, Canada
| | | | | | - Kent Conover
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
| | - Jill Caplan
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
| | - Pavel Solis
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
| | - Jacques Voisard
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
| | - Alexandra Yaksich
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
| | - Peter Shizgal
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
| |
Collapse
|
10
|
Wolff AR, Saunders BT. Sensory Cues Potentiate VTA Dopamine Mediated Reinforcement. eNeuro 2024; 11:ENEURO.0421-23.2024. [PMID: 38238080 PMCID: PMC10875637 DOI: 10.1523/eneuro.0421-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Revised: 12/19/2023] [Accepted: 01/04/2024] [Indexed: 01/25/2024] Open
Abstract
Sensory cues are critical for shaping decisions and invigorating actions during reward seeking. Dopamine neurons in the ventral tegmental area (VTA) are central in this process, supporting associative learning in Pavlovian and instrumental settings. Studies of intracranial self-stimulation (ICSS) behavior, which show that animals will work hard to receive stimulation of dopamine neurons, support the notion that dopamine transmits a reward or value signal to support learning. Recent studies have begun to question this, however, emphasizing dopamine's value-free functions, leaving its contribution to behavioral reinforcement somewhat muddled. Here, we investigated the role of sensory stimuli in dopamine-mediated reinforcement, using an optogenetic ICSS paradigm in tyrosine hydroxylase (TH)-Cre rats. We find that while VTA dopamine neuron activation in the absence of explicit external cues is sufficient to maintain robust self-stimulation, the presence of cues dramatically potentiates ICSS behavior. Our results support a framework where dopamine can have some base value as a reinforcer, but the impact of this signal is modulated heavily by the sensory learning context.
Collapse
Affiliation(s)
- Amy R Wolff
- Department of Neuroscience, University of Minnesota, Minneapolis 55455, Minnesota
- Medical Discovery Team on Addiction, University of Minnesota, Minneapolis 55455, Minnesota
| | - Benjamin T Saunders
- Department of Neuroscience, University of Minnesota, Minneapolis 55455, Minnesota
- Medical Discovery Team on Addiction, University of Minnesota, Minneapolis 55455, Minnesota
| |
Collapse
|
11
|
Fraser KM, Chen BJ, Janak PH. Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance. Eur J Neurosci 2024; 59:220-237. [PMID: 38093522 PMCID: PMC10841748 DOI: 10.1111/ejn.16210] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2023] [Revised: 11/09/2023] [Accepted: 11/15/2023] [Indexed: 01/23/2024]
Abstract
Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.
Collapse
Affiliation(s)
- Kurt M. Fraser
- Department of Psychological & Brain Sciences, Krieger School of Arts & Sciences, Johns Hopkins University, Baltimore, MD, 21218
| | - Bridget J. Chen
- Department of Psychological & Brain Sciences, Krieger School of Arts & Sciences, Johns Hopkins University, Baltimore, MD, 21218
| | - Patricia H. Janak
- Department of Psychological & Brain Sciences, Krieger School of Arts & Sciences, Johns Hopkins University, Baltimore, MD, 21218
- Solomon H. Snyder Department of Neuroscience, School of Medicine, Johns Hopkins University, Baltimore, MD, 21205
- Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD 21218
| |
Collapse
|
12
|
van der Merwe R, Nadel J, Copes-Finke D, Pawelko S, Scott J, Ghanem M, Fox M, Morehouse C, McLaughlin R, Maddox C, Albert-Lyons R, Malaki G, Groce V, Turocy A, Aggadi N, Jin X, Howard C. Characterization of striatal dopamine projections across striatal subregions in behavioral flexibility. Eur J Neurosci 2023; 58:4466-4486. [PMID: 36617434 PMCID: PMC10329096 DOI: 10.1111/ejn.15910] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2021] [Revised: 12/13/2022] [Accepted: 12/30/2022] [Indexed: 01/10/2023]
Abstract
Behavioural flexibility is key to survival in a dynamic environmentWhile flexible, goal-directed behaviours are initially dependent on dorsomedial striatum, they become dependent on lateral striatum as behaviours become inflexible. Similarly, lesions of dopamine terminals in lateral striatum disrupt the development of inflexible habits. This work suggests that dopamine release in lateral striatum may drive inflexible behaviours, though few studies have investigated a causative role of subpopulations of striatal dopamine terminals in reversal learning, a measure of flexibility. Here, we performed two optogenetic experiments to activate dopamine terminals in dorsomedial (DMS), dorsolateral (DLS) or ventral (nucleus accumbens [NAc]) striatum in DAT-Cre mice that expressed channelrhodopsin-2 via viral injection (Experiment I) or through transgenic breeding with an Ai32 reporter line (Experiment II) to determine how specific dopamine subpopulations impact reversal learning. Mice performed a reversal task in which they self-stimulated DMS, DLS, or NAc dopamine terminals by pressing one of two levers before action-outcome lever contingencies were reversed. Largely consistent with presumed ventromedial/lateral striatal function, we found that mice self-stimulating medial dopamine terminals reversed lever preference following contingency reversal, while mice self-stimulating NAc showed parial flexibility, and DLS self-stimulation resulted in impaired reversal. Impairments in DLS mice were characterized by more regressive errors and reliance on lose-stay strategies following reversal, as well as reduced within-session learning, suggesting reward insensitivity and overreliance on previously learned actions. This study supports a model of striatal function in which DMS and ventral dopamine facilitate goal-directed responding, and DLS dopamine supports more inflexible responding.
Collapse
Affiliation(s)
- R.K. van der Merwe
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - J.A. Nadel
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
- Northwestern University Interdepartmental Neuroscience Program (NUIN), Evanston, IL, USA
| | - D. Copes-Finke
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - S. Pawelko
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - J.S. Scott
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - M. Ghanem
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - M. Fox
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - C. Morehouse
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - R. McLaughlin
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - C. Maddox
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - R. Albert-Lyons
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - G. Malaki
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - V. Groce
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - A. Turocy
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - N. Aggadi
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| | - X. Jin
- Center for Motor Control and Disease, Key Laboratory of Brain Functional Genomics, East China Normal University, Shanghai 200062, China
- NYU–ECNU Institute of Brain and Cognitive Science, New York University Shanghai, Shanghai 200062, China
| | - C.D. Howard
- Neuroscience Department, Oberlin College, 173 Lorain St., Oberlin, OH, USA
| |
Collapse
|
13
|
Conn KA, Alexander S, Burne THJ, Kesby JP. Antagonism of D2 receptors via raclopride ameliorates amphetamine-induced associative learning deficits in male mice. Behav Brain Res 2023; 454:114649. [PMID: 37643667 DOI: 10.1016/j.bbr.2023.114649] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 08/09/2023] [Accepted: 08/25/2023] [Indexed: 08/31/2023]
Abstract
Dopamine levels in the dorsomedial striatum (DMS) are highly dynamic and are thought to underly the encoding of action-outcome associations. Although it is known that amphetamine disrupts the learning that is required for goal-directed action, the role of D1 and D2 receptors in this process has not been established. In this study, we examined the role of D1 and D2 receptor antagonists on learning in response to amphetamine. We used the outcome-specific devaluation task to examine goal-directed action in male C57BL6/J mice treated systemically with either a D1 antagonist (SCH-23990; 0.01 mg/kg) or a D2 antagonist (raclopride; 0.5 mg/kg) and then administered amphetamine (1 mg/kg). The mice were injected repeatedly throughout the instrumental training phase of the task to assess the impact on the learning of action-outcomes, and the subsequent choice test assessing performance of goal-directed action was conducted drug free. Effects of chronic drug administration on locomotor behaviour was assessed before and after the choice test. Treatment during learning with either amphetamine, or the D1 or D2 antagonists, impaired the subsequent performance of goal-directed action. The amphetamine-induced impairment in goal-directed action was reversed in mice treated with raclopride, but not when treated with SCH-23990. By contrast, amphetamine-induced hyperactivity was reversed in mice treated with SCH-23990, but not in mice treated with raclopride. Taken together, these data support the role of a balance of dopamine receptor signalling after amphetamine treatment. While overall D1 receptor availability is necessary to promote learning, in a state of elevated dopamine, modifying D2 receptor function can ameliorate learning deficits.
Collapse
Affiliation(s)
- Kyna-Anne Conn
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia
| | - Suzy Alexander
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia
| | - Thomas H J Burne
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia; Queensland Centre for Mental Health Research, Wacol, QLD 4076, Australia
| | - James P Kesby
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia; Queensland Centre for Mental Health Research, Wacol, QLD 4076, Australia.
| |
Collapse
|
14
|
Faget L, Oriol L, Lee WC, Sargent C, Ramanathan D, Hnasko TS. Ventral pallidum GABA and glutamate neurons drive approach and avoidance through distinct modulation of VTA cell types. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.07.12.548750. [PMID: 37502884 PMCID: PMC10369949 DOI: 10.1101/2023.07.12.548750] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/29/2023]
Abstract
The ventral pallidum (VP) contains GABA and glutamate (Glut) neurons projecting to ventral tegmental area (VTA) whose stimulation drives approach and avoidance, respectively. Yet little is known about the cell-type-specific mechanisms by which VP projections to VTA drive behavior. Here, we found that both VP GABA and Glut neurons were activated during approach to reward or delivery of an aversive stimulus. Stimulation of VP GABA neurons inhibited VTA GABA, but activated dopamine (DA) and glutamate neurons. Remarkably, this cell-type-specific recruitment was behavior-contingent such that VTA recruitment was inhibited when evoked by the subject's own action. Conversely, VP Glut neurons activated VTA GABA, as well as DA and Glut neurons, despite driving aversion. However, VP Glut neurons evoked DA in reward-associated ventromedial nucleus accumbens (NAc), but reduced DA in aversion-associated dorsomedial NAc. These findings show how heterogeneous VP cell types can engage VTA cell types to shape approach and avoidance behaviors.
Collapse
Affiliation(s)
- Lauren Faget
- Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA 92161, USA
| | - Lucie Oriol
- Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA
| | - Wen-Chun Lee
- Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA
| | - Cody Sargent
- Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA
| | - Dhakshin Ramanathan
- Department of Psychiatry, University of California, San Diego, La Jolla, CA 92093, USA
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA 92161, USA
| | - Thomas S. Hnasko
- Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA 92161, USA
| |
Collapse
|
15
|
Fraser KM, Pribut HJ, Janak PH, Keiflin R. From Prediction to Action: Dissociable Roles of Ventral Tegmental Area and Substantia Nigra Dopamine Neurons in Instrumental Reinforcement. J Neurosci 2023; 43:3895-3908. [PMID: 37185097 PMCID: PMC10217998 DOI: 10.1523/jneurosci.0028-23.2023] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2023] [Revised: 04/06/2023] [Accepted: 04/14/2023] [Indexed: 05/17/2023] Open
Abstract
Reward seeking requires the coordination of motor programs to achieve goals. Midbrain dopamine neurons are critical for reinforcement, and their activation is sufficient for learning about cues, actions, and outcomes. Here we examine in detail the mechanisms underlying the ability of ventral tegmental area (VTA) and substantia nigra (SNc) dopamine neurons to support instrumental learning. By exploiting numerous behavioral tasks in combination with time-limited optogenetic manipulations in male and female rats, we reveal that VTA and SNc dopamine neurons generate reinforcement through separable psychological processes. VTA dopamine neurons imbue actions and their associated cues with motivational value that allows flexible and persistent pursuit, whereas SNc dopamine neurons support time-limited, precise, action-specific learning that is nonscalable and inflexible. This architecture is reminiscent of actor-critic reinforcement learning models with VTA and SNc instructing the critic and actor, respectively. Our findings indicate that heterogeneous dopamine systems support unique forms of instrumental learning that ultimately result in disparate reward-seeking strategies.SIGNIFICANCE STATEMENT Dopamine neurons in the midbrain are essential for learning, motivation, and movement. Here we describe in detail the ability of VTA and SNc dopamine neurons to generate instrumental reinforcement, a process where an agent learns about actions they can emit to earn reward. While rats will avidly work and learn to respond for activation of VTA and SNc dopamine neurons, we find that only VTA dopamine neurons imbue actions and their associated cues with motivational value that spur continued pursuit of reward. Our data support a hypothesis that VTA and SNc dopamine neurons engage distinct psychological processes that have consequences for our understanding of these neurons in health and disease.
Collapse
Affiliation(s)
- Kurt M Fraser
- Department of Psychological & Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, Maryland 21218
| | - Heather J Pribut
- Department of Psychological & Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, Maryland 21218
| | - Patricia H Janak
- Department of Psychological & Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, Maryland 21218
- The Solomon H. Snyder Department of Neuroscience, Johns Hopkins School of Medicine, Johns Hopkins University, Baltimore, Maryland 21205
- Kavli Neuroscience Discovery Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland 21205
| | - Ronald Keiflin
- Department of Psychological & Brain Sciences, University of California, Santa Barbara, Santa Barbara, California 93106
- Neuroscience Research Institute, University of California Santa Barbara, Santa Barbara, California, 93106
| |
Collapse
|
16
|
Mesolimbic dopamine adapts the rate of learning from action. Nature 2023; 614:294-302. [PMID: 36653450 PMCID: PMC9908546 DOI: 10.1038/s41586-022-05614-z] [Citation(s) in RCA: 27] [Impact Index Per Article: 27.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2021] [Accepted: 11/30/2022] [Indexed: 01/20/2023]
Abstract
Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions1-3. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction4; however, so far there has been little consideration of how direct policy learning might inform our understanding5. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning6.
Collapse
|
17
|
Jeong H, Taylor A, Floeder JR, Lohmann M, Mihalas S, Wu B, Zhou M, Burke DA, Namboodiri VMK. Mesolimbic dopamine release conveys causal associations. Science 2022; 378:eabq6740. [PMID: 36480599 PMCID: PMC9910357 DOI: 10.1126/science.abq6740] [Citation(s) in RCA: 55] [Impact Index Per Article: 27.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
Learning to predict rewards based on environmental cues is essential for survival. It is believed that animals learn to predict rewards by updating predictions whenever the outcome deviates from expectations, and that such reward prediction errors (RPEs) are signaled by the mesolimbic dopamine system-a key controller of learning. However, instead of learning prospective predictions from RPEs, animals can infer predictions by learning the retrospective cause of rewards. Hence, whether mesolimbic dopamine instead conveys a causal associative signal that sometimes resembles RPE remains unknown. We developed an algorithm for retrospective causal learning and found that mesolimbic dopamine release conveys causal associations but not RPE, thereby challenging the dominant theory of reward learning. Our results reshape the conceptual and biological framework for associative learning.
Collapse
Affiliation(s)
- Huijeong Jeong
- Department of Neurology, University of California, San Francisco, CA, USA
| | - Annie Taylor
- Neuroscience Graduate Program, University of California, San Francisco, CA, USA
| | - Joseph R Floeder
- Neuroscience Graduate Program, University of California, San Francisco, CA, USA
| | | | - Stefan Mihalas
- Allen Institute for Brain Science, Seattle, WA, USA
- Department of Applied Mathematics, University of Washington, Seattle, WA, USA
| | - Brenda Wu
- Department of Neurology, University of California, San Francisco, CA, USA
| | - Mingkang Zhou
- Department of Neurology, University of California, San Francisco, CA, USA
- Neuroscience Graduate Program, University of California, San Francisco, CA, USA
| | - Dennis A Burke
- Department of Neurology, University of California, San Francisco, CA, USA
| | - Vijay Mohan K Namboodiri
- Department of Neurology, University of California, San Francisco, CA, USA
- Neuroscience Graduate Program, University of California, San Francisco, CA, USA
- Weill Institute for Neuroscience, Kavli Institute for Fundamental Neuroscience, Center for Integrative Neuroscience, University of California, San Francisco, CA, USA
| |
Collapse
|