1
|
Blum K, Bowirrat A, Sunder K, Thanos PK, Hanna C, Gold MS, Dennen CA, Elman I, Murphy KT, Makale MT. Dopamine Dysregulation in Reward and Autism Spectrum Disorder. Brain Sci 2024; 14:733. [PMID: 39061473 PMCID: PMC11274922 DOI: 10.3390/brainsci14070733] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2024] [Revised: 07/09/2024] [Accepted: 07/12/2024] [Indexed: 07/28/2024] Open
Abstract
Autism spectrum disorder (ASD) is primarily characterized by core deficits in social skills, communication, and cognition and by repetitive stereotyped behaviors. These manifestations are variable between individuals, and ASD pathogenesis is complex, with over a thousand implicated genes, many epigenetic factors, and multiple environmental influences. The mesolimbic dopamine (DA) mediated brain reward system is held to play a key role, but the rapidly expanding literature reveals intricate, nuanced signaling involving a wide array of mesolimbic loci, neurotransmitters and receptor subtypes, and neuronal variants. How altered DA signaling may constitute a downstream convergence of the manifold causal origins of ASD is not well understood. A clear working framework of ASD pathogenesis may help delineate common stages and potential diagnostic and interventional opportunities. Hence, we summarize the known natural history of ASD in the context of emerging data and perspectives to update ASD reward signaling. Then, against this backdrop, we proffer a provisional framework that organizes ASD pathogenesis into successive levels, including (1) genetic and epigenetic changes, (2) disrupted mesolimbic reward signaling pathways, (3) dysregulated neurotransmitter/DA signaling, and finally, (4) altered neurocognitive and social behavior and possible antagonist/agonist based ASD interventions. This subdivision of ASD into a logical progression of potentially addressable parts may help facilitate the rational formulation of diagnostics and targeted treatments.
Collapse
Affiliation(s)
- Kenneth Blum
- Division of Addiction Research & Education, Center for Exercise Sports, Mental Health, Western University of Health Sciences, Pomona, CA 91766, USA
- Sunder Foundation, Palm Springs, CA 92264, USA
- Division of Personalized Neuromodulations, PeakLogic, LLC, Del Mar, CA 92130, USA
| | - Abdalla Bowirrat
- Department of Molecular Biology, Adelson School of Medicine, Ariel University, Ariel 40700, Israel
| | | | - Panayotis K. Thanos
- Department of Pharmacology and Toxicology, State University of New York, SUNY, Buffalo, NY 14215, USA
| | - Colin Hanna
- Department of Pharmacology and Toxicology, State University of New York, SUNY, Buffalo, NY 14215, USA
| | - Mark S. Gold
- Department of Psychiatry, Washington University School of Medicine, St. Louis, MO 63110, USA
| | - Catherine A. Dennen
- Department of Family Medicine, Jefferson Health Northeast, Philadelphia, PA 19145, USA
| | - Igor Elman
- Department of Psychiatry, Harvard University School of Medicine, Cambridge, MA 02215, USA
| | - Kevin T. Murphy
- Division of Personalized Neuromodulations, PeakLogic, LLC, Del Mar, CA 92130, USA
| | - Milan T. Makale
- Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA 92093, USA
| |
Collapse
|
2
|
Faget L, Oriol L, Lee WC, Zell V, Sargent C, Flores A, Hollon NG, Ramanathan D, Hnasko TS. Ventral pallidum GABA and glutamate neurons drive approach and avoidance through distinct modulation of VTA cell types. Nat Commun 2024; 15:4233. [PMID: 38762463 PMCID: PMC11102457 DOI: 10.1038/s41467-024-48340-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 04/26/2024] [Indexed: 05/20/2024] Open
Abstract
The ventral pallidum (VP) contains GABA and glutamate neurons projecting to ventral tegmental area (VTA) whose stimulation drives approach and avoidance, respectively. Yet little is known about the mechanisms by which VP cell types shape VTA activity and drive behavior. Here, we found that both VP GABA and glutamate neurons were activated during approach to reward or by delivery of an aversive stimulus. Stimulation of VP GABA neurons inhibited VTA GABA, but activated dopamine and glutamate neurons. Remarkably, stimulation-evoked activation was behavior-contingent such that VTA recruitment was inhibited when evoked by the subject's own action. Conversely, VP glutamate neurons activated VTA GABA, as well as dopamine and glutamate neurons, despite driving aversion. However, VP glutamate neurons evoked dopamine in aversion-associated ventromedial nucleus accumbens (NAc), but reduced dopamine release in reward-associated dorsomedial NAc. These findings show how heterogeneous VP projections to VTA can be engaged to shape approach and avoidance behaviors.
Collapse
Affiliation(s)
- Lauren Faget
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA.
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA, USA.
| | - Lucie Oriol
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
| | - Wen-Chun Lee
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
| | - Vivien Zell
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
| | - Cody Sargent
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
| | - Andrew Flores
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA, USA
| | - Nick G Hollon
- Department of Psychiatry, University of California, San Diego, La Jolla, CA, USA
| | - Dhakshin Ramanathan
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA, USA
- Department of Psychiatry, University of California, San Diego, La Jolla, CA, USA
| | - Thomas S Hnasko
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA.
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA, USA.
| |
Collapse
|
3
|
Parnas M, Manoim JE, Lin AC. Sensory encoding and memory in the mushroom body: signals, noise, and variability. Learn Mem 2024; 31:a053825. [PMID: 38862174 PMCID: PMC11199953 DOI: 10.1101/lm.053825.123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2023] [Accepted: 11/21/2023] [Indexed: 06/13/2024]
Abstract
To survive in changing environments, animals need to learn to associate specific sensory stimuli with positive or negative valence. How do they form stimulus-specific memories to distinguish between positively/negatively associated stimuli and other irrelevant stimuli? Solving this task is one of the functions of the mushroom body, the associative memory center in insect brains. Here we summarize recent work on sensory encoding and memory in the Drosophila mushroom body, highlighting general principles such as pattern separation, sparse coding, noise and variability, coincidence detection, and spatially localized neuromodulation, and placing the mushroom body in comparative perspective with mammalian memory systems.
Collapse
Affiliation(s)
- Moshe Parnas
- Department of Physiology and Pharmacology, Faculty of Medicine, Tel Aviv University, Tel Aviv 69978, Israel
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv 69978, Israel
| | - Julia E Manoim
- Department of Physiology and Pharmacology, Faculty of Medicine, Tel Aviv University, Tel Aviv 69978, Israel
| | - Andrew C Lin
- School of Biosciences, University of Sheffield, Sheffield S10 2TN, United Kingdom
- Neuroscience Institute, University of Sheffield, Sheffield S10 2TN, United Kingdom
| |
Collapse
|
4
|
Farnworth MS, Montgomery SH. Evolution of neural circuitry and cognition. Biol Lett 2024; 20:20230576. [PMID: 38747685 DOI: 10.1098/rsbl.2023.0576] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2023] [Accepted: 03/26/2024] [Indexed: 05/25/2024] Open
Abstract
Neural circuits govern the interface between the external environment, internal cues and outwardly directed behaviours. To process multiple environmental stimuli and integrate these with internal state requires considerable neural computation. Expansion in neural network size, most readily represented by whole brain size, has historically been linked to behavioural complexity, or the predominance of cognitive behaviours. Yet, it is largely unclear which aspects of circuit variation impact variation in performance. A key question in the field of evolutionary neurobiology is therefore how neural circuits evolve to allow improved behavioural performance or innovation. We discuss this question by first exploring how volumetric changes in brain areas reflect actual neural circuit change. We explore three major axes of neural circuit evolution-replication, restructuring and reconditioning of cells and circuits-and discuss how these could relate to broader phenotypes and behavioural variation. This discussion touches on the relevant uses and limitations of volumetrics, while advocating a more circuit-based view of cognition. We then use this framework to showcase an example from the insect brain, the multi-sensory integration and internal processing that is shared between the mushroom bodies and central complex. We end by identifying future trends in this research area, which promise to advance the field of evolutionary neurobiology.
Collapse
Affiliation(s)
- Max S Farnworth
- School of Biological Sciences, University of Bristol , Bristol, UK
| | | |
Collapse
|
5
|
Lowet AS, Zheng Q, Meng M, Matias S, Drugowitsch J, Uchida N. An opponent striatal circuit for distributional reinforcement learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.01.02.573966. [PMID: 38260354 PMCID: PMC10802299 DOI: 10.1101/2024.01.02.573966] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/24/2024]
Abstract
Machine learning research has achieved large performance gains on a wide range of tasks by expanding the learning target from mean rewards to entire probability distributions of rewards - an approach known as distributional reinforcement learning (RL)1. The mesolimbic dopamine system is thought to underlie RL in the mammalian brain by updating a representation of mean value in the striatum2,3, but little is known about whether, where, and how neurons in this circuit encode information about higher-order moments of reward distributions4. To fill this gap, we used high-density probes (Neuropixels) to acutely record striatal activity from well-trained, water-restricted mice performing a classical conditioning task in which reward mean, reward variance, and stimulus identity were independently manipulated. In contrast to traditional RL accounts, we found robust evidence for abstract encoding of variance in the striatum. Remarkably, chronic ablation of dopamine inputs disorganized these distributional representations in the striatum without interfering with mean value coding. Two-photon calcium imaging and optogenetics revealed that the two major classes of striatal medium spiny neurons - D1 and D2 MSNs - contributed to this code by preferentially encoding the right and left tails of the reward distribution, respectively. We synthesize these findings into a new model of the striatum and mesolimbic dopamine that harnesses the opponency between D1 and D2 MSNs5-15 to reap the computational benefits of distributional RL.
Collapse
Affiliation(s)
- Adam S Lowet
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
- Program in Neuroscience, Harvard University, Boston, MA, USA
| | - Qiao Zheng
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Melissa Meng
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| | - Sara Matias
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| | - Jan Drugowitsch
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Naoshige Uchida
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| |
Collapse
|
6
|
Ramarao M, Jones C, Goldberg JH, Roeser A. Songbird mesostriatal dopamine pathways are spatially segregated before the onset of vocal learning. PLoS One 2023; 18:e0285652. [PMID: 37972016 PMCID: PMC10653429 DOI: 10.1371/journal.pone.0285652] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Accepted: 10/23/2023] [Indexed: 11/19/2023] Open
Abstract
Diverse dopamine (DA) pathways send distinct reinforcement signals to different striatal regions. In adult songbirds, a DA pathway from the ventral tegmental area (VTA) to Area X, the striatal nucleus of the song system, carries singing-related performance error signals important for learning. Meanwhile, a parallel DA pathway to a medial striatal area (MST) arises from a distinct group of neighboring DA neurons that lack connectivity to song circuits and do not encode song error. To test if the structural and functional segregation of these two pathways depends on singing experience, we carried out anatomical studies early in development before the onset of song learning. We find that distinct VTA neurons project to either Area X or MST in juvenile birds before the onset of substantial vocal practice. Quantitative comparisons of early juveniles (30-35 days post hatch), late juveniles (60-65 dph), and adult (>90 dph) brains revealed an outsized expansion of Area X-projecting neurons relative to MST-projecting neurons in VTA over development. These results show that a mesostriatal DA system dedicated to social communication can exist and be spatially segregated before the onset of vocal practice and associated sensorimotor experience.
Collapse
Affiliation(s)
- Malavika Ramarao
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY, United States of America
| | - Caleb Jones
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY, United States of America
| | - Jesse H. Goldberg
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY, United States of America
| | - Andrea Roeser
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY, United States of America
| |
Collapse
|
7
|
Masset P, Tano P, Kim HR, Malik AN, Pouget A, Uchida N. Multi-timescale reinforcement learning in the brain. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.11.12.566754. [PMID: 38014166 PMCID: PMC10680596 DOI: 10.1101/2023.11.12.566754] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/29/2023]
Abstract
To thrive in complex environments, animals and artificial agents must learn to act adaptively to maximize fitness and rewards. Such adaptive behavior can be learned through reinforcement learning1, a class of algorithms that has been successful at training artificial agents2-6 and at characterizing the firing of dopamine neurons in the midbrain7-9. In classical reinforcement learning, agents discount future rewards exponentially according to a single time scale, controlled by the discount factor. Here, we explore the presence of multiple timescales in biological reinforcement learning. We first show that reinforcement agents learning at a multitude of timescales possess distinct computational benefits. Next, we report that dopamine neurons in mice performing two behavioral tasks encode reward prediction error with a diversity of discount time constants. Our model explains the heterogeneity of temporal discounting in both cue-evoked transient responses and slower timescale fluctuations known as dopamine ramps. Crucially, the measured discount factor of individual neurons is correlated across the two tasks suggesting that it is a cell-specific property. Together, our results provide a new paradigm to understand functional heterogeneity in dopamine neurons, a mechanistic basis for the empirical observation that humans and animals use non-exponential discounts in many situations10-14, and open new avenues for the design of more efficient reinforcement learning algorithms.
Collapse
Affiliation(s)
- Paul Masset
- Department of Molecular and Cellular Biology, Harvard University, USA
- Center for Brain Science, Harvard University, USA
| | - Pablo Tano
- Department of Basic Neuroscience, University of Geneva, Switzerland
| | - HyungGoo R. Kim
- Department of Molecular and Cellular Biology, Harvard University, USA
- Center for Brain Science, Harvard University, USA
- Department of Biomedical Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea
- Center for Neuroscience Imaging Research, Institute for Basic Science (IBS), Suwon 16419, Republic of Korea
| | - Athar N. Malik
- Department of Molecular and Cellular Biology, Harvard University, USA
- Center for Brain Science, Harvard University, USA
- Department of Neurosurgery, Warren Alpert Medical School of Brown University, USA
- Norman Prince Neurosciences Institute, Rhode Island Hospital, USA
| | - Alexandre Pouget
- Department of Basic Neuroscience, University of Geneva, Switzerland
| | - Naoshige Uchida
- Department of Molecular and Cellular Biology, Harvard University, USA
- Center for Brain Science, Harvard University, USA
| |
Collapse
|
8
|
Stetsenko A, Koos T. Neuronal implementation of the temporal difference learning algorithm in the midbrain dopaminergic system. Proc Natl Acad Sci U S A 2023; 120:e2309015120. [PMID: 37903252 PMCID: PMC10636325 DOI: 10.1073/pnas.2309015120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2023] [Accepted: 09/29/2023] [Indexed: 11/01/2023] Open
Abstract
The temporal difference learning (TDL) algorithm has been essential to conceptualizing the role of dopamine in reinforcement learning (RL). Despite its theoretical importance, it remains unknown whether a neuronal implementation of this algorithm exists in the brain. Here, we provide an interpretation of the recently described signaling properties of ventral tegmental area (VTA) GABAergic neurons and show that a circuitry of these neurons implements the TDL algorithm. Specifically, we identified the neuronal mechanism of three key components of the TDL model: a sustained state value signal encoded by an afferent input to the VTA, a temporal differentiation circuit formed by two types of VTA GABAergic neurons the combined output of which computes momentary reward prediction (RP) as the derivative of the state value, and the computation of reward prediction errors (RPEs) in dopamine neurons utilizing the output of the differentiation circuit. Using computational methods, we also show that this mechanism is optimally adapted to the biophysics of RPE signaling in dopamine neurons, mechanistically links the emergence of conditioned reinforcement to RP, and can naturally account for the temporal discounting of reinforcement. Elucidating the implementation of the TDL algorithm may further the investigation of RL in biological and artificial systems.
Collapse
Affiliation(s)
- Anya Stetsenko
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, NJ07102
| | - Tibor Koos
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, NJ07102
| |
Collapse
|
9
|
Faget L, Oriol L, Lee WC, Sargent C, Ramanathan D, Hnasko TS. Ventral pallidum GABA and glutamate neurons drive approach and avoidance through distinct modulation of VTA cell types. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.07.12.548750. [PMID: 37502884 PMCID: PMC10369949 DOI: 10.1101/2023.07.12.548750] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/29/2023]
Abstract
The ventral pallidum (VP) contains GABA and glutamate (Glut) neurons projecting to ventral tegmental area (VTA) whose stimulation drives approach and avoidance, respectively. Yet little is known about the cell-type-specific mechanisms by which VP projections to VTA drive behavior. Here, we found that both VP GABA and Glut neurons were activated during approach to reward or delivery of an aversive stimulus. Stimulation of VP GABA neurons inhibited VTA GABA, but activated dopamine (DA) and glutamate neurons. Remarkably, this cell-type-specific recruitment was behavior-contingent such that VTA recruitment was inhibited when evoked by the subject's own action. Conversely, VP Glut neurons activated VTA GABA, as well as DA and Glut neurons, despite driving aversion. However, VP Glut neurons evoked DA in reward-associated ventromedial nucleus accumbens (NAc), but reduced DA in aversion-associated dorsomedial NAc. These findings show how heterogeneous VP cell types can engage VTA cell types to shape approach and avoidance behaviors.
Collapse
Affiliation(s)
- Lauren Faget
- Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA 92161, USA
| | - Lucie Oriol
- Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA
| | - Wen-Chun Lee
- Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA
| | - Cody Sargent
- Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA
| | - Dhakshin Ramanathan
- Department of Psychiatry, University of California, San Diego, La Jolla, CA 92093, USA
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA 92161, USA
| | - Thomas S. Hnasko
- Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA
- Research Service, Veterans Affairs San Diego Healthcare System, San Diego, CA 92161, USA
| |
Collapse
|
10
|
Abstract
Striosomes form neurochemically specialized compartments of the striatum embedded in a large matrix made up of modules called matrisomes. Striosome-matrix architecture is multiplexed with the canonical direct-indirect organization of the striatum. Striosomal functions remain to be fully clarified, but key information is emerging. First, striosomes powerfully innervate nigral dopamine-containing neurons and can completely shut down their activity, with a following rebound excitation. Second, striosomes receive limbic and cognition-related corticostriatal afferents and are dynamically modulated in relation to value-based actions. Third, striosomes are spatially interspersed among matrisomes and interneurons and are influenced by local and global neuromodulatory and oscillatory activities. Fourth, striosomes tune engagement and the motivation to perform reinforcement learning, to manifest stereotypical behaviors, and to navigate valence conflicts and valence discriminations. We suggest that, at an algorithmic level, striosomes could serve as distributed scaffolds to provide formats of the striatal computations generated through development and refined through learning. We propose that striosomes affect subjective states. By transforming corticothalamic and other inputs to the functional formats of the striatum, they could implement state transitions in nigro-striato-nigral circuits to affect bodily and cognitive actions according to internal motives whose functions are compromised in neuropsychiatric conditions.
Collapse
Affiliation(s)
- Ann M Graybiel
- McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA;
| | - Ayano Matsushima
- McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA;
| |
Collapse
|
11
|
Perisse E, Miranda M, Trouche S. Modulation of aversive value coding in the vertebrate and invertebrate brain. Curr Opin Neurobiol 2023; 79:102696. [PMID: 36871400 DOI: 10.1016/j.conb.2023.102696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Revised: 01/30/2023] [Accepted: 01/31/2023] [Indexed: 03/06/2023]
Abstract
Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.
Collapse
Affiliation(s)
- Emmanuel Perisse
- Institute of Functional Genomics, University of Montpellier, CNRS, Inserm, 141 rue de la Cardonille, 34094 Montpellier Cedex 5, France.
| | - Magdalena Miranda
- Institute of Functional Genomics, University of Montpellier, CNRS, Inserm, 141 rue de la Cardonille, 34094 Montpellier Cedex 5, France
| | - Stéphanie Trouche
- Institute of Functional Genomics, University of Montpellier, CNRS, Inserm, 141 rue de la Cardonille, 34094 Montpellier Cedex 5, France.
| |
Collapse
|
12
|
França TFA, Pompeia S. Reappraising the role of dopamine in adolescent risk-taking behavior. Neurosci Biobehav Rev 2023; 147:105085. [PMID: 36773751 DOI: 10.1016/j.neubiorev.2023.105085] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Revised: 01/30/2023] [Accepted: 02/07/2023] [Indexed: 02/12/2023]
Abstract
Adolescence is characterized by increased risk-taking, which is often ascribed to developmental changes in dopaminergic signaling. Popular models propose that these behaviors are caused by dopamine-induced hypersensitivity to rewards, which overrides adolescents' immature self-control mechanisms. However, these models are often based on oversimplified notions about the workings and functions of dopamine. Here we discuss the relationship between changes in the dopaminergic system and adolescent behavior in light of current theories/models about the functions of dopamine. We show that dopamine is linked to learning, adaptive decision-making under uncertainty, and increased motivation to work for rewards. Thus, changes in the dopaminergic system contribute to the maturation of cognitive control through various mechanisms, contrary to the false dichotomy between reward processing and self-control. Finally, we note that dopamine interacts with a number of other neuromodulator systems, which also change during adolescence, but that have been largely ignored in the field of adolescent development. A full understanding of adolescent behavior will require these neurochemicals and their interactions with dopamine to be taken into account.
Collapse
Affiliation(s)
- Thiago F A França
- Universidade Federal de São Paulo. Escola Paulista de Medicina. Departamento de Psicobiologia. São Paulo - SP, Brasil
| | - Sabine Pompeia
- Universidade Federal de São Paulo. Escola Paulista de Medicina. Departamento de Psicobiologia. São Paulo - SP, Brasil.
| |
Collapse
|
13
|
Yamada D, Bushey D, Li F, Hibbard KL, Sammons M, Funke J, Litwin-Kumar A, Hige T, Aso Y. Hierarchical architecture of dopaminergic circuits enables second-order conditioning in Drosophila. eLife 2023; 12:79042. [PMID: 36692262 PMCID: PMC9937650 DOI: 10.7554/elife.79042] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Accepted: 01/23/2023] [Indexed: 01/25/2023] Open
Abstract
Dopaminergic neurons with distinct projection patterns and physiological properties compose memory subsystems in a brain. However, it is poorly understood whether or how they interact during complex learning. Here, we identify a feedforward circuit formed between dopamine subsystems and show that it is essential for second-order conditioning, an ethologically important form of higher-order associative learning. The Drosophila mushroom body comprises a series of dopaminergic compartments, each of which exhibits distinct memory dynamics. We find that a slow and stable memory compartment can serve as an effective 'teacher' by instructing other faster and transient memory compartments via a single key interneuron, which we identify by connectome analysis and neurotransmitter prediction. This excitatory interneuron acquires enhanced response to reward-predicting odor after first-order conditioning and, upon activation, evokes dopamine release in the 'student' compartments. These hierarchical connections between dopamine subsystems explain distinct properties of first- and second-order memory long known by behavioral psychologists.
Collapse
Affiliation(s)
- Daichi Yamada
- Department of Biology, University of North Carolina at Chapel Hill, Chapel Hill, United States
| | - Daniel Bushey
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| | - Feng Li
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| | - Karen L Hibbard
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| | - Megan Sammons
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| | - Jan Funke
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| | - Ashok Litwin-Kumar
- Department of Neuroscience, Columbia University, New York, United States
| | - Toshihide Hige
- Department of Biology, University of North Carolina at Chapel Hill, Chapel Hill, United States
- Department of Cell Biology and Physiology, University of North Carolina at Chapel Hill, Chapel Hill, United States
- Integrative Program for Biological and Genome Sciences, University of North Carolina at Chapel Hill, Chapel Hill, United States
| | - Yoshinori Aso
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| |
Collapse
|
14
|
Evans AK, Defensor E, Shamloo M. Selective Vulnerability of the Locus Coeruleus Noradrenergic System and its Role in Modulation of Neuroinflammation, Cognition, and Neurodegeneration. Front Pharmacol 2022; 13:1030609. [PMID: 36532725 PMCID: PMC9748190 DOI: 10.3389/fphar.2022.1030609] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Accepted: 11/14/2022] [Indexed: 05/13/2024] Open
Abstract
Locus coeruleus (LC) noradrenergic (NE) neurons supply the main adrenergic input to the forebrain. NE is a dual modulator of cognition and neuroinflammation. NE neurons of the LC are particularly vulnerable to degeneration both with normal aging and in neurodegenerative disorders. Consequences of this vulnerability can be observed in both cognitive impairment and dysregulation of neuroinflammation. LC NE neurons are pacemaker neurons that are active during waking and arousal and are responsive to stressors in the environment. Chronic overactivation is thought to be a major contributor to the vulnerability of these neurons. Here we review what is known about the mechanisms underlying this neuronal vulnerability and combinations of environmental and genetic factors that contribute to confer risk to these important brainstem neuromodulatory and immunomodulatory neurons. Finally, we discuss proposed and potential interventions that may reduce the overall risk for LC NE neuronal degeneration.
Collapse
Affiliation(s)
- Andrew K. Evans
- School of Medicine, Stanford University, Stanford, CA, United States
| | | | - Mehrdad Shamloo
- School of Medicine, Stanford University, Stanford, CA, United States
| |
Collapse
|
15
|
Akiti K, Tsutsui-Kimura I, Xie Y, Mathis A, Markowitz JE, Anyoha R, Datta SR, Mathis MW, Uchida N, Watabe-Uchida M. Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction. Neuron 2022; 110:3789-3804.e9. [PMID: 36130595 PMCID: PMC9671833 DOI: 10.1016/j.neuron.2022.08.022] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2021] [Revised: 06/03/2022] [Accepted: 08/18/2022] [Indexed: 12/15/2022]
Abstract
Animals both explore and avoid novel objects in the environment, but the neural mechanisms that underlie these behaviors and their dynamics remain uncharacterized. Here, we used multi-point tracking (DeepLabCut) and behavioral segmentation (MoSeq) to characterize the behavior of mice freely interacting with a novel object. Novelty elicits a characteristic sequence of behavior, starting with investigatory approach and culminating in object engagement or avoidance. Dopamine in the tail of the striatum (TS) suppresses engagement, and dopamine responses were predictive of individual variability in behavior. Behavioral dynamics and individual variability are explained by a reinforcement-learning (RL) model of threat prediction in which behavior arises from a novelty-induced initial threat prediction (akin to "shaping bonus") and a threat prediction that is learned through dopamine-mediated threat prediction errors. These results uncover an algorithmic similarity between reward- and threat-related dopamine sub-systems.
Collapse
Affiliation(s)
- Korleki Akiti
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
| | - Iku Tsutsui-Kimura
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
| | - Yudi Xie
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA 02138, USA; Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Alexander Mathis
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA 02138, USA; The Rowland Institute at Harvard, Harvard University, Cambridge, MA 02138, USA; Swiss Federal Institute of Technology Lausanne, Geneve 1202, Switzerland
| | - Jeffrey E Markowitz
- Department of Neurobiology, Harvard Medical School, Boston, MA 02115, USA; Wallace H. Coulter Department of Biomedical Engineering, Emory School of Medicine, Georgia Institute of Technology, Atlanta, GA 30322, USA
| | - Rockwell Anyoha
- Department of Neurobiology, Harvard Medical School, Boston, MA 02115, USA
| | | | - Mackenzie Weygandt Mathis
- The Rowland Institute at Harvard, Harvard University, Cambridge, MA 02138, USA; Swiss Federal Institute of Technology Lausanne, Geneve 1202, Switzerland
| | - Naoshige Uchida
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
| | - Mitsuko Watabe-Uchida
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA 02138, USA.
| |
Collapse
|
16
|
Manoim JE, Davidson AM, Weiss S, Hige T, Parnas M. Lateral axonal modulation is required for stimulus-specific olfactory conditioning in Drosophila. Curr Biol 2022; 32:4438-4450.e5. [PMID: 36130601 PMCID: PMC9613607 DOI: 10.1016/j.cub.2022.09.007] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2022] [Revised: 08/15/2022] [Accepted: 09/04/2022] [Indexed: 11/26/2022]
Abstract
Effective and stimulus-specific learning is essential for animals' survival. Two major mechanisms are known to aid stimulus specificity of associative learning. One is accurate stimulus-specific representations in neurons. The second is a limited effective temporal window for the reinforcing signals to induce neuromodulation after sensory stimuli. However, these mechanisms are often imperfect in preventing unspecific associations; different sensory stimuli can be represented by overlapping populations of neurons, and more importantly, the reinforcing signals alone can induce neuromodulation even without coincident sensory-evoked neuronal activity. Here, we report a crucial neuromodulatory mechanism that counteracts both limitations and is thereby essential for stimulus specificity of learning. In Drosophila, olfactory signals are sparsely represented by cholinergic Kenyon cells (KCs), which receive dopaminergic reinforcing input. We find that KCs have numerous axo-axonic connections mediated by the muscarinic type-B receptor (mAChR-B). By using functional imaging and optogenetic approaches, we show that these axo-axonic connections suppress both odor-evoked calcium responses and dopamine-evoked cAMP signals in neighboring KCs. Strikingly, behavior experiments demonstrate that mAChR-B knockdown in KCs impairs olfactory learning by inducing undesired changes to the valence of an odor that was not associated with the reinforcer. Thus, this local neuromodulation acts in concert with sparse sensory representations and global dopaminergic modulation to achieve effective and accurate memory formation.
Collapse
Affiliation(s)
- Julia E Manoim
- Department of Physiology and Pharmacology, Sackler School of Medicine, Tel Aviv University, Tel Aviv 69978, Israel
| | - Andrew M Davidson
- Department of Biology, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Shirley Weiss
- Department of Physiology and Pharmacology, Sackler School of Medicine, Tel Aviv University, Tel Aviv 69978, Israel
| | - Toshihide Hige
- Department of Biology, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA; Department of Cell Biology and Physiology, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA; Integrative Program for Biological and Genome Sciences, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA.
| | - Moshe Parnas
- Department of Physiology and Pharmacology, Sackler School of Medicine, Tel Aviv University, Tel Aviv 69978, Israel; Sagol School of Neuroscience, Tel Aviv University, Tel Aviv 69978, Israel.
| |
Collapse
|
17
|
Dutta CN, Christov-Moore L, Ombao H, Douglas PK. Neuroprotection in late life attention-deficit/hyperactivity disorder: A review of pharmacotherapy and phenotype across the lifespan. Front Hum Neurosci 2022; 16:938501. [PMID: 36226261 PMCID: PMC9548548 DOI: 10.3389/fnhum.2022.938501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2022] [Accepted: 08/16/2022] [Indexed: 11/13/2022] Open
Abstract
For decades, psychostimulants have been the gold standard pharmaceutical treatment for attention-deficit/hyperactivity disorder (ADHD). In the United States, an astounding 9% of all boys and 4% of girls will be prescribed stimulant drugs at some point during their childhood. Recent meta-analyses have revealed that individuals with ADHD have reduced brain volume loss later in life (>60 y.o.) compared to the normal aging brain, which suggests that either ADHD or its treatment may be neuroprotective. Crucially, these neuroprotective effects were significant in brain regions (e.g., hippocampus, amygdala) where severe volume loss is linked to cognitive impairment and Alzheimer's disease. Historically, the ADHD diagnosis and its pharmacotherapy came about nearly simultaneously, making it difficult to evaluate their effects in isolation. Certain evidence suggests that psychostimulants may normalize structural brain changes typically observed in the ADHD brain. If ADHD itself is neuroprotective, perhaps exercising the brain, then psychostimulants may not be recommended across the lifespan. Alternatively, if stimulant drugs are neuroprotective, then this class of medications may warrant further investigation for their therapeutic effects. Here, we take a bottom-up holistic approach to review the psychopharmacology of ADHD in the context of recent models of attention. We suggest that future studies are greatly needed to better appreciate the interactions amongst an ADHD diagnosis, stimulant treatment across the lifespan, and structure-function alterations in the aging brain.
Collapse
Affiliation(s)
- Cintya Nirvana Dutta
- Biostatistics Group, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia
- School of Modeling, Simulation, and Training, and Computer Science, University of Central Florida, Orlando, FL, United States
| | - Leonardo Christov-Moore
- Brain and Creativity Institute, University of Southern California, Los Angeles, CA, United States
| | - Hernando Ombao
- Biostatistics Group, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia
| | - Pamela K. Douglas
- School of Modeling, Simulation, and Training, and Computer Science, University of Central Florida, Orlando, FL, United States
- Department of Psychiatry and Biobehavioral Medicine, University of California, Los Angeles, Los Angeles, CA, United States
| |
Collapse
|
18
|
Villar ME, Pavão-Delgado M, Amigo M, Jacob PF, Merabet N, Pinot A, Perry SA, Waddell S, Perisse E. Differential coding of absolute and relative aversive value in the Drosophila brain. Curr Biol 2022; 32:4576-4592.e5. [DOI: 10.1016/j.cub.2022.08.058] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Revised: 06/24/2022] [Accepted: 08/19/2022] [Indexed: 11/30/2022]
|
19
|
Abstract
Dopamine was first described by George Barger, James Ewens, and Henry Dale in 1910 as an epinephrine-like monoamine compound. Initially believed to be a mere precursor of norepinephrine, it was mostly ignored for the next four decades (Figure 1A). However, in the 1950s Kathleen Montagu showed that dopamine occurred in the brain by itself, and a series of studies by Arvid Carlsson and collaborators demonstrated that dopamine is a bona fide neurotransmitter, a finding that would earn Carlsson the 2000 Nobel Prize in Physiology and Medicine. In a landmark experiment, he pharmacologically blocked all dopamine neurotransmission in rabbits, which rendered them completely paralyzed, and then fully recovered their behavior with an injection of the dopamine precursor L-DOPA, demonstrating that dopamine was essential for self-initiated movement (Figure 1B). A similar effect was quickly reproduced by Oleg Hornykiewicz and collaborators in human Parkinsonian patients. Within a few years, dopamine jumped from relative obscurity to being critical for life as we know it.
Collapse
|
20
|
Robert B, Kimchi EY, Watanabe Y, Chakoma T, Jing M, Li Y, Polley DB. A functional topography within the cholinergic basal forebrain for encoding sensory cues and behavioral reinforcement outcomes. eLife 2021; 10:e69514. [PMID: 34821218 PMCID: PMC8654357 DOI: 10.7554/elife.69514] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2021] [Accepted: 11/16/2021] [Indexed: 01/01/2023] Open
Abstract
Basal forebrain cholinergic neurons (BFCNs) project throughout the cortex to regulate arousal, stimulus salience, plasticity, and learning. Although often treated as a monolithic structure, the basal forebrain features distinct connectivity along its rostrocaudal axis that could impart regional differences in BFCN processing. Here, we performed simultaneous bulk calcium imaging from rostral and caudal BFCNs over a 1-month period of variable reinforcement learning in mice. BFCNs in both regions showed equivalently weak responses to unconditioned visual stimuli and anticipated rewards. Rostral BFCNs in the horizontal limb of the diagonal band were more responsive to reward omission, more accurately classified behavioral outcomes, and more closely tracked fluctuations in pupil-indexed global brain state. Caudal tail BFCNs in globus pallidus and substantia innominata were more responsive to unconditioned auditory stimuli, orofacial movements, aversive reinforcement, and showed robust associative plasticity for punishment-predicting cues. These results identify a functional topography that diversifies cholinergic modulatory signals broadcast to downstream brain regions.
Collapse
Affiliation(s)
- Blaise Robert
- Eaton-Peabody Laboratories, Massachusetts Eye and Ear InfirmaryBostonUnited States
| | - Eyal Y Kimchi
- Eaton-Peabody Laboratories, Massachusetts Eye and Ear InfirmaryBostonUnited States
- Department of Neurology, Massachusetts General HospitalBostonUnited States
| | - Yurika Watanabe
- Eaton-Peabody Laboratories, Massachusetts Eye and Ear InfirmaryBostonUnited States
| | - Tatenda Chakoma
- Eaton-Peabody Laboratories, Massachusetts Eye and Ear InfirmaryBostonUnited States
| | - Miao Jing
- Chinese Institute for Brain ResearchBeijingChina
| | - Yulong Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences; PKU-IDG/McGovern Institute for Brain Research; Peking-Tsinghua Center for Life Sciences, BeijingBeijingChina
| | - Daniel B Polley
- Eaton-Peabody Laboratories, Massachusetts Eye and Ear InfirmaryBostonUnited States
- Department of Otolaryngology - Head and Neck Surgery, Harvard Medical SchoolBostonUnited States
| |
Collapse
|
21
|
Pereira Luppi M, Azcorra M, Caronia-Brown G, Poulin JF, Gaertner Z, Gatica S, Moreno-Ramos OA, Nouri N, Dubois M, Ma YC, Ramakrishnan C, Fenno L, Kim YS, Deisseroth K, Cicchetti F, Dombeck DA, Awatramani R. Sox6 expression distinguishes dorsally and ventrally biased dopamine neurons in the substantia nigra with distinctive properties and embryonic origins. Cell Rep 2021; 37:109975. [PMID: 34758317 PMCID: PMC8607753 DOI: 10.1016/j.celrep.2021.109975] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2019] [Revised: 09/15/2021] [Accepted: 10/19/2021] [Indexed: 12/12/2022] Open
Abstract
Dopamine (DA) neurons in the ventral tier of the substantia nigra pars compacta (SNc) degenerate prominently in Parkinson's disease, while those in the dorsal tier are relatively spared. Defining the molecular, functional, and developmental characteristics of each SNc tier is crucial to understand their distinct susceptibility. We demonstrate that Sox6 expression distinguishes ventrally and dorsally biased DA neuron populations in the SNc. The Sox6+ population in the ventral SNc includes an Aldh1a1+ subset and is enriched in gene pathways that underpin vulnerability. Sox6+ neurons project to the dorsal striatum and show activity correlated with acceleration. Sox6- neurons project to the medial, ventral, and caudal striatum and respond to rewards. Moreover, we show that this adult division is encoded early in development. Overall, our work demonstrates a dual origin of the SNc that results in DA neuron cohorts with distinct molecular profiles, projections, and functions.
Collapse
Affiliation(s)
- Milagros Pereira Luppi
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Maite Azcorra
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA; Department of Neurobiology, Northwestern University, Evanston, IL, USA
| | - Giuliana Caronia-Brown
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Jean-Francois Poulin
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA; Montreal Neurological Institute, McGill University, Montreal, QC, Canada
| | - Zachary Gaertner
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Serafin Gatica
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | | | - Navid Nouri
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Marilyn Dubois
- Department of Psychiatry and Neuroscience, Faculty of Medicine, Université Laval, Québec, QC, Canada
| | - Yongchao C Ma
- Department of Pediatrics, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Charu Ramakrishnan
- Department of Bioengineering, Stanford University School of Medicine, Stanford, CA, USA
| | - Lief Fenno
- Department of Bioengineering, Stanford University School of Medicine, Stanford, CA, USA
| | - Yoon Seok Kim
- Department of Bioengineering, Stanford University School of Medicine, Stanford, CA, USA
| | - Karl Deisseroth
- Department of Bioengineering, Stanford University School of Medicine, Stanford, CA, USA
| | - Francesca Cicchetti
- Department of Psychiatry and Neuroscience, Faculty of Medicine, Université Laval, Québec, QC, Canada
| | - Daniel A Dombeck
- Department of Neurobiology, Northwestern University, Evanston, IL, USA.
| | - Rajeshwar Awatramani
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA.
| |
Collapse
|
22
|
Zolin A, Cohn R, Pang R, Siliciano AF, Fairhall AL, Ruta V. Context-dependent representations of movement in Drosophila dopaminergic reinforcement pathways. Nat Neurosci 2021; 24:1555-1566. [PMID: 34697455 PMCID: PMC8556349 DOI: 10.1038/s41593-021-00929-y] [Citation(s) in RCA: 41] [Impact Index Per Article: 13.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2020] [Accepted: 09/01/2021] [Indexed: 11/09/2022]
Abstract
Dopamine plays a central role in motivating and modifying behavior, serving to invigorate current behavioral performance and guide future actions through learning. Here we examine how this single neuromodulator can contribute to such diverse forms of behavioral modulation. By recording from the dopaminergic reinforcement pathways of the Drosophila mushroom body during active odor navigation, we reveal how their ongoing motor-associated activity relates to goal-directed behavior. We found that dopaminergic neurons correlate with different behavioral variables depending on the specific navigational strategy of an animal, such that the activity of these neurons preferentially reflects the actions most relevant to odor pursuit. Furthermore, we show that these motor correlates are translated to ongoing dopamine release, and acutely perturbing dopaminergic signaling alters the strength of odor tracking. Context-dependent representations of movement and reinforcement cues are thus multiplexed within the mushroom body dopaminergic pathways, enabling them to coordinately influence both ongoing and future behavior.
Collapse
Affiliation(s)
- Aryeh Zolin
- Laboratory of Neurophysiology and Behavior, The Rockefeller University, New York, NY, USA
| | - Raphael Cohn
- Laboratory of Neurophysiology and Behavior, The Rockefeller University, New York, NY, USA
| | - Rich Pang
- Neuroscience Graduate Program, Department of Physiology and Biophysics and Computational Neuroscience Center, University of Washington, Seattle, WA, USA
| | - Andrew F Siliciano
- Laboratory of Neurophysiology and Behavior, The Rockefeller University, New York, NY, USA
| | - Adrienne L Fairhall
- Neuroscience Graduate Program, Department of Physiology and Biophysics and Computational Neuroscience Center, University of Washington, Seattle, WA, USA
| | - Vanessa Ruta
- Laboratory of Neurophysiology and Behavior, The Rockefeller University, New York, NY, USA.
| |
Collapse
|
23
|
Jiang L, Litwin-Kumar A. Models of heterogeneous dopamine signaling in an insect learning and memory center. PLoS Comput Biol 2021; 17:e1009205. [PMID: 34375329 PMCID: PMC8354444 DOI: 10.1371/journal.pcbi.1009205] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2020] [Accepted: 06/22/2021] [Indexed: 11/25/2022] Open
Abstract
The Drosophila mushroom body exhibits dopamine dependent synaptic plasticity that underlies the acquisition of associative memories. Recordings of dopamine neurons in this system have identified signals related to external reinforcement such as reward and punishment. However, other factors including locomotion, novelty, reward expectation, and internal state have also recently been shown to modulate dopamine neurons. This heterogeneity is at odds with typical modeling approaches in which these neurons are assumed to encode a global, scalar error signal. How is dopamine dependent plasticity coordinated in the presence of such heterogeneity? We develop a modeling approach that infers a pattern of dopamine activity sufficient to solve defined behavioral tasks, given architectural constraints informed by knowledge of mushroom body circuitry. Model dopamine neurons exhibit diverse tuning to task parameters while nonetheless producing coherent learned behaviors. Notably, reward prediction error emerges as a mode of population activity distributed across these neurons. Our results provide a mechanistic framework that accounts for the heterogeneity of dopamine activity during learning and behavior. Dopamine neurons across the animal kingdom are involved in the formation of associative memories. While numerous studies have recorded activity in these neurons related to external and predicted rewards, the diversity of these neurons’ activity and their tuning to non-reward-related quantities such as novelty, movement, and internal state have proved challenging to account for in traditional modeling approaches. Using a well-characterized model system for learning and memory, the mushroom body of Drosophila fruit flies, Jiang and Litwin-Kumar provide an account of the diversity of signals across dopamine neurons. They show that models optimized to solve tasks like those encountered by flies exhibit heterogeneous activity across dopamine neurons, but nonetheless this activity is sufficient for the system to solve the tasks. The models will be useful to generate testable hypotheses about dopamine neuron activity across different experimental conditions.
Collapse
Affiliation(s)
- Linnie Jiang
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Department of Neuroscience, Columbia University, New York, New York, United States of America
- Neurosciences Program, Stanford University, Stanford, California, United States of America
| | - Ashok Litwin-Kumar
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Department of Neuroscience, Columbia University, New York, New York, United States of America
- * E-mail:
| |
Collapse
|
24
|
Saputra F, Lai YH, Fernandez RAT, Macabeo APG, Lai HT, Huang JC, Hsiao CD. Acute and Sub-Chronic Exposure to Artificial Sweeteners at the Highest Environmentally Relevant Concentration Induce Less Cardiovascular Physiology Alterations in Zebrafish Larvae. BIOLOGY 2021; 10:548. [PMID: 34207293 PMCID: PMC8233861 DOI: 10.3390/biology10060548] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Revised: 06/14/2021] [Accepted: 06/15/2021] [Indexed: 12/16/2022]
Abstract
Artificial sweeteners are widely used food ingredients in beverages and drinks to lower calorie intake which in turn helps prevent lifestyle diseases such as obesity. However, as their popularity has increased, the release of artificial sweetener to the aquatic environment has also increased at a tremendous rate. Thus, our study aims to systematically explore the potential cardiovascular physiology alterations caused by eight commercial artificial sweeteners, including acesulfame-K, alitame, aspartame, sodium cyclamate, dulcin, neotame, saccharine and sucralose, at the highest environmentally relevant concentration on cardiovascular performance using zebrafish (Danio rerio) as a model system. Embryonic zebrafish were exposed to the eight artificial sweeteners at 100 ppb and their cardiovascular performance (heart rate, ejection fraction, fractional shortening, stroke volume, cardiac output, heartbeat variability, and blood flow velocity) was measured and compared. Overall, our finding supports the safety of artificial sweetener exposure. However, several finding like a significant increase in the heart rate and heart rate variability after incubation in several artificial sweeteners are noteworthy. Biomarker testing also revealed that saccharine significantly increase the dopamine level in zebrafish larvae, which is might be the reason for the cardiac physiology changes observed after saccharine exposure.
Collapse
Affiliation(s)
- Ferry Saputra
- Department of Bioscience Technology, Chung Yuan Christian University, Taoyuan 320314, Taiwan;
| | - Yu-Heng Lai
- Department of Chemistry, Chinese Culture University, Taipei 11114, Taiwan;
| | - Rey Arturo T. Fernandez
- Laboratory for Organic Reactivity, Discovery and Synthesis (LORDS), Research Center for the Natural and Applied Sciences, University of Santo Tomas, Espana St., Manila 1015, Philippines; (R.A.T.F.); (A.P.G.M.)
| | - Allan Patrick G. Macabeo
- Laboratory for Organic Reactivity, Discovery and Synthesis (LORDS), Research Center for the Natural and Applied Sciences, University of Santo Tomas, Espana St., Manila 1015, Philippines; (R.A.T.F.); (A.P.G.M.)
| | - Hong-Thih Lai
- Department of Aquatic Biosciences, National Chiayi University, Chiayi 600355, Taiwan
| | - Jong-Chin Huang
- Department of Applied Chemistry, National Pingtung University, Pingtung 90003, Taiwan
| | - Chung-Der Hsiao
- Department of Bioscience Technology, Chung Yuan Christian University, Taoyuan 320314, Taiwan;
- Center for Nanotechnology, Chung Yuan Christian University, Taoyuan 320314, Taiwan
- Research Center for Aquatic Toxicology and Pharmacology, Chung Yuan Christian University, Taoyuan 320314, Taiwan
| |
Collapse
|
25
|
Noradrenergic correlates of chronic cocaine craving: neuromelanin and functional brain imaging. Neuropsychopharmacology 2021; 46:851-859. [PMID: 33408330 PMCID: PMC8027452 DOI: 10.1038/s41386-020-00937-9] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/09/2020] [Revised: 11/18/2020] [Accepted: 11/24/2020] [Indexed: 12/13/2022]
Abstract
Preclinical studies have implicated noradrenergic (NA) dysfunction in cocaine addiction. In particular, the NA system plays a central role in motivated behavior and may partake in the regulation of craving and drug use. Yet, human studies of the NA system are scarce, likely hampered by the difficulty in precisely localizing the locus coeruleus (LC). Here, we used neuromelanin imaging to localize the LC and quantified LC neuromelanin signal (NMS) intensity in 44 current cocaine users (CU; 37 men) and 59 nondrug users (NU; 44 men). We also employed fMRI to investigate cue-induced regional responses and LC functional connectivities, as quantified by generalized psychophysiological interaction (gPPI), in CU. Imaging data were processed by published routines and the findings were evaluated with a corrected threshold. We examined how these neural measures were associated with chronic cocaine craving, as assessed by the Cocaine Craving Questionnaire (CCQ). Compared to NU, CU demonstrated higher LC NMS for all probabilistic thresholds defined of 50-90% of the peak. In contrast, NMS of the ventral tegmental area/substantia nigra (VTA/SN) did not show significant group differences. Drug as compared to neutral cues elicited higher activations of many cortical and subcortical regions, none of which were significantly correlated with CCQ score. Drug vs. neutral cues also elicited "deactivation" of bilateral parahippocampal gyri (PHG) and PHG gPPI with a wide array of cortical and subcortical regions, including the ventral striatum and, with small volume correction, the LC. Less deactivation of the PHG (r = 0.40, p = 0.008) and higher PHG-LC gPPI (r = 0.44, p = 0.003) were positively correlated with the CCQ score. In contrast, PHG-VTA/SN connectivity did not correlate with the CCQ score. Together, chronic cocaine exposure may induce higher NMS intensity, suggesting neurotoxic effects on the LC. The correlation of cue-elicited PHG LC connectivity with CCQ score suggests a noradrenergic correlate of chronic cocaine craving. Potentially compensating for memory functions as in neurodegenerative conditions, cue-elicited PHG LC circuit connectivity plays an ill-adaptive role in supporting cocaine craving.
Collapse
|
26
|
Siju KP, De Backer JF, Grunwald Kadow IC. Dopamine modulation of sensory processing and adaptive behavior in flies. Cell Tissue Res 2021; 383:207-225. [PMID: 33515291 PMCID: PMC7873103 DOI: 10.1007/s00441-020-03371-x] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2020] [Accepted: 11/26/2020] [Indexed: 12/31/2022]
Abstract
Behavioral flexibility for appropriate action selection is an advantage when animals are faced with decisions that will determine their survival or death. In order to arrive at the right decision, animals evaluate information from their external environment, internal state, and past experiences. How these different signals are integrated and modulated in the brain, and how context- and state-dependent behavioral decisions are controlled are poorly understood questions. Studying the molecules that help convey and integrate such information in neural circuits is an important way to approach these questions. Many years of work in different model organisms have shown that dopamine is a critical neuromodulator for (reward based) associative learning. However, recent findings in vertebrates and invertebrates have demonstrated the complexity and heterogeneity of dopaminergic neuron populations and their functional implications in many adaptive behaviors important for survival. For example, dopaminergic neurons can integrate external sensory information, internal and behavioral states, and learned experience in the decision making circuitry. Several recent advances in methodologies and the availability of a synaptic level connectome of the whole-brain circuitry of Drosophila melanogaster make the fly an attractive system to study the roles of dopamine in decision making and state-dependent behavior. In particular, a learning and memory center-the mushroom body-is richly innervated by dopaminergic neurons that enable it to integrate multi-modal information according to state and context, and to modulate decision-making and behavior.
Collapse
Affiliation(s)
- K. P. Siju
- School of Life Sciences, Department of Molecular Life Sciences, Technical University of Munich, 85354 Freising, Germany
| | - Jean-Francois De Backer
- School of Life Sciences, Department of Molecular Life Sciences, Technical University of Munich, 85354 Freising, Germany
| | - Ilona C. Grunwald Kadow
- School of Life Sciences, Department of Molecular Life Sciences, Technical University of Munich, 85354 Freising, Germany
| |
Collapse
|
27
|
Tsutsui-Kimura I, Matsumoto H, Akiti K, Yamada MM, Uchida N, Watabe-Uchida M. Distinct temporal difference error signals in dopamine axons in three regions of the striatum in a decision-making task. eLife 2020; 9:e62390. [PMID: 33345774 PMCID: PMC7771962 DOI: 10.7554/elife.62390] [Citation(s) in RCA: 51] [Impact Index Per Article: 12.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2020] [Accepted: 12/18/2020] [Indexed: 12/24/2022] Open
Abstract
Different regions of the striatum regulate different types of behavior. However, how dopamine signals differ across striatal regions and how dopamine regulates different behaviors remain unclear. Here, we compared dopamine axon activity in the ventral, dorsomedial, and dorsolateral striatum, while mice performed a perceptual and value-based decision task. Surprisingly, dopamine axon activity was similar across all three areas. At a glance, the activity multiplexed different variables such as stimulus-associated values, confidence, and reward feedback at different phases of the task. Our modeling demonstrates, however, that these modulations can be inclusively explained by moment-by-moment changes in the expected reward, that is the temporal difference error. A major difference between areas was the overall activity level of reward responses: reward responses in dorsolateral striatum were positively shifted, lacking inhibitory responses to negative prediction errors. The differences in dopamine signals put specific constraints on the properties of behaviors controlled by dopamine in these regions.
Collapse
Affiliation(s)
- Iku Tsutsui-Kimura
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard UniversityCambridgeUnited States
| | - Hideyuki Matsumoto
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard UniversityCambridgeUnited States
- Department of Physiology, Osaka City University Graduate School of MedicineOsakaJapan
| | - Korleki Akiti
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard UniversityCambridgeUnited States
| | - Melissa M Yamada
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard UniversityCambridgeUnited States
| | - Naoshige Uchida
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard UniversityCambridgeUnited States
| | - Mitsuko Watabe-Uchida
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard UniversityCambridgeUnited States
| |
Collapse
|
28
|
Li F, Lindsey JW, Marin EC, Otto N, Dreher M, Dempsey G, Stark I, Bates AS, Pleijzier MW, Schlegel P, Nern A, Takemura SY, Eckstein N, Yang T, Francis A, Braun A, Parekh R, Costa M, Scheffer LK, Aso Y, Jefferis GSXE, Abbott LF, Litwin-Kumar A, Waddell S, Rubin GM. The connectome of the adult Drosophila mushroom body provides insights into function. eLife 2020; 9:e62576. [PMID: 33315010 PMCID: PMC7909955 DOI: 10.7554/elife.62576] [Citation(s) in RCA: 174] [Impact Index Per Article: 43.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2020] [Accepted: 12/11/2020] [Indexed: 12/12/2022] Open
Abstract
Making inferences about the computations performed by neuronal circuits from synapse-level connectivity maps is an emerging opportunity in neuroscience. The mushroom body (MB) is well positioned for developing and testing such an approach due to its conserved neuronal architecture, recently completed dense connectome, and extensive prior experimental studies of its roles in learning, memory, and activity regulation. Here, we identify new components of the MB circuit in Drosophila, including extensive visual input and MB output neurons (MBONs) with direct connections to descending neurons. We find unexpected structure in sensory inputs, in the transfer of information about different sensory modalities to MBONs, and in the modulation of that transfer by dopaminergic neurons (DANs). We provide insights into the circuitry used to integrate MB outputs, connectivity between the MB and the central complex and inputs to DANs, including feedback from MBONs. Our results provide a foundation for further theoretical and experimental work.
Collapse
Affiliation(s)
- Feng Li
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| | - Jack W Lindsey
- Department of Neuroscience, Columbia University, Zuckerman InstituteNew YorkUnited States
| | - Elizabeth C Marin
- Drosophila Connectomics Group, Department of Zoology, University of CambridgeCambridgeUnited Kingdom
| | - Nils Otto
- Drosophila Connectomics Group, Department of Zoology, University of CambridgeCambridgeUnited Kingdom
- Centre for Neural Circuits & Behaviour, University of OxfordOxfordUnited Kingdom
| | - Marisa Dreher
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| | - Georgia Dempsey
- Drosophila Connectomics Group, Department of Zoology, University of CambridgeCambridgeUnited Kingdom
| | - Ildiko Stark
- Drosophila Connectomics Group, Department of Zoology, University of CambridgeCambridgeUnited Kingdom
| | - Alexander S Bates
- Neurobiology Division, MRC Laboratory of Molecular BiologyCambridgeUnited Kingdom
| | | | - Philipp Schlegel
- Drosophila Connectomics Group, Department of Zoology, University of CambridgeCambridgeUnited Kingdom
- Neurobiology Division, MRC Laboratory of Molecular BiologyCambridgeUnited Kingdom
| | - Aljoscha Nern
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| | - Shin-ya Takemura
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| | - Nils Eckstein
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| | - Tansy Yang
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| | - Audrey Francis
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| | - Amalia Braun
- Drosophila Connectomics Group, Department of Zoology, University of CambridgeCambridgeUnited Kingdom
| | - Ruchi Parekh
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| | - Marta Costa
- Drosophila Connectomics Group, Department of Zoology, University of CambridgeCambridgeUnited Kingdom
| | - Louis K Scheffer
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| | - Yoshinori Aso
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| | - Gregory SXE Jefferis
- Drosophila Connectomics Group, Department of Zoology, University of CambridgeCambridgeUnited Kingdom
- Neurobiology Division, MRC Laboratory of Molecular BiologyCambridgeUnited Kingdom
| | - Larry F Abbott
- Department of Neuroscience, Columbia University, Zuckerman InstituteNew YorkUnited States
| | - Ashok Litwin-Kumar
- Department of Neuroscience, Columbia University, Zuckerman InstituteNew YorkUnited States
| | - Scott Waddell
- Centre for Neural Circuits & Behaviour, University of OxfordOxfordUnited Kingdom
| | - Gerald M Rubin
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| |
Collapse
|
29
|
Kim HR, Malik AN, Mikhael JG, Bech P, Tsutsui-Kimura I, Sun F, Zhang Y, Li Y, Watabe-Uchida M, Gershman SJ, Uchida N. A Unified Framework for Dopamine Signals across Timescales. Cell 2020; 183:1600-1616.e25. [PMID: 33248024 PMCID: PMC7736562 DOI: 10.1016/j.cell.2020.11.013] [Citation(s) in RCA: 121] [Impact Index Per Article: 30.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2019] [Revised: 08/20/2020] [Accepted: 11/09/2020] [Indexed: 01/06/2023]
Abstract
Rapid phasic activity of midbrain dopamine neurons is thought to signal reward prediction errors (RPEs), resembling temporal difference errors used in machine learning. However, recent studies describing slowly increasing dopamine signals have instead proposed that they represent state values and arise independent from somatic spiking activity. Here we developed experimental paradigms using virtual reality that disambiguate RPEs from values. We examined dopamine circuit activity at various stages, including somatic spiking, calcium signals at somata and axons, and striatal dopamine concentrations. Our results demonstrate that ramping dopamine signals are consistent with RPEs rather than value, and this ramping is observed at all stages examined. Ramping dopamine signals can be driven by a dynamic stimulus that indicates a gradual approach to a reward. We provide a unified computational understanding of rapid phasic and slowly ramping dopamine signals: dopamine neurons perform a derivative-like computation over values on a moment-by-moment basis.
Collapse
Affiliation(s)
- HyungGoo R Kim
- Center for Brain Science, Department of Molecular and Cellular Biology, Harvard University, 16 Divinity Avenue, Cambridge, MA 02138, USA.
| | - Athar N Malik
- Center for Brain Science, Department of Molecular and Cellular Biology, Harvard University, 16 Divinity Avenue, Cambridge, MA 02138, USA; Department of Neurosurgery, Massachusetts General Hospital, 55 Fruit Street, Boston, MA 02114, USA
| | - John G Mikhael
- Program in Neuroscience, Harvard Medical School, 220 Longwood Avenue, Boston, MA 02115, USA; MD-PhD Program, Harvard Medical School, 260 Longwood Avenue, Boston, MA 02115, USA
| | - Pol Bech
- Center for Brain Science, Department of Molecular and Cellular Biology, Harvard University, 16 Divinity Avenue, Cambridge, MA 02138, USA
| | - Iku Tsutsui-Kimura
- Center for Brain Science, Department of Molecular and Cellular Biology, Harvard University, 16 Divinity Avenue, Cambridge, MA 02138, USA
| | - Fangmiao Sun
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China; Peking-Tsinghua Center for Life Sciences, Beijing 100871, China; PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
| | - Yajun Zhang
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China; Peking-Tsinghua Center for Life Sciences, Beijing 100871, China; PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
| | - Yulong Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China; Peking-Tsinghua Center for Life Sciences, Beijing 100871, China; PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
| | - Mitsuko Watabe-Uchida
- Center for Brain Science, Department of Molecular and Cellular Biology, Harvard University, 16 Divinity Avenue, Cambridge, MA 02138, USA
| | - Samuel J Gershman
- Department of Psychology, Center for Brain Science, Harvard University, 52 Oxford Street, Cambridge, MA 02138, USA
| | - Naoshige Uchida
- Center for Brain Science, Department of Molecular and Cellular Biology, Harvard University, 16 Divinity Avenue, Cambridge, MA 02138, USA.
| |
Collapse
|
30
|
Eschbach C, Zlatic M. Useful road maps: studying Drosophila larva's central nervous system with the help of connectomics. Curr Opin Neurobiol 2020; 65:129-137. [PMID: 33242722 PMCID: PMC7773133 DOI: 10.1016/j.conb.2020.09.008] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2020] [Revised: 09/21/2020] [Accepted: 09/24/2020] [Indexed: 12/16/2022]
Abstract
The larva of Drosophila melanogaster is emerging as a powerful model system for comprehensive brain-wide understanding of the circuit implementation of neural computations. With an unprecedented amount of tools in hand, including synaptic-resolution connectomics, whole-brain imaging, and genetic tools for selective targeting of single neuron types, it is possible to dissect which circuits and computations are at work behind behaviors that have an interesting level of complexity. Here we present some of the recent advances regarding multisensory integration, learning, and action selection in Drosophila larva.
Collapse
Affiliation(s)
- Claire Eschbach
- Department of Zoology, University of Cambridge, United Kingdom.
| | - Marta Zlatic
- Department of Zoology, University of Cambridge, United Kingdom; MRC Laboratory of Molecular Biology, United Kingdom.
| |
Collapse
|
31
|
Lowet AS, Zheng Q, Matias S, Drugowitsch J, Uchida N. Distributional Reinforcement Learning in the Brain. Trends Neurosci 2020; 43:980-997. [PMID: 33092893 PMCID: PMC8073212 DOI: 10.1016/j.tins.2020.09.004] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2020] [Revised: 08/14/2020] [Accepted: 09/08/2020] [Indexed: 12/11/2022]
Abstract
Learning about rewards and punishments is critical for survival. Classical studies have demonstrated an impressive correspondence between the firing of dopamine neurons in the mammalian midbrain and the reward prediction errors of reinforcement learning algorithms, which express the difference between actual reward and predicted mean reward. However, it may be advantageous to learn not only the mean but also the complete distribution of potential rewards. Recent advances in machine learning have revealed a biologically plausible set of algorithms for reconstructing this reward distribution from experience. Here, we review the mathematical foundations of these algorithms as well as initial evidence for their neurobiological implementation. We conclude by highlighting outstanding questions regarding the circuit computation and behavioral readout of these distributional codes.
Collapse
Affiliation(s)
- Adam S Lowet
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
| | - Qiao Zheng
- Department of Neurobiology, Harvard Medical School, Boston, MA 02115, USA
| | - Sara Matias
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
| | - Jan Drugowitsch
- Department of Neurobiology, Harvard Medical School, Boston, MA 02115, USA.
| | - Naoshige Uchida
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA 02138, USA.
| |
Collapse
|
32
|
Midbrain circuits of novelty processing. Neurobiol Learn Mem 2020; 176:107323. [PMID: 33053429 DOI: 10.1016/j.nlm.2020.107323] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2020] [Revised: 09/22/2020] [Accepted: 10/02/2020] [Indexed: 12/22/2022]
Abstract
Novelty triggers an increase in orienting behavior that is critical to evaluate the potential salience of unknown events. As novelty becomes familiar upon repeated encounters, this increase in response rapidly habituates as a form of behavioral adaptation underlying goal-directed behaviors. Many neurodevelopmental, psychiatric and neurodegenerative disorders are associated with abnormal responses to novelty and/or familiarity, although the neuronal circuits and cellular/molecular mechanisms underlying these natural behaviors in the healthy brain are largely unknown, as is the maladaptive processes that occur to induce impairment of novelty signaling in diseased brains. In rodents, the development of cutting-edge tools that allow for measurements of real time activity dynamics in selectively identified neuronal ensembles by gene expression signatures is beginning to provide advances in understanding the neural bases of the novelty response. Accumulating evidence indicate that midbrain circuits, the majority of which linked to dopamine transmission, promote exploratory assessments and guide approach/avoidance behaviors to different types of novelty via specific projection sites. The present review article focuses on midbrain circuit analysis relevant to novelty processing and habituation with familiarity.
Collapse
|
33
|
Collins AL, Saunders BT. Heterogeneity in striatal dopamine circuits: Form and function in dynamic reward seeking. J Neurosci Res 2020; 98:1046-1069. [PMID: 32056298 PMCID: PMC7183907 DOI: 10.1002/jnr.24587] [Citation(s) in RCA: 46] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2019] [Revised: 01/08/2020] [Accepted: 01/16/2020] [Indexed: 01/03/2023]
Abstract
The striatal dopamine system has long been studied in the context of reward learning, motivation, and movement. Given the prominent role dopamine plays in a variety of adaptive behavioral states, as well as diseases like addiction, it is essential to understand the full complexity of dopamine neurons and the striatal systems they target. A growing number of studies are uncovering details of the heterogeneity in dopamine neuron subpopulations. Here, we review that work to synthesize current understanding of dopamine system heterogeneity across three levels, anatomical organization, functions in behavior, and modes of action, wherein we focus on signaling profiles and local mechanisms for modulation of dopamine release. Together, these studies reveal new and emerging dimensions of the striatal dopamine system, informing its contribution to dynamic motivational and decision-making processes.
Collapse
Affiliation(s)
- Anne L. Collins
- University of Minnesota, Department of Neuroscience, Medical Discovery Team on Addiction, Minneapolis, MN 55455
| | - Benjamin T. Saunders
- University of Minnesota, Department of Neuroscience, Medical Discovery Team on Addiction, Minneapolis, MN 55455
| |
Collapse
|
34
|
Steinberg EE, Gore F, Heifets BD, Taylor MD, Norville ZC, Beier KT, Földy C, Lerner TN, Luo L, Deisseroth K, Malenka RC. Amygdala-Midbrain Connections Modulate Appetitive and Aversive Learning. Neuron 2020; 106:1026-1043.e9. [PMID: 32294466 DOI: 10.1016/j.neuron.2020.03.016] [Citation(s) in RCA: 48] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2019] [Revised: 02/03/2020] [Accepted: 03/18/2020] [Indexed: 01/28/2023]
Abstract
The central amygdala (CeA) orchestrates adaptive responses to emotional events. While CeA substrates for defensive behaviors have been studied extensively, CeA circuits for appetitive behaviors and their relationship to threat-responsive circuits remain poorly defined. Here, we demonstrate that the CeA sends robust inhibitory projections to the lateral substantia nigra (SNL) that contribute to appetitive and aversive learning in mice. CeA→SNL neural responses to appetitive and aversive stimuli were modulated by expectation and magnitude consistent with a population-level salience signal, which was required for Pavlovian conditioned reward-seeking and defensive behaviors. CeA→SNL terminal activation elicited reinforcement when linked to voluntary actions but failed to support Pavlovian associations that rely on incentive value signals. Consistent with a disinhibitory mechanism, CeA inputs preferentially target SNL GABA neurons, and CeA→SNL and SNL dopamine neurons respond similarly to salient stimuli. Collectively, our results suggest that amygdala-nigra interactions represent a previously unappreciated mechanism for influencing emotional behaviors.
Collapse
Affiliation(s)
- Elizabeth E Steinberg
- Nancy Pritzker Laboratory, Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford, CA 94305, USA
| | - Felicity Gore
- Nancy Pritzker Laboratory, Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford, CA 94305, USA; Departments of Bioengineering and Psychiatry and Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA
| | - Boris D Heifets
- Nancy Pritzker Laboratory, Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford, CA 94305, USA; Department of Anesthesiology, Stanford University, Stanford, CA 94305, USA
| | - Madison D Taylor
- Nancy Pritzker Laboratory, Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford, CA 94305, USA
| | - Zane C Norville
- Nancy Pritzker Laboratory, Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford, CA 94305, USA
| | - Kevin T Beier
- Nancy Pritzker Laboratory, Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford, CA 94305, USA; Department of Biology and Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA
| | - Csaba Földy
- Nancy Pritzker Laboratory, Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford, CA 94305, USA; Brain Research Institute, University of Zurich, Zurich, Switzerland
| | - Talia N Lerner
- Departments of Bioengineering and Psychiatry and Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA
| | - Liqun Luo
- Department of Biology and Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA
| | - Karl Deisseroth
- Departments of Bioengineering and Psychiatry and Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA
| | - Robert C Malenka
- Nancy Pritzker Laboratory, Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford, CA 94305, USA.
| |
Collapse
|
35
|
Verharen JPH, Zhu Y, Lammel S. Aversion hot spots in the dopamine system. Curr Opin Neurobiol 2020; 64:46-52. [PMID: 32146296 DOI: 10.1016/j.conb.2020.02.002] [Citation(s) in RCA: 39] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2019] [Revised: 02/02/2020] [Accepted: 02/03/2020] [Indexed: 01/06/2023]
Abstract
Through the development of optogenetics and other viral vector-based technologies, our view of the dopamine system has substantially advanced over the last decade. In particular, progress has been made in the reclassification of dopamine neurons based on subtypes displaying specific projections, which are associated with different features at the anatomical, molecular and behavioral level. Together, these discoveries have raised the possibility that individual groups of dopamine cells make a unique contribution to the processing of reward and aversion. Here, we review recent studies that have identified non-canonical dopamine pathways that are excited in response to aversive stimuli, including dopamine projections to the ventromedial shell of the nucleus accumbens, prefrontal cortex, tail of the striatum, and amygdala.
Collapse
Affiliation(s)
- J P H Verharen
- Department of Molecular and Cell Biology and Helen Wills Neuroscience Institute, University of California Berkeley, USA
| | - Yichen Zhu
- Department of Molecular and Cell Biology and Helen Wills Neuroscience Institute, University of California Berkeley, USA
| | - Stephan Lammel
- Department of Molecular and Cell Biology and Helen Wills Neuroscience Institute, University of California Berkeley, USA.
| |
Collapse
|
36
|
Kepecs A. Summary: Order and Disorder in Brains and Behavior. COLD SPRING HARBOR SYMPOSIA ON QUANTITATIVE BIOLOGY 2019; 83:219-225. [PMID: 31358660 DOI: 10.1101/sqb.2018.83.038885] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
The 83rd Cold Spring Harbor Symposium on Quantitative Biology on Brains and Behavior: Order and Disorder in the Nervous System explored the tremendous recent progress in neuroscience and how these advances may be used to improve brain health and address psychiatric and neurological disorders. The Symposium explored a vast array of topics from cell types to cognition. My summary focuses on a few emerging themes. Innovative techniques were ever-present, opening up new experimental possibilities. The commoditization of many state-of-the-art technologies is pushing neuroscience beyond its artisanal ways. Another important theme was "circuits in the middle": Numerous presentations dissected cell type-specific circuits that connect different levels of analysis from molecules to behavior. These new technologies have enabled curiosity-driven investigations in animals to connect more directly with preclinical and clinical studies of human brain disorders. Numerous emerging approaches were presented in human neuroscience, bolstering the hope that circuit-specific manipulations will soon provide improved treatments for brain disorders.
Collapse
Affiliation(s)
- Adam Kepecs
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York 11724, USA
| |
Collapse
|