1
|
Goyal A, Yuen J, Sinicrope S, Winter B, Randall L, Rusheen AE, Blaha CD, Bennet KE, Lee KH, Shin H, Oh Y. Resolution of tonic concentrations of highly similar neurotransmitters using voltammetry and deep learning. Mol Psychiatry 2024; 29:3076-3085. [PMID: 38664492 PMCID: PMC11449650 DOI: 10.1038/s41380-024-02537-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Revised: 03/11/2024] [Accepted: 03/15/2024] [Indexed: 06/27/2024]
Abstract
With advances in our understanding regarding the neurochemical underpinnings of neurological and psychiatric diseases, there is an increased demand for advanced computational methods for neurochemical analysis. Despite having a variety of techniques for measuring tonic extracellular concentrations of neurotransmitters, including voltammetry, enzyme-based sensors, amperometry, and in vivo microdialysis, there is currently no means to resolve concentrations of structurally similar neurotransmitters from mixtures in the in vivo environment with high spatiotemporal resolution and limited tissue damage. Since a variety of research and clinical investigations involve brain regions containing electrochemically similar monoamines, such as dopamine and norepinephrine, developing a model to resolve the respective contributions of these neurotransmitters is of vital importance. Here we have developed a deep learning network, DiscrimNet, a convolutional autoencoder capable of accurately predicting individual tonic concentrations of dopamine, norepinephrine, and serotonin from both in vitro mixtures and the in vivo environment in anesthetized rats, measured using voltammetry. The architecture of DiscrimNet is described, and its ability to accurately predict in vitro and unseen in vivo concentrations is shown to vastly outperform a variety of shallow learning algorithms previously used for neurotransmitter discrimination. DiscrimNet is shown to generalize well to data captured from electrodes unseen during model training, eliminating the need to retrain the model for each new electrode. DiscrimNet is also shown to accurately predict the expected changes in dopamine and serotonin after cocaine and oxycodone administration in anesthetized rats in vivo. DiscrimNet therefore offers an exciting new method for real-time resolution of in vivo voltammetric signals into component neurotransmitters.
Collapse
Affiliation(s)
- Abhinav Goyal
- Medical Scientist Training Program, Mayo Clinic, Rochester, MN, 55905, USA
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, 55905, USA
| | - Jason Yuen
- Department of Neurosurgery, Southmead Hospital, Bristol, BS10 5NB, UK
| | - Stephen Sinicrope
- Department of Neuroscience, University of Chicago, Chicago, IL, 60637, USA
| | - Bailey Winter
- Medical Scientist Training Program, Mayo Clinic, Rochester, MN, 55905, USA
| | - Lindsey Randall
- Medical Scientist Training Program, Mayo Clinic, Rochester, MN, 55905, USA
| | - Aaron E Rusheen
- Department of Neurosurgery, Johns Hopkins University, Baltimore, MD, 21287, USA
| | - Charles D Blaha
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, 55905, USA
| | - Kevin E Bennet
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, 55905, USA
- Division of Engineering, Mayo Clinic, Rochester, MN, 55905, USA
| | - Kendall H Lee
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, 55905, USA
- Department of Biomedical Engineering, Mayo Clinic, Rochester, MN, 55905, USA
| | - Hojin Shin
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, 55905, USA.
- Department of Biomedical Engineering, Mayo Clinic, Rochester, MN, 55905, USA.
| | - Yoonbae Oh
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, 55905, USA.
- Department of Biomedical Engineering, Mayo Clinic, Rochester, MN, 55905, USA.
| |
Collapse
|
2
|
Goyal A, Karanovic U, Blaha CD, Lee KH, Shin H, Oh Y. Toward Precise Modeling of Dopamine Release Kinetics: Comparison and Validation of Kinetic Models Using Voltammetry. ACS OMEGA 2024; 9:33563-33573. [PMID: 39130585 PMCID: PMC11307285 DOI: 10.1021/acsomega.4c01322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/09/2024] [Revised: 07/12/2024] [Accepted: 07/18/2024] [Indexed: 08/13/2024]
Abstract
Dopamine (DA) is a neurotransmitter present within the animal brain that is responsible for a wide range of physiologic functions, including motivation, reward, and movement control. Changes or dysfunction in the dynamics of DA release are thought to play a pivotal role in regulating various physiological and behavioral processes, as well as leading to neuropsychiatric diseases. Therefore, it is of fundamental interest to neuroscientists to understand and accurately model the kinetics that govern dopaminergic neurotransmission. In the past several decades, many mathematical models have been proposed to attempt to capture the biologic parameters that govern dopaminergic kinetics, with each model seeking to improve upon a previous model. In this review, each of these models are derived, and the ability of each model to properly fit two fast-scan cyclic voltammetry (FSCV) data sets will be demonstrated and discussed. The dopamine oxidation current in both FSCV data sets exhibits hang-up and overshoot behaviors, which have traditionally been difficult for mathematical models to capture. We show that more recent models are better able to model DA release that exhibits these behaviors but that no single model is clearly the best. Rather, models should be selected based on their mathematical properties to best fit the FSCV data one is trying to model. Developing such differential equation models to describe the kinetics of DA release from the synapse confers significant applications both for advancing scientific understanding of DA neurotransmission and for advancing clinical ability to treat neuropsychiatric diseases.
Collapse
Affiliation(s)
- Abhinav Goyal
- Mayo
Clinic Medical Scientist Training Program, Mayo Clinic, Rochester, Minnesota 55905, United States
- Department
of Neurologic Surgery, Mayo Clinic, Rochester, Minnesota 55905, United States
| | - Una Karanovic
- Department
of Neurologic Surgery, Mayo Clinic, Rochester, Minnesota 55905, United States
- Mayo
Clinic Graduate School of Biomedical Sciences, Mayo Clinic, Rochester, Minnesota 55905, United States
| | - Charles D. Blaha
- Department
of Neurologic Surgery, Mayo Clinic, Rochester, Minnesota 55905, United States
| | - Kendall H. Lee
- Department
of Neurologic Surgery, Mayo Clinic, Rochester, Minnesota 55905, United States
- Department
of Biomedical Engineering, Mayo Clinic, Rochester, Minnesota 55905, United States
| | - Hojin Shin
- Department
of Neurologic Surgery, Mayo Clinic, Rochester, Minnesota 55905, United States
- Department
of Biomedical Engineering, Mayo Clinic, Rochester, Minnesota 55905, United States
| | - Yoonbae Oh
- Department
of Neurologic Surgery, Mayo Clinic, Rochester, Minnesota 55905, United States
- Department
of Biomedical Engineering, Mayo Clinic, Rochester, Minnesota 55905, United States
| |
Collapse
|
3
|
Song MR, Lee SW. Rethinking dopamine-guided action sequence learning. Eur J Neurosci 2024; 60:3447-3465. [PMID: 38798086 DOI: 10.1111/ejn.16426] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Revised: 04/21/2024] [Accepted: 05/08/2024] [Indexed: 05/29/2024]
Abstract
As opposed to those requiring a single action for reward acquisition, tasks necessitating action sequences demand that animals learn action elements and their sequential order and sustain the behaviour until the sequence is completed. With repeated learning, animals not only exhibit precise execution of these sequences but also demonstrate enhanced smoothness and efficiency. Previous research has demonstrated that midbrain dopamine and its major projection target, the striatum, play crucial roles in these processes. Recent studies have shown that dopamine from the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA) serve distinct functions in action sequence learning. The distinct contributions of dopamine also depend on the striatal subregions, namely the ventral, dorsomedial and dorsolateral striatum. Here, we have reviewed recent findings on the role of striatal dopamine in action sequence learning, with a focus on recent rodent studies.
Collapse
Affiliation(s)
- Minryung R Song
- Department of Brain and Cognitive Sciences, KAIST, Daejeon, South Korea
| | - Sang Wan Lee
- Department of Brain and Cognitive Sciences, KAIST, Daejeon, South Korea
- Kim Jaechul Graduate School of AI, KAIST, Daejeon, South Korea
- KI for Health Science and Technology, KAIST, Daejeon, South Korea
- Center for Neuroscience-inspired AI, KAIST, Daejeon, South Korea
| |
Collapse
|
4
|
Phillips CD, Hodge AT, Myers CC, Leventhal DK, Burgess CR. Striatal Dopamine Contributions to Skilled Motor Learning. J Neurosci 2024; 44:e0240242024. [PMID: 38806248 PMCID: PMC11211718 DOI: 10.1523/jneurosci.0240-24.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2024] [Revised: 05/15/2024] [Accepted: 05/21/2024] [Indexed: 05/30/2024] Open
Abstract
Coordinated multijoint limb and digit movements-"manual dexterity"-underlie both specialized skills (e.g., playing the piano) and more mundane tasks (e.g., tying shoelaces). Impairments in dexterous skill cause significant disability, as occurs with motor cortical injury, Parkinson's disease, and a range of other pathologies. Clinical observations, as well as basic investigations, suggest that corticostriatal circuits play a critical role in learning and performing dexterous skills. Furthermore, dopaminergic signaling in these regions is implicated in synaptic plasticity and motor learning. Nonetheless, the role of striatal dopamine signaling in skilled motor learning remains poorly understood. Here, we use fiber photometry paired with a genetically encoded dopamine sensor to investigate striatal dopamine release in both male and female mice as they learn and perform a skilled reaching task. Dopamine rapidly increases during a skilled reach and peaks near pellet consumption. In the dorsolateral striatum, dopamine dynamics are faster than in the dorsomedial and ventral striatum. Across training, as reaching performance improves, dopamine signaling shifts from pellet consumption to cues that predict pellet availability, particularly in medial and ventral areas of the striatum. Furthermore, performance prediction errors are present across the striatum, with reduced dopamine release after an unsuccessful reach. These findings show that dopamine dynamics during skilled motor behaviors change with learning and are differentially regulated across striatal subregions.
Collapse
Affiliation(s)
- Chris D Phillips
- Michigan Neuroscience Institute, University of Michigan, Ann Arbor, Michigan 48109
- Department of Molecular and Integrative Physiology, University of Michigan, Ann Arbor, Michigan 48109
- Department of Neuroscience, University of Texas at Dallas, Richardson, Texas 75080
| | - Alexander T Hodge
- Department of Neurology, University of Michigan, Ann Arbor, Michigan 48109
| | - Courtney C Myers
- Michigan Neuroscience Institute, University of Michigan, Ann Arbor, Michigan 48109
- Neuroscience Graduate Program, University of Michigan, Ann Arbor, Michigan 48109
| | - Daniel K Leventhal
- Michigan Neuroscience Institute, University of Michigan, Ann Arbor, Michigan 48109
- Department of Neurology, University of Michigan, Ann Arbor, Michigan 48109
- Neuroscience Graduate Program, University of Michigan, Ann Arbor, Michigan 48109
- Parkinson's Disease Foundation Research Center of Excellence, University of Michigan, Ann Arbor, Michigan 48109
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, Michigan 48109
- Department of Neurology, VA Ann Arbor Health System, Ann Arbor, Michigan 48109
| | - Christian R Burgess
- Michigan Neuroscience Institute, University of Michigan, Ann Arbor, Michigan 48109
- Department of Molecular and Integrative Physiology, University of Michigan, Ann Arbor, Michigan 48109
- Neuroscience Graduate Program, University of Michigan, Ann Arbor, Michigan 48109
| |
Collapse
|
5
|
Montgomery SE, Li L, Russo SJ, Calipari ES, Nestler EJ, Morel C, Han MH. Mesolimbic Neural Response Dynamics Predict Future Individual Alcohol Drinking in Mice. Biol Psychiatry 2024; 95:951-962. [PMID: 38061466 DOI: 10.1016/j.biopsych.2023.11.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 11/11/2023] [Accepted: 11/14/2023] [Indexed: 01/27/2024]
Abstract
BACKGROUND Individual variability in response to rewarding stimuli is a striking but understudied phenomenon. The mesolimbic dopamine system is critical in encoding the reinforcing properties of both natural reward and alcohol; however, how innate or baseline differences in the response dynamics of this circuit define individual behavior and shape future vulnerability to alcohol remain unknown. METHODS Using naturalistic behavioral assays, a voluntary alcohol drinking paradigm, in vivo fiber photometry, in vivo electrophysiology, and chemogenetics, we investigated how differences in mesolimbic neural circuit activity contribute to the individual variability seen in reward processing and, by proxy, alcohol drinking. RESULTS We first characterized heterogeneous behavioral and neural responses to natural reward and defined how these baseline responses predicted future individual alcohol-drinking phenotypes in male mice. We then determined spontaneous ventral tegmental area dopamine neuron firing profiles associated with responses to natural reward that predicted alcohol drinking. Using a dual chemogenetic approach, we mimicked specific mesolimbic dopamine neuron firing activity before or during voluntary alcohol drinking to link unique neurophysiological profiles to individual phenotype. We show that hyperdopaminergic individuals exhibit a lower neuronal response to both natural reward and alcohol that predicts lower levels of alcohol consumption in the future. CONCLUSIONS These findings reveal unique, circuit-specific neural signatures that predict future individual vulnerability or resistance to alcohol and expand the current knowledge base on how some individuals are able to titrate their alcohol consumption whereas others go on to engage in unhealthy alcohol-drinking behaviors.
Collapse
Affiliation(s)
- Sarah E Montgomery
- Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York; Friedman Brain Institute and the Center for Affective Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York; Department of Pharmacological Sciences, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Long Li
- Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York; Friedman Brain Institute and the Center for Affective Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Scott J Russo
- Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York; Friedman Brain Institute and the Center for Affective Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Erin S Calipari
- Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York; Friedman Brain Institute and the Center for Affective Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York; Departments of Pharmacology, Molecular Physiology and Biophysics, and Psychiatry and Behavioral Sciences, Vanderbilt Center for Addiction Research, Vanderbilt Brain Institute, Vanderbilt University, Nashville, Tennessee
| | - Eric J Nestler
- Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York; Friedman Brain Institute and the Center for Affective Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York; Department of Pharmacological Sciences, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Carole Morel
- Department of Pharmacological Sciences, Icahn School of Medicine at Mount Sinai, New York, New York.
| | - Ming-Hu Han
- Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York; Friedman Brain Institute and the Center for Affective Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York; Department of Pharmacological Sciences, Icahn School of Medicine at Mount Sinai, New York, New York; Department of Mental Health and Public Health, Faculty of Life and Health Sciences, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, Guangdong, China.
| |
Collapse
|
6
|
Aster HC, Waltmann M, Busch A, Romanos M, Gamer M, Maria van Noort B, Beck A, Kappel V, Deserno L. Impaired flexible reward learning in ADHD patients is associated with blunted reinforcement sensitivity and neural signals in ventral striatum and parietal cortex. Neuroimage Clin 2024; 42:103588. [PMID: 38471434 PMCID: PMC10943992 DOI: 10.1016/j.nicl.2024.103588] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 02/06/2024] [Accepted: 02/28/2024] [Indexed: 03/14/2024]
Abstract
Reward-based learning and decision-making are prime candidates to understand symptoms of attention deficit hyperactivity disorder (ADHD). However, only limited evidence is available regarding the neurocomputational underpinnings of the alterations seen in ADHD. This concerns flexible behavioral adaption in dynamically changing environments, which is challenging for individuals with ADHD. One previous study points to elevated choice switching in adolescent ADHD, which was accompanied by disrupted learning signals in medial prefrontal cortex. Here, we investigated young adults with ADHD (n = 17) as compared to age- and sex-matched controls (n = 17) using a probabilistic reversal learning experiment during functional magnetic resonance imaging (fMRI). The task requires continuous learning to guide flexible behavioral adaptation to changing reward contingencies. To disentangle the neurocomputational underpinnings of the behavioral data, we used reinforcement learning (RL) models, which informed the analysis of fMRI data. ADHD patients performed worse than controls particularly in trials before reversals, i.e., when reward contingencies were stable. This pattern resulted from 'noisy' choice switching regardless of previous feedback. RL modelling showed decreased reinforcement sensitivity and enhanced learning rates for negative feedback in ADHD patients. At the neural level, this was reflected in a diminished representation of choice probability in the left posterior parietal cortex in ADHD. Moreover, modelling showed a marginal reduction of learning about the unchosen option, which was paralleled by a marginal reduction in learning signals incorporating the unchosen option in the left ventral striatum. Taken together, we show that impaired flexible behavior in ADHD is due to excessive choice switching ('hyper-flexibility'), which can be detrimental or beneficial depending on the learning environment. Computationally, this resulted from blunted sensitivity to reinforcement of which we detected neural correlates in the attention-control network, specifically in the parietal cortex. These neurocomputational findings remain preliminary due to the relatively small sample size.
Collapse
Affiliation(s)
- Hans-Christoph Aster
- Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University Hospital Würzburg, Würzburg, Germany.
| | - Maria Waltmann
- Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University Hospital Würzburg, Würzburg, Germany; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Anika Busch
- Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University Hospital Würzburg, Würzburg, Germany
| | - Marcel Romanos
- Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University Hospital Würzburg, Würzburg, Germany
| | - Matthias Gamer
- Department of Psychology, University of Würzburg, Würzburg, Germany
| | - Betteke Maria van Noort
- Department of Child and Adolescent Psychiatry, Charité University Medicine, Campus Virchow Klinikum, Berlin, Germany; MSB Medical School Berlin, Department of Psychology, Germany
| | - Anne Beck
- Department of Psychiatry and Neurosciences, Charité University Medicine, Berlin, Germany; Department of Psychology, Faculty of Health, Health and Medical University, Potsdam, Germany
| | - Viola Kappel
- Department of Child and Adolescent Psychiatry, Charité University Medicine, Campus Virchow Klinikum, Berlin, Germany
| | - Lorenz Deserno
- Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University Hospital Würzburg, Würzburg, Germany; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany; Department of Psychiatry and Psychotherapy, Technische Universität Dresden, Dresden, Germany
| |
Collapse
|
7
|
Phillips CD, Myers CC, Leventhal DK, Burgess CR. Striatal dopamine contributions to skilled motor learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.02.06.579240. [PMID: 38370850 PMCID: PMC10871330 DOI: 10.1101/2024.02.06.579240] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/20/2024]
Abstract
Coordinated multi-joint limb and digit movements - "manual dexterity" - underlie both specialized skills (e.g., playing the piano) and more mundane tasks (e.g., tying shoelaces). Impairments in dexterous skill cause significant disability, as occurs with motor cortical injury, Parkinson's Disease, and a range of other pathologies. Clinical observations, as well as basic investigations, suggest that cortico-striatal circuits play a critical role in learning and performing dexterous skills. Furthermore, dopaminergic signaling in these regions is implicated in synaptic plasticity and motor learning. Nonetheless, the role of striatal dopamine signaling in skilled motor learning remains poorly understood. Here, we use fiber photometry paired with a genetically encoded dopamine sensor to investigate striatal dopamine release as mice learn and perform a skilled reaching task. Dopamine rapidly increases during a skilled reach and peaks near pellet consumption. In dorsolateral striatum, dopamine dynamics are faster than in dorsomedial and ventral striatum. Across training, as reaching performance improves, dopamine signaling shifts from pellet consumption to cues that predict pellet availability, particularly in medial and ventral areas of striatum. Furthermore, performance prediction errors are present across the striatum, with reduced dopamine release after an unsuccessful reach. These findings show that dopamine dynamics during skilled motor behaviors change with learning and are differentially regulated across striatal subregions.
Collapse
Affiliation(s)
- Chris D. Phillips
- Michigan Neuroscience Institute, University of Michigan, Ann Arbor, MI, USA, 48109
- Department of Molecular and Integrative Physiology, University of Michigan, Ann Arbor, MI, USA, 48109
- Department of Neuroscience, University of Texas at Dallas, Richardson, TX, USA, 75080
| | - Courtney C. Myers
- Michigan Neuroscience Institute, University of Michigan, Ann Arbor, MI, USA, 48109
- Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, USA, 48109
| | - Daniel K. Leventhal
- Department of Neurology, University of Michigan, Ann Arbor, MI, USA, 48109
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA, 48109
- Parkinson Disease Foundation Research Center of Excellence, University of Michigan, Ann Arbor, MI, USA, 48109
- Department of Neurology, VA Ann Arbor Health System, Ann Arbor, MI, USA, 48109
| | - Christian R. Burgess
- Michigan Neuroscience Institute, University of Michigan, Ann Arbor, MI, USA, 48109
- Department of Molecular and Integrative Physiology, University of Michigan, Ann Arbor, MI, USA, 48109
- Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, USA, 48109
| |
Collapse
|
8
|
Gillespie B, Houghton MJ, Ganio K, McDevitt CA, Bennett D, Dunn A, Raju S, Schroeder A, Hill RA, Cardoso BR. Maternal selenium dietary supplementation alters sociability and reinforcement learning deficits induced by in utero exposure to maternal immune activation in mice. Brain Behav Immun 2024; 116:349-361. [PMID: 38142918 DOI: 10.1016/j.bbi.2023.12.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Revised: 11/24/2023] [Accepted: 12/19/2023] [Indexed: 12/26/2023] Open
Abstract
Maternal immune activation (MIA) during pregnancy increases the risk for the unborn foetus to develop neurodevelopmental conditions such as autism spectrum disorder and schizophrenia later in life. MIA mouse models recapitulate behavioural and biological phenotypes relevant to both conditions, and are valuable models to test novel treatment approaches. Selenium (Se) has potent anti-inflammatory properties suggesting it may be an effective prophylactic treatment against MIA. The aim of this study was to determine if Se supplementation during pregnancy can prevent adverse effects of MIA on offspring brain and behaviour in a mouse model. Selenium was administered via drinking water (1.5 ppm) to pregnant dams from gestational day (GD) 9 to birth, and MIA was induced at GD17 using polyinosinic:polycytidylic acid (poly-I:C, 20 mg/kg via intraperitoneal injection). Foetal placenta and brain cytokine levels were assessed using a Luminex assay and brain elemental nutrients assessed using inductively coupled plasma- mass spectrometry. Adult offspring were behaviourally assessed using a reinforcement learning paradigm, the three-chamber sociability test and the open field test. MIA elevated placental IL-1β and IL-17, and Se supplementation successfully prevented this elevation. MIA caused an increase in foetal brain calcium, which was prevented by Se supplement. MIA caused in offspring a female-specific reduction in sociability, which was recovered by Se, and a male-specific reduction in social memory, which was not recovered by Se. Exposure to poly-I:C or selenium, but not both, reduced performance in the reinforcement learning task. Computational modelling indicated that this was predominantly due to increased exploratory behaviour, rather than reduced rate of learning the location of the food reward. This study demonstrates that while Se may be beneficial in ameliorating sociability deficits caused by MIA, it may have negative effects in other behavioural domains. Caution in the use of Se supplementation during pregnancy is therefore warranted.
Collapse
Affiliation(s)
- Brendan Gillespie
- Department of Psychiatry, Monash University, Clayton, VIC 3168, Australia
| | - Michael J Houghton
- Department of Nutrition, Dietetics and Food, Monash University, Notting Hill, VIC 3168, Australia; Victorian Heart Institute, Monash University, Clayton, VIC 3168, Australia
| | - Katherine Ganio
- Department of Microbiology and Immunology, The Peter Doherty Institute for Infection and Immunity, The University of Melbourne, Melbourne, VIC 3000, Australia
| | - Christopher A McDevitt
- Department of Microbiology and Immunology, The Peter Doherty Institute for Infection and Immunity, The University of Melbourne, Melbourne, VIC 3000, Australia
| | - Daniel Bennett
- Department of Psychiatry, Monash University, Clayton, VIC 3168, Australia
| | - Ariel Dunn
- Department of Psychiatry, Monash University, Clayton, VIC 3168, Australia
| | - Sharvada Raju
- Department of Psychiatry, Monash University, Clayton, VIC 3168, Australia
| | - Anna Schroeder
- Department of Psychiatry, Monash University, Clayton, VIC 3168, Australia.
| | - Rachel A Hill
- Department of Psychiatry, Monash University, Clayton, VIC 3168, Australia.
| | - Barbara R Cardoso
- Department of Nutrition, Dietetics and Food, Monash University, Notting Hill, VIC 3168, Australia.
| |
Collapse
|
9
|
Salinas AG, Lee JO, Augustin SM, Zhang S, Patriarchi T, Tian L, Morales M, Mateo Y, Lovinger DM. Distinct sub-second dopamine signaling in dorsolateral striatum measured by a genetically-encoded fluorescent sensor. Nat Commun 2023; 14:5915. [PMID: 37739964 PMCID: PMC10517008 DOI: 10.1038/s41467-023-41581-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2022] [Accepted: 09/06/2023] [Indexed: 09/24/2023] Open
Abstract
The development of genetically encoded dopamine sensors such as dLight has provided a new approach to measuring slow and fast dopamine dynamics both in brain slices and in vivo, possibly enabling dopamine measurements in areas like the dorsolateral striatum (DLS) where previously such recordings with fast-scan cyclic voltammetry (FSCV) were difficult. To test this, we first evaluated dLight photometry in mouse brain slices with simultaneous FSCV and found that both techniques yielded comparable results, but notable differences in responses to dopamine transporter inhibitors, including cocaine. We then used in vivo fiber photometry with dLight in mice to examine responses to cocaine in DLS. We also compared dopamine responses during Pavlovian conditioning across the striatum. We show that dopamine increases were readily detectable in DLS and describe transient dopamine kinetics, as well as slowly developing signals during conditioning. Overall, our findings indicate that dLight photometry is well suited to measuring dopamine dynamics in DLS.
Collapse
Affiliation(s)
- Armando G Salinas
- Laboratory for Integrative Neuroscience, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Rockville, MD, USA.
- Department of Bioengineering, George Mason University, Fairfax, VA, USA.
- Department of Pharmacology, Toxicology & Neuroscience, Louisiana State University Health Sciences Center - Shreveport, Shreveport, LA, USA.
| | - Jeong Oen Lee
- Laboratory for Integrative Neuroscience, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Rockville, MD, USA
| | - Shana M Augustin
- Laboratory for Integrative Neuroscience, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Rockville, MD, USA
- Department of Pharmacology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Shiliang Zhang
- Confocal and Electron Microscopy Core, National Institute on Drug Abuse, Baltimore, MD, USA
| | - Tommaso Patriarchi
- Department of Biochemistry and Molecular Medicine, University of California at Davis, Davis, CA, USA
- Institute of Pharmacology and Toxicology, University of Zurich, Zurich, Switzerland
| | - Lin Tian
- Department of Biochemistry and Molecular Medicine, University of California at Davis, Davis, CA, USA
| | - Marisela Morales
- Neuronal Networks Section, Integrative Neuroscience Research Branch, National Institute on Drug Abuse, Baltimore, MD, USA
| | - Yolanda Mateo
- Laboratory for Integrative Neuroscience, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Rockville, MD, USA
| | - David M Lovinger
- Laboratory for Integrative Neuroscience, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Rockville, MD, USA.
| |
Collapse
|
10
|
Delignat-Lavaud B, Kano J, Ducrot C, Massé I, Mukherjee S, Giguère N, Moquin L, Lévesque C, Burke S, Denis R, Bourque MJ, Tchung A, Rosa-Neto P, Lévesque D, De Beaumont L, Trudeau LÉ. Synaptotagmin-1-dependent phasic axonal dopamine release is dispensable for basic motor behaviors in mice. Nat Commun 2023; 14:4120. [PMID: 37433762 PMCID: PMC10336101 DOI: 10.1038/s41467-023-39805-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Accepted: 06/27/2023] [Indexed: 07/13/2023] Open
Abstract
In Parkinson's disease (PD), motor dysfunctions only become apparent after extensive loss of DA innervation. This resilience has been hypothesized to be due to the ability of many motor behaviors to be sustained through a diffuse basal tone of DA; but experimental evidence for this is limited. Here we show that conditional deletion of the calcium sensor synaptotagmin-1 (Syt1) in DA neurons (Syt1 cKODA mice) abrogates most activity-dependent axonal DA release in the striatum and mesencephalon, leaving somatodendritic (STD) DA release intact. Strikingly, Syt1 cKODA mice showed intact performance in multiple unconditioned DA-dependent motor tasks and even in a task evaluating conditioned motivation for food. Considering that basal extracellular DA levels in the striatum were unchanged, our findings suggest that activity-dependent DA release is dispensable for such tasks and that they can be sustained by a basal tone of extracellular DA. Taken together, our findings reveal the striking resilience of DA-dependent motor functions in the context of a near-abolition of phasic DA release, shedding new light on why extensive loss of DA innervation is required to reveal motor dysfunctions in PD.
Collapse
Affiliation(s)
- Benoît Delignat-Lavaud
- Department of Pharmacology and Physiology, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- Department of Neurosciences, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- SNC and CIRCA Research Groups, Université de Montréal, Montréal, QC, Canada
| | - Jana Kano
- Department of Pharmacology and Physiology, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- Department of Neurosciences, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- SNC and CIRCA Research Groups, Université de Montréal, Montréal, QC, Canada
| | - Charles Ducrot
- Department of Pharmacology and Physiology, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- Department of Neurosciences, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- SNC and CIRCA Research Groups, Université de Montréal, Montréal, QC, Canada
| | - Ian Massé
- Hôpital du Sacré-Cœur-de-Montréal, CIUSSS NIM, Université de Montréal, Montreal, QC, Canada
| | - Sriparna Mukherjee
- Department of Pharmacology and Physiology, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- Department of Neurosciences, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- SNC and CIRCA Research Groups, Université de Montréal, Montréal, QC, Canada
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, 20815, USA
| | - Nicolas Giguère
- Department of Pharmacology and Physiology, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- Department of Neurosciences, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- SNC and CIRCA Research Groups, Université de Montréal, Montréal, QC, Canada
| | - Luc Moquin
- Centre intégré universitaire de santé et de services sociaux (CIUSSS) de l'Ouest-de-l'Île-de-Montréal; Department of Neurology and Neurosurgery, Psychiatry and Pharmacology and Therapeutics, McGill University, Montreal, QC, Canada
| | | | - Samuel Burke
- Department of Pharmacology and Physiology, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- Department of Neurosciences, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- SNC and CIRCA Research Groups, Université de Montréal, Montréal, QC, Canada
| | - Raphaëlle Denis
- Department of Pharmacology and Physiology, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- Department of Neurosciences, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- SNC and CIRCA Research Groups, Université de Montréal, Montréal, QC, Canada
| | - Marie-Josée Bourque
- Department of Pharmacology and Physiology, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- Department of Neurosciences, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- SNC and CIRCA Research Groups, Université de Montréal, Montréal, QC, Canada
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, 20815, USA
| | - Alex Tchung
- Department of Pharmacology and Physiology, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- Department of Neurosciences, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
- SNC and CIRCA Research Groups, Université de Montréal, Montréal, QC, Canada
| | - Pedro Rosa-Neto
- Centre intégré universitaire de santé et de services sociaux (CIUSSS) de l'Ouest-de-l'Île-de-Montréal; Department of Neurology and Neurosurgery, Psychiatry and Pharmacology and Therapeutics, McGill University, Montreal, QC, Canada
| | - Daniel Lévesque
- Faculty of Pharmacy, Université de Montréal, Montreal, QC, Canada
| | - Louis De Beaumont
- Hôpital du Sacré-Cœur-de-Montréal, CIUSSS NIM, Université de Montréal, Montreal, QC, Canada
| | - Louis-Éric Trudeau
- Department of Pharmacology and Physiology, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada.
- Department of Neurosciences, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada.
- SNC and CIRCA Research Groups, Université de Montréal, Montréal, QC, Canada.
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, 20815, USA.
| |
Collapse
|
11
|
Chen CS, Mueller D, Knep E, Ebitz RB, Grissom NM. Dopamine and norepinephrine differentially mediate the exploration-exploitation tradeoff. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.01.09.523322. [PMID: 36711959 PMCID: PMC9881999 DOI: 10.1101/2023.01.09.523322] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
The catecholamines dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision making processes. Although the two neuromodulators share a synthesis pathway and are co-activated under states of arousal, they engage in distinct circuits and roles in modulating neural activity across the brain. However, in the computational neuroscience literature, they have been assigned similar roles in modulating the latent cognitive processes of decision making, in particular the exploration-exploitation tradeoff. Revealing how each neuromodulator contributes to this explore-exploit process will be important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration and exploitation, a direct comparison using the same dynamic decision making task is needed. Here, we ran mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA receptor antagonist (flupenthixol), a nonselective DA receptor agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol), and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine receptor activity on the level of exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. Beta-noradrenergic receptor activity also modulated exploration, but the modulatory effect was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via outcome sensitivity. Together, these findings suggested that the mechanisms that govern the transition between exploration and exploitation are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.
Collapse
|
12
|
Darcey VL, Guo J, Courville AB, Gallagher I, Avery JA, Simmons WK, Ingeholm JE, Herscovitch P, Martin A, Hall KD. Dietary fat restriction affects brain reward regions in a randomized crossover trial. JCI Insight 2023; 8:e169759. [PMID: 37345661 PMCID: PMC10371234 DOI: 10.1172/jci.insight.169759] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Accepted: 05/10/2023] [Indexed: 06/23/2023] Open
Abstract
BACKGROUNDWeight-loss diets often target dietary fat or carbohydrates, macronutrients that are sensed via distinct gut-brain pathways and differentially affect peripheral hormones and metabolism. However, the effects of such diet changes on the human brain are unclear. METHODSWe investigated whether selective isocaloric reductions in dietary fat or carbohydrates altered dopamine D2/3 receptor binding potential (D2BP) and neural activity in brain-reward regions in response to visual food cues in 17 inpatient adults with obesity as compared with a eucaloric baseline diet using a randomized crossover design. RESULTSOn the fifth day of dietary fat restriction, but not carbohydrate restriction, both D2BP and neural activity to food cues were decreased in brain-reward regions. After the reduced-fat diet, ad libitum intake shifted toward foods high in both fat and carbohydrates. CONCLUSIONThese results suggest that dietary fat restriction increases tonic dopamine in brain-reward regions and affects food choice in ways that may hamper diet adherence. TRIAL REGISTRATIONClinicalTrials.gov NCT00846040 FUNDING. NIDDK 1ZIADK013037.
Collapse
Affiliation(s)
- Valerie L Darcey
- Integrative Physiology Section, National Institute of Diabetes & Digestive & Kidney Diseases, Bethesda, Maryland, USA
| | - Juen Guo
- Integrative Physiology Section, National Institute of Diabetes & Digestive & Kidney Diseases, Bethesda, Maryland, USA
| | - Amber B Courville
- Human Energy and Body Weight Regulation Core, National Institute of Diabetes & Digestive & Kidney Diseases, Bethesda, Maryland, USA
| | - Isabelle Gallagher
- Integrative Physiology Section, National Institute of Diabetes & Digestive & Kidney Diseases, Bethesda, Maryland, USA
| | - Jason A Avery
- Laboratory of Brain and Cognition, National Institute of Mental Health, Rockland, Maryland, USA
| | - W Kyle Simmons
- Biomedical Imaging Center, Oklahoma State University, Stillwater, Oklahoma, USA
| | - John E Ingeholm
- Laboratory of Brain and Cognition, National Institute of Mental Health, Rockland, Maryland, USA
| | - Peter Herscovitch
- Clinical Center Positron Emission Tomography Department, NIH, Bethesda, Maryland, USA
| | - Alex Martin
- Laboratory of Brain and Cognition, National Institute of Mental Health, Rockland, Maryland, USA
| | - Kevin D Hall
- Integrative Physiology Section, National Institute of Diabetes & Digestive & Kidney Diseases, Bethesda, Maryland, USA
| |
Collapse
|
13
|
Savchenko A, Targa G, Fesenko Z, Leo D, Gainetdinov RR, Sukhanov I. Dopamine Transporter Deficient Rodents: Perspectives and Limitations for Neuroscience. Biomolecules 2023; 13:806. [PMID: 37238676 PMCID: PMC10216310 DOI: 10.3390/biom13050806] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Revised: 05/04/2023] [Accepted: 05/05/2023] [Indexed: 05/28/2023] Open
Abstract
The key element of dopamine (DA) neurotransmission is undoubtedly DA transporter (DAT), a transmembrane protein responsible for the synaptic reuptake of the mediator. Changes in DAT's function can be a key mechanism of pathological conditions associated with hyperdopaminergia. The first strain of gene-modified rodents with a lack of DAT were created more than 25 years ago. Such animals are characterized by increased levels of striatal DA, resulting in locomotor hyperactivity, increased levels of motor stereotypes, cognitive deficits, and other behavioral abnormalities. The administration of dopaminergic and pharmacological agents affecting other neurotransmitter systems can mitigate those abnormalities. The main purpose of this review is to systematize and analyze (1) known data on the consequences of changes in DAT expression in experimental animals, (2) results of pharmacological studies in these animals, and (3) to estimate the validity of animals lacking DAT as models for discovering new treatments of DA-related disorders.
Collapse
Affiliation(s)
- Artem Savchenko
- Valdman Institute of Pharmacology, Pavlov First St. Petersburg State Medical University, Lev Tolstoy Str. 6-8, 197022 St. Petersburg, Russia;
| | - Giorgia Targa
- Department of Pharmacological and Biomolecular Sciences “Rodolfo Paoletti”, Università degli Studi di Milano, Via Balzaretti 9, 20133 Milano, Italy
| | - Zoia Fesenko
- Institute of Translational Biomedicine, St. Petersburg State University, 7/9 Universitetskaya Emb., 199034 St. Petersburg, Russia
| | - Damiana Leo
- Department of Neurosciences, University of Mons, 7000 Mons, Belgium
| | - Raul R. Gainetdinov
- Institute of Translational Biomedicine, St. Petersburg State University, 7/9 Universitetskaya Emb., 199034 St. Petersburg, Russia
- St. Petersburg University Hospital, St. Petersburg State University, Fontanka River Emb. 154, 190121 St. Petersburg, Russia
| | - Ilya Sukhanov
- Valdman Institute of Pharmacology, Pavlov First St. Petersburg State Medical University, Lev Tolstoy Str. 6-8, 197022 St. Petersburg, Russia;
- St. Petersburg University Hospital, St. Petersburg State University, Fontanka River Emb. 154, 190121 St. Petersburg, Russia
| |
Collapse
|
14
|
Wang S, Gerken B, Wieland JR, Wilson RC, Fellous JM. The effects of time horizon and guided choices on explore-exploit decisions in rodents. Behav Neurosci 2023; 137:127-142. [PMID: 36633987 PMCID: PMC10787949 DOI: 10.1037/bne0000549] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
Humans and animals have to balance the need for exploring new options with exploiting known options that yield good outcomes. This tradeoff is known as the explore-exploit dilemma. To better understand the neural mechanisms underlying how humans and animals address the explore-exploit dilemma, a good animal behavioral model is critical. Most previous rodents explore-exploit studies used ethologically unrealistic operant boxes and reversal learning paradigms in which the decision to abandon a bad option is confounded by the need for exploring a novel option for information collection, making it difficult to separate different drives and heuristics for exploration. In this study, we investigated how rodents make explore-exploit decisions using a spatial navigation horizon task (Wilson et al., 2014) adapted to rats to address the above limitations. We compared the rats' performance to that of humans using identical measures. We showed that rats use prior information to effectively guide exploration. In addition, rats use information-driven directed exploration like humans, but the extent to which they explore has the opposite dependance on time horizon than humans. Moreover, we found that free choices and guided choices have different influences on exploration in rodents, a finding that has not yet been tested in humans. This study reveals that the explore-exploit spatial behavior of rats is more complex than previously thought. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Collapse
|
15
|
Speers LJ, Bilkey DK. Maladaptive explore/exploit trade-offs in schizophrenia. Trends Neurosci 2023; 46:341-354. [PMID: 36878821 DOI: 10.1016/j.tins.2023.02.001] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Revised: 01/30/2023] [Accepted: 02/08/2023] [Indexed: 03/07/2023]
Abstract
Schizophrenia is a complex disorder that remains poorly understood, particularly at the systems level. In this opinion article we argue that the explore/exploit trade-off concept provides a holistic and ecologically valid framework to resolve some of the apparent paradoxes that have emerged within schizophrenia research. We review recent evidence suggesting that fundamental explore/exploit behaviors may be maladaptive in schizophrenia during physical, visual, and cognitive foraging. We also describe how theories from the broader optimal foraging literature, such as the marginal value theorem (MVT), could provide valuable insight into how aberrant processing of reward, context, and cost/effort evaluations interact to produce maladaptive responses.
Collapse
Affiliation(s)
- Lucinda J Speers
- Department of Psychology, University of Otago, Dunedin 9016, New Zealand
| | - David K Bilkey
- Department of Psychology, University of Otago, Dunedin 9016, New Zealand.
| |
Collapse
|
16
|
Desch S, Schweinhardt P, Seymour B, Flor H, Becker S. Evidence for dopaminergic involvement in endogenous modulation of pain relief. eLife 2023; 12:e81436. [PMID: 36722857 PMCID: PMC9988263 DOI: 10.7554/elife.81436] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Accepted: 01/31/2023] [Indexed: 02/02/2023] Open
Abstract
Relief of ongoing pain is a potent motivator of behavior, directing actions to escape from or reduce potentially harmful stimuli. Whereas endogenous modulation of pain events is well characterized, relatively little is known about the modulation of pain relief and its corresponding neurochemical basis. Here, we studied pain modulation during a probabilistic relief-seeking task (a 'wheel of fortune' gambling task), in which people actively or passively received reduction of a tonic thermal pain stimulus. We found that relief perception was enhanced by active decisions and unpredictability, and greater in high novelty-seeking trait individuals, consistent with a model in which relief is tuned by its informational content. We then probed the roles of dopaminergic and opioidergic signaling, both of which are implicated in relief processing, by embedding the task in a double-blinded cross-over design with administration of the dopamine precursor levodopa and the opioid receptor antagonist naltrexone. We found that levodopa enhanced each of these information-specific aspects of relief modulation but no significant effects of the opioidergic manipulation. These results show that dopaminergic signaling has a key role in modulating the perception of pain relief to optimize motivation and behavior.
Collapse
Affiliation(s)
- Simon Desch
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg UniversityMannheimGermany
- Clinical Psychology, Department of Experimental Psychology, Heinrich Heine University DüsseldorfDüsseldorfGermany
| | - Petra Schweinhardt
- Integrative Spinal Research, Department of Chiropractic Medicine, Balgrist University Hospital, University of ZurichZurichSwitzerland
| | - Ben Seymour
- Wellcome Centre for Integrative Neuroimaging, John Radcliffe HospitalOxfordUnited Kingdom
| | - Herta Flor
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg UniversityMannheimGermany
| | - Susanne Becker
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg UniversityMannheimGermany
- Clinical Psychology, Department of Experimental Psychology, Heinrich Heine University DüsseldorfDüsseldorfGermany
- Integrative Spinal Research, Department of Chiropractic Medicine, Balgrist University Hospital, University of ZurichZurichSwitzerland
| |
Collapse
|
17
|
Goyal A, Hwang S, Rusheen AE, Blaha CD, Bennet KE, Lee KH, Jang DP, Oh Y, Shin H. Software for near-real-time voltammetric tracking of tonic neurotransmitter levels in vivo. Front Neurosci 2022; 16:899436. [PMID: 36213749 PMCID: PMC9537688 DOI: 10.3389/fnins.2022.899436] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2022] [Accepted: 08/30/2022] [Indexed: 11/26/2022] Open
Abstract
Tonic extracellular neurotransmitter concentrations are important modulators of central network homeostasis. Disruptions in these tonic levels are thought to play a role in neurologic and psychiatric disease. Therefore, ways to improve their quantification are actively being investigated. Previously published voltammetric software packages have implemented FSCV, which is not capable of measuring tonic concentrations of neurotransmitters in vivo. In this paper, custom software was developed for near-real-time tracking (scans every 10 s) of neurotransmitters’ tonic concentrations with high sensitivity and spatiotemporal resolution both in vitro and in vivo using cyclic voltammetry combined with dynamic background subtraction (M-CSWV and FSCAV). This software was designed with flexibility, speed, and user-friendliness in mind. This software enables near-real-time measurement by reducing data analysis time through an optimized modeling algorithm, and efficient memory handling makes long-term measurement possible. The software permits customization of the cyclic voltammetric waveform shape, enabling experiments to detect a specific analyte of interest. Finally, flexibility considerations allow the user to alter the fitting parameters, filtering characteristics, and size and shape of the analyte kernel, based on data obtained live during the experiment to obtain accurate measurements as experimental conditions change. Herein, the design and advantages of this near-real-time voltammetric software are described, and its use is demonstrated in in vivo experiments.
Collapse
Affiliation(s)
- Abhinav Goyal
- Mayo Clinic Medical Scientist Training Program, Mayo Clinic, Rochester, MN, United States
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, United States
| | - Sangmun Hwang
- Department of Biomedical Engineering, Hanyang University, Seoul, South Korea
| | - Aaron E. Rusheen
- Mayo Clinic Medical Scientist Training Program, Mayo Clinic, Rochester, MN, United States
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, United States
| | - Charles D. Blaha
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, United States
| | - Kevin E. Bennet
- Division of Engineering, Mayo Clinic, Rochester, MN, United States
| | - Kendall H. Lee
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, United States
- Department of Physiology and Biomedical Engineering, Mayo Clinic, Rochester, MN, United States
| | - Dong Pyo Jang
- Department of Biomedical Engineering, Hanyang University, Seoul, South Korea
| | - Yoonbae Oh
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, United States
- Department of Physiology and Biomedical Engineering, Mayo Clinic, Rochester, MN, United States
| | - Hojin Shin
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, United States
- Department of Physiology and Biomedical Engineering, Mayo Clinic, Rochester, MN, United States
- *Correspondence: Hojin Shin,
| |
Collapse
|
18
|
van den Bosch R, Lambregts B, Määttä J, Hofmans L, Papadopetraki D, Westbrook A, Verkes RJ, Booij J, Cools R. Striatal dopamine dissociates methylphenidate effects on value-based versus surprise-based reversal learning. Nat Commun 2022; 13:4962. [PMID: 36002446 PMCID: PMC9402573 DOI: 10.1038/s41467-022-32679-1] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2021] [Accepted: 08/10/2022] [Indexed: 11/13/2022] Open
Abstract
Psychostimulants such as methylphenidate are widely used for their cognitive enhancing effects, but there is large variability in the direction and extent of these effects. We tested the hypothesis that methylphenidate enhances or impairs reward/punishment-based reversal learning depending on baseline striatal dopamine levels and corticostriatal gating of reward/punishment-related representations in stimulus-specific sensory cortex. Young healthy adults (N = 100) were scanned with functional magnetic resonance imaging during a reward/punishment reversal learning task, after intake of methylphenidate or the selective D2/3-receptor antagonist sulpiride. Striatal dopamine synthesis capacity was indexed with [18F]DOPA positron emission tomography. Methylphenidate improved and sulpiride decreased overall accuracy and response speed. Both drugs boosted reward versus punishment learning signals to a greater degree in participants with higher dopamine synthesis capacity. By contrast, striatal and stimulus-specific sensory surprise signals were boosted in participants with lower dopamine synthesis. These results unravel the mechanisms by which methylphenidate gates both attention and reward learning. The mechanisms underpinning the variability in methylphenidate’s effects on cognition remain unclear. Here, the authors show that such effects reflect changes in striatal dopamine-related output gating of task-relevant cortical signals, and that these changes depend on baseline dopamine synthesis capacity.
Collapse
Affiliation(s)
- Ruben van den Bosch
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| | - Britt Lambregts
- Radboud University Medical Center, Department of Psychiatry, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Jessica Määttä
- Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden
| | - Lieke Hofmans
- Department of Developmental Psychology, University of Amsterdam, Amsterdam, The Netherlands
| | - Danae Papadopetraki
- Radboud University Medical Center, Department of Psychiatry, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Andrew Westbrook
- Cognitive, Linguistic & Psychological Sciences Department, Brown University, Providence, RI, USA
| | - Robbert-Jan Verkes
- Radboud University Medical Center, Department of Psychiatry, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Jan Booij
- Department of Radiology and Nuclear Medicine, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, The Netherlands.,Radboud University Medical Center, Department of Medical Imaging, Nijmegen, The Netherlands
| | - Roshan Cools
- Radboud University Medical Center, Department of Psychiatry, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| |
Collapse
|
19
|
Jepma M, Roy M, Ramlakhan K, van Velzen M, Dahan A. Different brain systems support learning from received and avoided pain during human pain-avoidance learning. eLife 2022; 11:74149. [PMID: 35731646 PMCID: PMC9217130 DOI: 10.7554/elife.74149] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2021] [Accepted: 06/07/2022] [Indexed: 12/14/2022] Open
Abstract
Both unexpected pain and unexpected pain absence can drive avoidance learning, but whether they do so via shared or separate neural and neurochemical systems is largely unknown. To address this issue, we combined an instrumental pain-avoidance learning task with computational modeling, functional magnetic resonance imaging (fMRI), and pharmacological manipulations of the dopaminergic (100 mg levodopa) and opioidergic (50 mg naltrexone) systems (N = 83). Computational modeling provided evidence that untreated participants learned more from received than avoided pain. Our dopamine and opioid manipulations negated this learning asymmetry by selectively increasing learning rates for avoided pain. Furthermore, our fMRI analyses revealed that pain prediction errors were encoded in subcortical and limbic brain regions, whereas no-pain prediction errors were encoded in frontal and parietal cortical regions. However, we found no effects of our pharmacological manipulations on the neural encoding of prediction errors. Together, our results suggest that human pain-avoidance learning is supported by separate threat- and safety-learning systems, and that dopamine and endogenous opioids specifically regulate learning from successfully avoided pain.
Collapse
Affiliation(s)
- Marieke Jepma
- Department of Psychology, University of Amsterdam, Amsterdam, Netherlands.,Department of Psychology, Leiden University, Leiden, Netherlands.,Leiden Institute for Brain and Cognition, Leiden, Netherlands
| | - Mathieu Roy
- Department of Psychology, McGill University, Montreal, Canada.,Alan Edwards Centre for Research on Pain, McGill University, Montreal, Canada
| | - Kiran Ramlakhan
- Department of Psychology, Leiden University, Leiden, Netherlands.,Department of Research and Statistics, Municipality of Amsterdam, Amsterdam, Netherlands
| | - Monique van Velzen
- Department of Anesthesiology, Leiden University Medical Center, Leiden, Netherlands
| | - Albert Dahan
- Department of Anesthesiology, Leiden University Medical Center, Leiden, Netherlands
| |
Collapse
|
20
|
Faure P, Fayad SL, Solié C, Reynolds LM. Social Determinants of Inter-Individual Variability and Vulnerability: The Role of Dopamine. Front Behav Neurosci 2022; 16:836343. [PMID: 35386723 PMCID: PMC8979673 DOI: 10.3389/fnbeh.2022.836343] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Accepted: 02/14/2022] [Indexed: 11/13/2022] Open
Abstract
Individuals differ in their traits and preferences, which shape their interactions, their prospects for survival and their susceptibility to diseases. These correlations are well documented, yet the neurophysiological mechanisms underlying the emergence of distinct personalities and their relation to vulnerability to diseases are poorly understood. Social ties, in particular, are thought to be major modulators of personality traits and psychiatric vulnerability, yet the majority of neuroscience studies are performed on rodents in socially impoverished conditions. Rodent micro-society paradigms are therefore key experimental paradigms to understand how social life generates diversity by shaping individual traits. Dopamine circuitry is implicated at the interface between social life experiences, the expression of essential traits, and the emergence of pathologies, thus proving a possible mechanism to link these three concepts at a neuromodulatory level. Evaluating inter-individual variability in automated social testing environments shows great promise for improving our understanding of the link between social life, personality, and precision psychiatry – as well as elucidating the underlying neurophysiological mechanisms.
Collapse
|
21
|
Mei J, Muller E, Ramaswamy S. Informing deep neural networks by multiscale principles of neuromodulatory systems. Trends Neurosci 2022; 45:237-250. [DOI: 10.1016/j.tins.2021.12.008] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2021] [Revised: 12/04/2021] [Accepted: 12/21/2021] [Indexed: 01/19/2023]
|
22
|
Mikhael JG, Gershman SJ. Impulsivity and risk-seeking as Bayesian inference under dopaminergic control. Neuropsychopharmacology 2022; 47:465-476. [PMID: 34376813 PMCID: PMC8674258 DOI: 10.1038/s41386-021-01125-z] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/01/2020] [Revised: 07/17/2021] [Accepted: 07/21/2021] [Indexed: 02/07/2023]
Abstract
Bayesian models successfully account for several of dopamine (DA)'s effects on contextual calibration in interval timing and reward estimation. In these models, tonic levels of DA control the precision of stimulus encoding, which is weighed against contextual information when making decisions. When DA levels are high, the animal relies more heavily on the (highly precise) stimulus encoding, whereas when DA levels are low, the context affects decisions more strongly. Here, we extend this idea to intertemporal choice and probability discounting tasks. In intertemporal choice tasks, agents must choose between a small reward delivered soon and a large reward delivered later, whereas in probability discounting tasks, agents must choose between a small reward that is always delivered and a large reward that may be omitted with some probability. Beginning with the principle that animals will seek to maximize their reward rates, we show that the Bayesian model predicts a number of curious empirical findings in both tasks. First, the model predicts that higher DA levels should normally promote selection of the larger/later option, which is often taken to imply that DA decreases 'impulsivity,' and promote selection of the large/risky option, often taken to imply that DA increases 'risk-seeking.' However, if the temporal precision is sufficiently decreased, higher DA levels should have the opposite effect-promoting selection of the smaller/sooner option (higher impulsivity) and the small/safe option (lower risk-seeking). Second, high enough levels of DA can result in preference reversals. Third, selectively decreasing the temporal precision, without manipulating DA, should promote selection of the larger/later and large/risky options. Fourth, when a different post-reward delay is associated with each option, animals will not learn the option-delay contingencies, but this learning can be salvaged when the post-reward delays are made more salient. Finally, the Bayesian model predicts correlations among behavioral phenotypes: Animals that are better timers will also appear less impulsive.
Collapse
Affiliation(s)
- John G. Mikhael
- grid.38142.3c000000041936754XProgram in Neuroscience, Harvard Medical School, Boston, MA USA ,grid.38142.3c000000041936754XMD-PhD Program, Harvard Medical School, Boston, MA USA
| | - Samuel J. Gershman
- grid.38142.3c000000041936754XDepartment of Psychology and Center for Brain Science, Harvard University, Cambridge, MA USA ,grid.116068.80000 0001 2341 2786Center for Brains, Minds and Machines, Massachusetts Institute of Technology, Cambridge, MA USA
| |
Collapse
|
23
|
Frydecka D, Misiak B, Piotrowski P, Bielawski T, Pawlak E, Kłosińska E, Krefft M, Al Noaimy K, Rymaszewska J, Moustafa AA, Drapała J. The Role of Dopaminergic Genes in Probabilistic Reinforcement Learning in Schizophrenia Spectrum Disorders. Brain Sci 2021; 12:brainsci12010007. [PMID: 35053751 PMCID: PMC8774082 DOI: 10.3390/brainsci12010007] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2021] [Revised: 11/30/2021] [Accepted: 12/19/2021] [Indexed: 12/27/2022] Open
Abstract
Schizophrenia spectrum disorders (SZ) are characterized by impairments in probabilistic reinforcement learning (RL), which is associated with dopaminergic circuitry encompassing the prefrontal cortex and basal ganglia. However, there are no studies examining dopaminergic genes with respect to probabilistic RL in SZ. Thus, the aim of our study was to examine the impact of dopaminergic genes on performance assessed by the Probabilistic Selection Task (PST) in patients with SZ in comparison to healthy control (HC) subjects. In our study, we included 138 SZ patients and 188 HC participants. Genetic analysis was performed with respect to the following genetic polymorphisms: rs4680 in COMT, rs907094 in DARP-32, rs2734839, rs936461, rs1800497, and rs6277 in DRD2, rs747302 and rs1800955 in DRD4 and rs28363170 and rs2975226 in DAT1 genes. The probabilistic RL task was completed by 59 SZ patients and 95 HC subjects. SZ patients performed significantly worse in acquiring reinforcement contingencies during the task in comparison to HCs. We found no significant association between genetic polymorphisms and RL among SZ patients; however, among HC participants with respect to the DAT1 rs28363170 polymorphism, individuals with 10-allele repeat genotypes performed better in comparison to 9-allele repeat carriers. The present study indicates the relevance of the DAT1 rs28363170 polymorphism in RL in HC participants.
Collapse
Affiliation(s)
- Dorota Frydecka
- Department of Psychiatry, Wroclaw Medical University, Pasteur Street 10, 50-367 Wroclaw, Poland; (T.B.); (M.K.); (K.A.N.); (J.R.)
- Correspondence:
| | - Błażej Misiak
- Department of Psychiatry, Division of Consultation Psychiatry and Neuroscience, Wroclaw Medical University, Pasteur Street 10, 50-367 Wroclaw, Poland; (B.M.); (P.P.)
| | - Patryk Piotrowski
- Department of Psychiatry, Division of Consultation Psychiatry and Neuroscience, Wroclaw Medical University, Pasteur Street 10, 50-367 Wroclaw, Poland; (B.M.); (P.P.)
| | - Tomasz Bielawski
- Department of Psychiatry, Wroclaw Medical University, Pasteur Street 10, 50-367 Wroclaw, Poland; (T.B.); (M.K.); (K.A.N.); (J.R.)
| | - Edyta Pawlak
- Department of Experimental Therapy, Institute of Immunology and Experimental Therapy, Polish Academy of Sciences, Weigel Street 12, 53-114 Wroclaw, Poland;
| | - Ewa Kłosińska
- Day-Care Psychiatric Unit, University Clinical Hospital, Pasteur Street 10, 50-367 Wroclaw, Poland;
| | - Maja Krefft
- Department of Psychiatry, Wroclaw Medical University, Pasteur Street 10, 50-367 Wroclaw, Poland; (T.B.); (M.K.); (K.A.N.); (J.R.)
| | - Kamila Al Noaimy
- Department of Psychiatry, Wroclaw Medical University, Pasteur Street 10, 50-367 Wroclaw, Poland; (T.B.); (M.K.); (K.A.N.); (J.R.)
| | - Joanna Rymaszewska
- Department of Psychiatry, Wroclaw Medical University, Pasteur Street 10, 50-367 Wroclaw, Poland; (T.B.); (M.K.); (K.A.N.); (J.R.)
| | - Ahmed A. Moustafa
- School of Psychology, Marcs Institute for Brain and Behaviour, Western Sydney University, Locked Bag 1797, Penrith, NSW 2751, Australia;
- Department of Human Anatomy and Physiology, The Faculty of Health Sciences, University of Johannesburg, Johannesburg 2006, South Africa
| | - Jarosław Drapała
- Department of Computer Science and Systems Engineering, Faculty of Information and Communication Technology, Wroclaw University of Science and Technology, Wybrzeze Wyspianskiego Street 27, 50-370 Wrocław, Poland;
| |
Collapse
|
24
|
Chronic nicotine increases midbrain dopamine neuron activity and biases individual strategies towards reduced exploration in mice. Nat Commun 2021; 12:6945. [PMID: 34836948 PMCID: PMC8635406 DOI: 10.1038/s41467-021-27268-7] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2021] [Accepted: 11/04/2021] [Indexed: 11/09/2022] Open
Abstract
Long-term exposure to nicotine alters brain circuits and induces profound changes in decision-making strategies, affecting behaviors both related and unrelated to drug seeking and consumption. Using an intracranial self-stimulation reward-based foraging task, we investigated in mice the impact of chronic nicotine on midbrain dopamine neuron activity and its consequence on the trade-off between exploitation and exploration. Model-based and archetypal analysis revealed substantial inter-individual variability in decision-making strategies, with mice passively exposed to nicotine shifting toward a more exploitative profile compared to non-exposed animals. We then mimicked the effect of chronic nicotine on the tonic activity of dopamine neurons using optogenetics, and found that photo-stimulated mice adopted a behavioral phenotype similar to that of mice exposed to chronic nicotine. Our results reveal a key role of tonic midbrain dopamine in the exploration/exploitation trade-off and highlight a potential mechanism by which nicotine affects the exploration/exploitation balance and decision-making.
Collapse
|
25
|
Chen CS, Knep E, Han A, Ebitz RB, Grissom N. Sex differences in learning from exploration. eLife 2021; 10:69748. [PMID: 34796870 PMCID: PMC8794469 DOI: 10.7554/elife.69748] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2021] [Accepted: 11/18/2021] [Indexed: 11/13/2022] Open
Abstract
Sex-based modulation of cognitive processes could set the stage for individual differences in vulnerability to neuropsychiatric disorders. While value-based decision making processes in particular have been proposed to be influenced by sex differences, the overall correct performance in decision making tasks often show variable or minimal differences across sexes. Computational tools allow us to uncover latent variables that define different decision making approaches, even in animals with similar correct performance. Here, we quantify sex differences in mice in the latent variables underlying behavior in a classic value-based decision making task: a restless 2-armed bandit. While male and female mice had similar accuracy, they achieved this performance via different patterns of exploration. Male mice tended to make more exploratory choices overall, largely because they appeared to get 'stuck' in exploration once they had started. Female mice tended to explore less but learned more quickly during exploration. Together, these results suggest that sex exerts stronger influences on decision making during periods of learning and exploration than during stable choices. Exploration during decision making is altered in people diagnosed with addictions, depression, and neurodevelopmental disabilities, pinpointing the neural mechanisms of exploration as a highly translational avenue for conferring sex-modulated vulnerability to neuropsychiatric diagnoses.
Collapse
Affiliation(s)
- Cathy S Chen
- University of Minnesota, Minneapolis, United States
| | - Evan Knep
- University of Minnesota, Minneapolis, United States
| | - Autumn Han
- University of Minnesota, Minneapolis, United States
| | - R Becket Ebitz
- Department of Neurosciences, Princeton University, Princeton, United States
| | | |
Collapse
|
26
|
Foo C, Lozada A, Aljadeff J, Li Y, Wang JW, Slesinger PA, Kleinfeld D. Reinforcement learning links spontaneous cortical dopamine impulses to reward. Curr Biol 2021; 31:4111-4119.e4. [PMID: 34302743 DOI: 10.1016/j.cub.2021.06.069] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Revised: 05/28/2021] [Accepted: 06/24/2021] [Indexed: 11/15/2022]
Abstract
In their pioneering study on dopamine release, Romo and Schultz speculated "...that the amount of dopamine released by unmodulated spontaneous impulse activity exerts a tonic, permissive influence on neuronal processes more actively engaged in preparation of self-initiated movements...."1 Motivated by the suggestion of "spontaneous impulses," as well as by the "ramp up" of dopaminergic neuronal activity that occurs when rodents navigate to a reward,2-5 we asked two questions. First, are there spontaneous impulses of dopamine that are released in cortex? Using cell-based optical sensors of extrasynaptic dopamine, [DA]ex,6 we found that spontaneous dopamine impulses in cortex of naive mice occur at a rate of ∼0.01 per second. Next, can mice be trained to change the amplitude and/or timing of dopamine events triggered by internal brain dynamics, much as they can change the amplitude and timing of dopamine impulses based on an external cue?7-9 Using a reinforcement learning paradigm based solely on rewards that were gated by feedback from real-time measurements of [DA]ex, we found that mice can volitionally modulate their spontaneous [DA]ex. In particular, by only the second session of daily, hour-long training, mice increased the rate of impulses of [DA]ex, increased the amplitude of the impulses, and increased their tonic level of [DA]ex for a reward. Critically, mice learned to reliably elicit [DA]ex impulses prior to receiving a reward. These effects reversed when the reward was removed. We posit that spontaneous dopamine impulses may serve as a salient cognitive event in behavioral planning.
Collapse
Affiliation(s)
- Conrad Foo
- Department of Physics, University of California at San Diego, La Jolla, CA 92093, USA
| | - Adrian Lozada
- Department of Physics, University of California at San Diego, La Jolla, CA 92093, USA
| | - Johnatan Aljadeff
- Section of Neurobiology, University of California at San Diego, La Jolla, CA 92093, USA
| | - Yulong Li
- Peking University, School of Life Sciences, Peking University, Beijing 100871, P.R. China
| | - Jing W Wang
- Section of Neurobiology, University of California at San Diego, La Jolla, CA 92093, USA
| | - Paul A Slesinger
- Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
| | - David Kleinfeld
- Department of Physics, University of California at San Diego, La Jolla, CA 92093, USA; Section of Neurobiology, University of California at San Diego, La Jolla, CA 92093, USA.
| |
Collapse
|
27
|
Tardiff N, Medaglia JD, Bassett DS, Thompson-Schill SL. The modulation of brain network integration and arousal during exploration. Neuroimage 2021; 240:118369. [PMID: 34242784 PMCID: PMC8507424 DOI: 10.1016/j.neuroimage.2021.118369] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2020] [Revised: 07/01/2021] [Accepted: 07/05/2021] [Indexed: 11/08/2022] Open
Abstract
There is growing interest in how neuromodulators shape brain networks. Recent neuroimaging studies provide evidence that brainstem arousal systems, such as the locus coeruleus-norepinephrine system (LC-NE), influence functional connectivity and brain network topology, suggesting they have a role in flexibly reconfiguring brain networks in order to adapt behavior and cognition to environmental demands. To date, however, the relationship between brainstem arousal systems and functional connectivity has not been assessed within the context of a task with an established relationship between arousal and behavior, with most prior studies relying on incidental variations in arousal or pharmacological manipulation and static brain networks constructed over long periods of time. These factors have likely contributed to a heterogeneity of effects across studies. To address these issues, we took advantage of the association between LC-NE-linked arousal and exploration to probe the relationships between exploratory choice, arousal—as measured indirectly via pupil diameter—and brain network dynamics. Exploration in a bandit task was associated with a shift toward fewer, more weakly connected modules that were more segregated in terms of connectivity and topology but more integrated with respect to the diversity of cognitive systems represented in each module. Functional connectivity strength decreased, and changes in connectivity were correlated with changes in pupil diameter, in line with the hypothesis that brainstem arousal systems influence the dynamic reorganization of brain networks. More broadly, we argue that carefully aligning dynamic network analyses with task designs can increase the temporal resolution at which behaviorally- and cognitively-relevant modulations can be identified, and offer these results as a proof of concept of this approach.
Collapse
Affiliation(s)
- Nathan Tardiff
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, United States.
| | - John D Medaglia
- Department of Psychology, Drexel University, Philadelphia, PA, United States; Department of Neurology, Drexel University, Philadelphia, PA, United States; Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Danielle S Bassett
- Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States; Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, United States; Department of Electrical & Systems Engineering, University of Pennsylvania, Philadelphia, PA, United States; Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, United States; Department of Physics & Astronomy, University of Pennsylvania, Philadelphia, PA, United States; Santa Fe Institute, Santa Fe, NM, United States
| | | |
Collapse
|
28
|
Jiménez-González A, Gómez-Acevedo C, Ochoa-Aguilar A, Chavarría A. The Role of Glia in Addiction: Dopamine as a Modulator of Glial Responses in Addiction. Cell Mol Neurobiol 2021; 42:2109-2120. [PMID: 34057683 DOI: 10.1007/s10571-021-01105-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Accepted: 05/19/2021] [Indexed: 02/07/2023]
Abstract
Addiction is a chronic and potentially deadly disease considered a global health problem. Nevertheless, there is still no ideal treatment for its management. The alterations in the reward system are the most known pathophysiological mechanisms. Dopamine is the pivotal neurotransmitter involved in neuronal drug reward mechanisms and its neuronal mechanisms have been intensely investigated in recent years. However, neuroglial interactions and their relation to drug addiction development and maintenance of drug addiction have been understudied. Many reports have found that most neuroglial cells express dopamine receptors and that dopamine activity may induce neuroimmunomodulatory effects. Furthermore, current research has also shown that pro- and anti-inflammatory molecules modulate dopaminergic neuron activity. Thus, studying the immune mechanisms of dopamine associated with drug abuse is vital in researching new pathophysiological mechanisms and new therapeutic targets for addiction management.
Collapse
Affiliation(s)
- Ariadna Jiménez-González
- Laboratorio de Biomembranas, Departamento de Farmacología, Facultad de Medicina, Universidad Nacional Autónoma de México, Mexico City, Mexico
| | - Claudia Gómez-Acevedo
- Laboratorio de Biomembranas, Departamento de Farmacología, Facultad de Medicina, Universidad Nacional Autónoma de México, Mexico City, Mexico
| | - Abraham Ochoa-Aguilar
- Plan de Estudios Combinados en Medicina, Facultad de Medicina, Universidad Nacional Autónoma de México, Mexico City, Mexico
| | - Anahí Chavarría
- Unidad de Investigación en Medicina Experimental, Facultad de Medicina, Universidad Nacional Autónoma de México, Mexico City, Mexico.
| |
Collapse
|
29
|
Gilbertson T, Steele D. Tonic dopamine, uncertainty and basal ganglia action selection. Neuroscience 2021; 466:109-124. [PMID: 34015370 DOI: 10.1016/j.neuroscience.2021.05.010] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2020] [Revised: 05/04/2021] [Accepted: 05/08/2021] [Indexed: 11/29/2022]
Abstract
To make optimal decisions in uncertain circumstances flexible adaption of behaviour is required; exploring alternatives when the best choice is unknown, exploiting what is known when that is best. Using a computational model of the basal ganglia, we propose that switches between exploratory and exploitative decisions are mediated by the interaction between tonic dopamine and cortical input to the basal ganglia. We show that a biologically detailed action selection circuit model, endowed with dopamine dependant striatal plasticity, can optimally solve the explore-exploit problem, estimating the true underlying state of a noisy Gaussian diffusion process. Critical to the model's performance was a fluctuating level of tonic dopamine which increased under conditions of uncertainty. With an optimal range of tonic dopamine, explore-exploit decisions were mediated by the effects of tonic dopamine on the precision of the model action selection mechanism. Under conditions of uncertain reward pay-out, the model's reduced selectivity allowed disinhibition of multiple alternative actions to be explored at random. Conversely, when uncertainly about reward pay-out was low, enhanced selectivity of the action selection circuit facilitated exploitation of the high value choice. Model performance was at the level of a Kalman filter which provides an optimal solution for the task. These simulations support the idea that this subcortical neural circuit may have evolved to facilitate decision making in non-stationary reward environments. The model generates several experimental predictions with relevance to abnormal decision making in neuropsychiatric and neurological disease.
Collapse
Affiliation(s)
- Tom Gilbertson
- Department of Neurology, Level 6, South Block, Ninewells Hospital & Medical School, Dundee DD2 4BF, UK; Division of Imaging Science and Technology, Medical School, University of Dundee, DD2 4BF, UK.
| | - Douglas Steele
- Division of Imaging Science and Technology, Medical School, University of Dundee, DD2 4BF, UK
| |
Collapse
|
30
|
Kurzina NP, Volnova AB, Aristova IY, Gainetdinov RR. A New Paradigm for Training Hyperactive Dopamine Transporter Knockout Rats: Influence of Novel Stimuli on Object Recognition. Front Behav Neurosci 2021; 15:654469. [PMID: 33967714 PMCID: PMC8100052 DOI: 10.3389/fnbeh.2021.654469] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2021] [Accepted: 03/04/2021] [Indexed: 01/07/2023] Open
Abstract
Attention deficit hyperactivity disorder (ADHD) is believed to be connected with a high level of hyperactivity caused by alterations of the control of dopaminergic transmission in the brain. The strain of hyperdopaminergic dopamine transporter knockout (DAT-KO) rats represents an optimal model for investigating ADHD-related pathological mechanisms. The goal of this work was to study the influence of the overactivated dopamine system in the brain on a motor cognitive task fulfillment. The DAT-KO rats were trained to learn an object recognition task and store it in long-term memory. We found that DAT-KO rats can learn to move an object and retrieve food from the rewarded familiar objects and not to move the non-rewarded novel objects. However, we observed that the time of task performance and the distances traveled were significantly increased in DAT-KO rats in comparison with wild-type controls. Both groups of rats explored the novel objects longer than the familiar cubes. However, unlike controls, DAT-KO rats explored novel objects significantly longer and with fewer errors, since they preferred not to move the non-rewarded novel objects. After a 3 months' interval that followed the training period, they were able to retain the learned skills in memory and to efficiently retrieve them. The data obtained indicate that DAT-KO rats have a deficiency in learning the cognitive task, but their hyperactivity does not prevent the ability to learn a non-spatial cognitive task under the presentation of novel stimuli. The longer exploration of novel objects during training may ensure persistent learning of the task paradigm. These findings may serve as a base for developing new ADHD learning paradigms.
Collapse
Affiliation(s)
- Natalia P. Kurzina
- Institute of Translational Biomedicine, Saint Petersburg State University, Saint Petersburg, Russia
| | - Anna B. Volnova
- Institute of Translational Biomedicine, Saint Petersburg State University, Saint Petersburg, Russia
- Department of Physiology, Faculty of Biology, Saint Petersburg State University, Saint Petersburg, Russia
| | - Irina Y. Aristova
- Department of Physiology, Faculty of Biology, Saint Petersburg State University, Saint Petersburg, Russia
| | - Raul R. Gainetdinov
- Institute of Translational Biomedicine, Saint Petersburg State University, Saint Petersburg, Russia
- Saint Petersburg State University Hospital, Saint Petersburg State University, Saint Petersburg, Russia
| |
Collapse
|
31
|
Oberrauch S, Metha JA, Brian ML, Barnes SA, Featherby TJ, Lawrence AJ, Hoyer D, Murawski C, Jacobson LH. Reward motivation and cognitive flexibility in tau null-mutation mice. Neurobiol Aging 2021; 100:106-117. [PMID: 33524848 DOI: 10.1016/j.neurobiolaging.2020.12.017] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2020] [Revised: 11/05/2020] [Accepted: 12/15/2020] [Indexed: 01/21/2023]
Abstract
The reduction of tau or hyperphosphorylated tau (p-tau) has been proposed as a therapeutic strategy for Alzheimer's disease (AD) and frontotemporal dementia (FTD). Cognitive decline and sleep-wake dysregulation seen in AD and FTD patients are mimicked in transgenic and null-mutation mouse models of tauopathy. Alterations in the reward system are additional symptoms of AD and FTD. However, the role of tau in reward processes is not well understood. The present study aimed to examine reward and reward-motivated cognitive processes in male and female tau knockout (tau-/-) and wild-type mice using progressive ratio and reversal learning tasks. Tau-/- mice were heavier, ate more in the home cage, and reached criterion in operant lever training faster than wild-type mice. Tau-/- mice had a higher breakpoint in progressive ratio but were unimpaired in reversal learning or reward sensitivity. These data indicate that tau loss of function alters reward processing. This may help to explain aberrant reward-related behaviors in tauopathy patients and highlights a potentially important area for consideration in the development of anti-tau therapies.
Collapse
Affiliation(s)
- Sara Oberrauch
- The Florey Institute of Neuroscience and Mental Health, Parkville Campus, University of Melbourne, Parkville, Australia; Department of Pharmacology & Therapeutics, School of Medicine, Faculty of Medicine, Dentistry and Health Sciences, The University of Melbourne, Parkville, Victoria, Australia
| | - Jeremy A Metha
- The Florey Institute of Neuroscience and Mental Health, Parkville Campus, University of Melbourne, Parkville, Australia; Department of Pharmacology & Therapeutics, School of Medicine, Faculty of Medicine, Dentistry and Health Sciences, The University of Melbourne, Parkville, Victoria, Australia; Department of Finance, Brain, Mind & Markets Laboratory, The University of Melbourne, Melbourne, Victoria, Australia
| | - Maddison L Brian
- The Florey Institute of Neuroscience and Mental Health, Parkville Campus, University of Melbourne, Parkville, Australia; Department of Pharmacology & Therapeutics, School of Medicine, Faculty of Medicine, Dentistry and Health Sciences, The University of Melbourne, Parkville, Victoria, Australia
| | - Samuel A Barnes
- Department of Psychiatry, School of Medicine, University of California San Diego, La Jolla, CA, USA
| | - Travis J Featherby
- The Florey Institute of Neuroscience and Mental Health, Parkville Campus, University of Melbourne, Parkville, Australia
| | - Andrew J Lawrence
- The Florey Institute of Neuroscience and Mental Health, Parkville Campus, University of Melbourne, Parkville, Australia
| | - Daniel Hoyer
- The Florey Institute of Neuroscience and Mental Health, Parkville Campus, University of Melbourne, Parkville, Australia; Department of Pharmacology & Therapeutics, School of Medicine, Faculty of Medicine, Dentistry and Health Sciences, The University of Melbourne, Parkville, Victoria, Australia; Department of Molecular Medicine, The Scripps Research Institute, La Jolla, CA, USA
| | - Carsten Murawski
- Department of Finance, Brain, Mind & Markets Laboratory, The University of Melbourne, Melbourne, Victoria, Australia
| | - Laura H Jacobson
- The Florey Institute of Neuroscience and Mental Health, Parkville Campus, University of Melbourne, Parkville, Australia; Department of Pharmacology & Therapeutics, School of Medicine, Faculty of Medicine, Dentistry and Health Sciences, The University of Melbourne, Parkville, Victoria, Australia; Melbourne Dementia Research Centre, The Florey Institute of Neuroscience and Mental Health and The University of Melbourne, Victoria, Australia.
| |
Collapse
|
32
|
Mikhael JG, Lai L, Gershman SJ. Rational inattention and tonic dopamine. PLoS Comput Biol 2021; 17:e1008659. [PMID: 33760806 PMCID: PMC7990190 DOI: 10.1371/journal.pcbi.1008659] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2019] [Accepted: 12/28/2020] [Indexed: 11/27/2022] Open
Abstract
Slow-timescale (tonic) changes in dopamine (DA) contribute to a wide variety of processes in reinforcement learning, interval timing, and other domains. Furthermore, changes in tonic DA exert distinct effects depending on when they occur (e.g., during learning vs. performance) and what task the subject is performing (e.g., operant vs. classical conditioning). Two influential theories of tonic DA-the average reward theory and the Bayesian theory in which DA controls precision-have each been successful at explaining a subset of empirical findings. But how the same DA signal performs two seemingly distinct functions without creating crosstalk is not well understood. Here we reconcile the two theories under the unifying framework of 'rational inattention,' which (1) conceptually links average reward and precision, (2) outlines how DA manipulations affect this relationship, and in so doing, (3) captures new empirical phenomena. In brief, rational inattention asserts that agents can increase their precision in a task (and thus improve their performance) by paying a cognitive cost. Crucially, whether this cost is worth paying depends on average reward availability, reported by DA. The monotonic relationship between average reward and precision means that the DA signal contains the information necessary to retrieve the precision. When this information is needed after the task is performed, as presumed by Bayesian inference, acute manipulations of DA will bias behavior in predictable ways. We show how this framework reconciles a remarkably large collection of experimental findings. In reinforcement learning, the rational inattention framework predicts that learning from positive and negative feedback should be enhanced in high and low DA states, respectively, and that DA should tip the exploration-exploitation balance toward exploitation. In interval timing, this framework predicts that DA should increase the speed of the internal clock and decrease the extent of interference by other temporal stimuli during temporal reproduction (the central tendency effect). Finally, rational inattention makes the new predictions that these effects should be critically dependent on the controllability of rewards, that post-reward delays in intertemporal choice tasks should be underestimated, and that average reward manipulations should affect the speed of the clock-thus capturing empirical findings that are unexplained by either theory alone. Our results suggest that a common computational repertoire may underlie the seemingly heterogeneous roles of DA.
Collapse
Affiliation(s)
- John G. Mikhael
- Program in Neuroscience, Harvard Medical School, Boston, Massachusetts, United States of America
- MD-PhD Program, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Lucy Lai
- Program in Neuroscience, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Samuel J. Gershman
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, Massachusetts, United States of America
| |
Collapse
|
33
|
Ito Y, Toyoizumi T. Learning poly-synaptic paths with traveling waves. PLoS Comput Biol 2021; 17:e1008700. [PMID: 33561118 PMCID: PMC7928500 DOI: 10.1371/journal.pcbi.1008700] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2020] [Revised: 03/03/2021] [Accepted: 01/11/2021] [Indexed: 11/18/2022] Open
Abstract
Traveling waves are commonly observed across the brain. While previous studies have suggested the role of traveling waves in learning, the mechanism remains unclear. We adopted a computational approach to investigate the effect of traveling waves on synaptic plasticity. Our results indicate that traveling waves facilitate the learning of poly-synaptic network paths when combined with a reward-dependent local synaptic plasticity rule. We also demonstrate that traveling waves expedite finding the shortest paths and learning nonlinear input/output mapping, such as exclusive or (XOR) function. There are approximately 1011 neurons with 1014 connections in the human brain. Information transmission among neurons in this large network is considered crucial for our behavior. To achieve this, multiple synaptic connections along a poly-synaptic network path must be adjusted coherently during learning. Because the previously proposed reward-dependent synaptic plasticity rule requires coactivation of presynaptic and postsynaptic neurons, learning can fail if a subset of neurons along a distant network path is inactive at the beginning of learning. We suggest that traveling waves that are initiated at an information source can mitigate this problem. We performed computer simulations of spiking neural networks with reward-dependent local synaptic plasticity rules and traveling waves. Our results show that this combination facilitates the learning and refinement of synaptic network paths. We argue that these features are a general biological strategy for maintaining and optimizing our brain function. Our research provides new insights into how complex neural networks in the brain form during learning and memory consolidation.
Collapse
Affiliation(s)
- Yoshiki Ito
- Graduate School of Information and Technology, the Department of Mechano-Informatics, the University of Tokyo, Tokyo, Japan
- * E-mail: (YI); (TT)
| | - Taro Toyoizumi
- Lab for Neural Computation and Adaptation, RIKEN Center for Brain Science, Saitama, Japan
- Department of Mathematical Informatics, Graduate School of Information Science and Technology, the University of Tokyo, Tokyo, Japan
- * E-mail: (YI); (TT)
| |
Collapse
|
34
|
Attention-deficit/hyperactivity disorder and the explore/exploit trade-off. Neuropsychopharmacology 2021; 46:614-621. [PMID: 33040092 PMCID: PMC8027173 DOI: 10.1038/s41386-020-00881-8] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/19/2020] [Revised: 09/22/2020] [Accepted: 09/29/2020] [Indexed: 11/09/2022]
Abstract
The ability to maximize rewards and minimize the costs of obtaining them is vital to making advantageous explore/exploit decisions. Exploratory decisions are theorized to be greater among individuals with attention-deficit/hyperactivity disorder (ADHD), potentially due to deficient catecholamine transmission. Here, we examined the effects of ADHD status and methylphenidate, a common ADHD medication, on explore/exploit decisions using a 6-armed bandit task. We hypothesized that ADHD participants would make more exploratory decisions than controls, and that MPH would reduce group differences. On separate study days, adults with (n = 26) and without (n = 23) ADHD completed the bandit task at baseline, and after methylphenidate or placebo in counter-balanced order. Explore/exploit decisions were modeled using reinforcement learning algorithms. ADHD participants made more exploratory decisions (i.e., chose options without the highest expected reward value) and earned fewer points than controls in all three study days, and methylphenidate did not affect these outcomes. Baseline exploratory choices were positively associated with hyperactive ADHD symptoms across all participants. These results support several theoretical models of increased exploratory choices in ADHD and suggest the unexplained variance in ADHD decisions may be due to less value tracking. The inability to suppress actions with little to no reward value may be a key feature of hyperactive ADHD symptoms.
Collapse
|
35
|
Pisupati S, Chartarifsky-Lynn L, Khanal A, Churchland AK. Lapses in perceptual decisions reflect exploration. eLife 2021; 10:55490. [PMID: 33427198 PMCID: PMC7846276 DOI: 10.7554/elife.55490] [Citation(s) in RCA: 40] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2020] [Accepted: 01/10/2021] [Indexed: 12/17/2022] Open
Abstract
Perceptual decision-makers often display a constant rate of errors independent of evidence strength. These ‘lapses’ are treated as a nuisance arising from noise tangential to the decision, e.g. inattention or motor errors. Here, we use a multisensory decision task in rats to demonstrate that these explanations cannot account for lapses’ stimulus dependence. We propose a novel explanation: lapses reflect a strategic trade-off between exploiting known rewarding actions and exploring uncertain ones. We tested this model’s predictions by selectively manipulating one action’s reward magnitude or probability. As uniquely predicted by this model, changes were restricted to lapses associated with that action. Finally, we show that lapses are a powerful tool for assigning decision-related computations to neural structures based on disruption experiments (here, posterior striatum and secondary motor cortex). These results suggest that lapses reflect an integral component of decision-making and are informative about action values in normal and disrupted brain states.
Collapse
Affiliation(s)
- Sashank Pisupati
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York, United States.,CSHL School of Biological Sciences, Cold Spring Harbor, New York, United States
| | - Lital Chartarifsky-Lynn
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York, United States.,CSHL School of Biological Sciences, Cold Spring Harbor, New York, United States
| | - Anup Khanal
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York, United States
| | | |
Collapse
|
36
|
Matikainen-Ankney BA, Earnest T, Ali M, Casey E, Wang JG, Sutton AK, Legaria AA, Barclay KM, Murdaugh LB, Norris MR, Chang YH, Nguyen KP, Lin E, Reichenbach A, Clarke RE, Stark R, Conway SM, Carvalho F, Al-Hasani R, McCall JG, Creed MC, Cazares V, Buczynski MW, Krashes MJ, Andrews ZB, Kravitz AV. An open-source device for measuring food intake and operant behavior in rodent home-cages. eLife 2021; 10:66173. [PMID: 33779547 PMCID: PMC8075584 DOI: 10.7554/elife.66173] [Citation(s) in RCA: 48] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2021] [Accepted: 03/26/2021] [Indexed: 01/26/2023] Open
Abstract
Feeding is critical for survival, and disruption in the mechanisms that govern food intake underlies disorders such as obesity and anorexia nervosa. It is important to understand both food intake and food motivation to reveal mechanisms underlying feeding disorders. Operant behavioral testing can be used to measure the motivational component to feeding, but most food intake monitoring systems do not measure operant behavior. Here, we present a new solution for monitoring both food intake and motivation in rodent home-cages: the Feeding Experimentation Device version 3 (FED3). FED3 measures food intake and operant behavior in rodent home-cages, enabling longitudinal studies of feeding behavior with minimal experimenter intervention. It has a programmable output for synchronizing behavior with optogenetic stimulation or neural recordings. Finally, FED3 design files are open-source and freely available, allowing researchers to modify FED3 to suit their needs.
Collapse
Affiliation(s)
| | - Thomas Earnest
- Department of Psychiatry, Washington University in St. LouisSt. LouisUnited States
| | - Mohamed Ali
- National Institute of Diabetes and Digestive and Kidney DiseasesBethesdaUnited States,Department of Bioengineering, University of MarylandCollege ParkUnited States
| | - Eric Casey
- Department of Psychiatry, Washington University in St. LouisSt. LouisUnited States
| | - Justin G Wang
- Department of Neuroscience, Washington University in St. LouisSt. LouisUnited States
| | - Amy K Sutton
- National Institute of Diabetes and Digestive and Kidney DiseasesBethesdaUnited States
| | - Alex A Legaria
- Department of Neuroscience, Washington University in St. LouisSt. LouisUnited States
| | - Kia M Barclay
- Department of Neuroscience, Washington University in St. LouisSt. LouisUnited States
| | - Laura B Murdaugh
- Department of Neuroscience, Virginia Polytechnic and State UniversityBlacksburgUnited States
| | - Makenzie R Norris
- Department of Neuroscience, Washington University in St. LouisSt. LouisUnited States,Center for Clinical Pharmacology, University of Health Sciences and PharmacySt. LouisUnited States
| | - Yu-Hsuan Chang
- Department of Neuroscience, Washington University in St. LouisSt. LouisUnited States
| | - Katrina P Nguyen
- National Institute of Diabetes and Digestive and Kidney DiseasesBethesdaUnited States
| | - Eric Lin
- Department of Psychiatry, Washington University in St. LouisSt. LouisUnited States
| | | | | | - Romana Stark
- Department of Physiology, Monash UniversityClaytonAustralia
| | - Sineadh M Conway
- Center for Clinical Pharmacology, University of Health Sciences and PharmacySt. LouisUnited States,Department of Anesthesiology, Washington University in St. LouisSt. LouisUnited States
| | | | - Ream Al-Hasani
- Center for Clinical Pharmacology, University of Health Sciences and PharmacySt. LouisUnited States,Department of Anesthesiology, Washington University in St. LouisSt. LouisUnited States
| | - Jordan G McCall
- Center for Clinical Pharmacology, University of Health Sciences and PharmacySt. LouisUnited States,Department of Anesthesiology, Washington University in St. LouisSt. LouisUnited States
| | - Meaghan C Creed
- Department of Psychiatry, Washington University in St. LouisSt. LouisUnited States,Department of Neuroscience, Washington University in St. LouisSt. LouisUnited States,Department of Anesthesiology, Washington University in St. LouisSt. LouisUnited States
| | - Victor Cazares
- Department of Psychology, Williams CollegeWilliamstownUnited States
| | - Matthew W Buczynski
- Department of Neuroscience, Virginia Polytechnic and State UniversityBlacksburgUnited States
| | - Michael J Krashes
- National Institute of Diabetes and Digestive and Kidney DiseasesBethesdaUnited States
| | - Zane B Andrews
- Department of Physiology, Monash UniversityClaytonAustralia
| | - Alexxai V Kravitz
- Department of Psychiatry, Washington University in St. LouisSt. LouisUnited States,Department of Neuroscience, Washington University in St. LouisSt. LouisUnited States,Department of Anesthesiology, Washington University in St. LouisSt. LouisUnited States
| |
Collapse
|
37
|
Distinct roles for dopamine clearance mechanisms in regulating behavioral flexibility. Mol Psychiatry 2021; 26:7188-7199. [PMID: 34193974 PMCID: PMC8872990 DOI: 10.1038/s41380-021-01194-y] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/08/2021] [Revised: 05/21/2021] [Accepted: 06/08/2021] [Indexed: 02/06/2023]
Abstract
Dopamine plays a crucial role in adaptive behavior, and dysfunctional dopamine is implicated in multiple psychiatric conditions characterized by inflexible or inconsistent choices. However, the precise relationship between dopamine and flexible decision making remains unclear. One reason is that, while many studies have focused on the activity of dopamine neurons, efficient dopamine signaling also relies on clearance mechanisms, notably the dopamine transporter (DAT), which predominates in striatum, and catechol-O-methyltransferase (COMT), which predominates in cortex. The exact locus, extent, and timescale of the effects of DAT and COMT are uncertain. Moreover, there is limited data on how acute disruption of either mechanism affects flexible decision making strategies mediated by cortico-striatal networks. To address these issues, we combined pharmacological modulation of DAT and COMT with electrochemistry and behavior in mice. DAT blockade, but not COMT inhibition, regulated sub-second dopamine release in the nucleus accumbens core, but surprisingly neither clearance mechanism affected evoked release in prelimbic cortex. This was not due to a lack of sensitivity, as both amphetamine and atomoxetine changed the kinetics of sub-second release. In a multi-step decision making task where mice had to respond to reversals in either reward probabilities or the choice sequence to reach the goal, DAT blockade selectively impaired, and COMT inhibition improved, performance after reward reversals, but neither manipulation affected the adaptation of choices after action-state transition reversals. Together, our data suggest that DAT and COMT shape specific aspects of behavioral flexibility by regulating different aspects of the kinetics of striatal and cortical dopamine, respectively.
Collapse
|
38
|
Roh M, Lee H, Seo H, Lim CS, Park P, Choi JE, Kwak JH, Lee J, Kaang BK, McHugh TJ, Lee K. Perseverative stereotypic behavior of Epac2 KO mice in a reward-based decision making task. Neurosci Res 2020; 161:8-17. [PMID: 33007326 DOI: 10.1016/j.neures.2020.08.010] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2020] [Revised: 08/14/2020] [Accepted: 08/21/2020] [Indexed: 11/18/2022]
Abstract
Successfully navigating dynamic environments requires balancing the decision to stay at an optimal choice with that to switch to an alternative to acquire new knowledge. However, the genetic factors and cellular activity shaping this "stay or switch" action decision remains largely unidentified. Here we find that mice carrying a deletion of the exchange protein directly activated by cAMP 2 (Epac2) gene, a putative autism locus, exhibit perseverative "stay" behavior in a dynamic foraging task. Anatomical analysis found that the loss of Epac2 resulted in a significant decrease in the density of PV-expressing interneurons in the ventrolateral orbitofrontal cortex (OFC) and dorsal striatum (dSTR). Further, in vitro whole cell patch clamp recordings of PV+ GABAergic interneurons in the dSTR revealed altered neural activity in Epac2 KO mice in response to dopamine. Our findings highlight a potential role of Epac2 in structural changes and neural responses of PV-expressing GABAergic interneurons in the ventrolateral OFC and dSTR during value-based reinforcement learning and link Epac2 function to abnormal decision-making processes and perseverative behaviors seen in autism.
Collapse
Affiliation(s)
- Mootaek Roh
- Department of Anatomy, Brain Science & Engineering Institute, School of Medicine, Kyungpook National University, 680 Gukchaebosang-ro, Jung-gu, Daegu 41944, South Korea
| | - Hyunjung Lee
- Department of Anatomy, Brain Science & Engineering Institute, School of Medicine, Kyungpook National University, 680 Gukchaebosang-ro, Jung-gu, Daegu 41944, South Korea
| | - Hyunhyo Seo
- Department of Anatomy, Brain Science & Engineering Institute, School of Medicine, Kyungpook National University, 680 Gukchaebosang-ro, Jung-gu, Daegu 41944, South Korea
| | - Chae-Seok Lim
- School of Biological Sciences, Seoul National University, 1 Gwanangno, Gwanak-gu, Seoul 08826, South Korea; Department of Pharmacology, Wonkwang University School of Medicine, 460 Iksandae-ro, Iksan, Jeonbuk 54538, South Korea
| | - Pojeong Park
- School of Biological Sciences, Seoul National University, 1 Gwanangno, Gwanak-gu, Seoul 08826, South Korea
| | - Ja Eun Choi
- School of Biological Sciences, Seoul National University, 1 Gwanangno, Gwanak-gu, Seoul 08826, South Korea
| | - Ji-Hye Kwak
- Department of Anatomy, Brain Science & Engineering Institute, School of Medicine, Kyungpook National University, 680 Gukchaebosang-ro, Jung-gu, Daegu 41944, South Korea
| | - Juhyun Lee
- Department of Anatomy, Brain Science & Engineering Institute, School of Medicine, Kyungpook National University, 680 Gukchaebosang-ro, Jung-gu, Daegu 41944, South Korea
| | - Bong-Kiun Kaang
- School of Biological Sciences, Seoul National University, 1 Gwanangno, Gwanak-gu, Seoul 08826, South Korea
| | - Thomas J McHugh
- Laboratory for Circuit and Behavioral Physiology, RIKEN Center for Brain Science, 2-1, Hirosawa, Wako, Saitama 351-0198, Japan.
| | - Kyungmin Lee
- Department of Anatomy, Brain Science & Engineering Institute, School of Medicine, Kyungpook National University, 680 Gukchaebosang-ro, Jung-gu, Daegu 41944, South Korea.
| |
Collapse
|
39
|
Chakroun K, Mathar D, Wiehler A, Ganzer F, Peters J. Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making. eLife 2020; 9:e51260. [PMID: 32484779 PMCID: PMC7266623 DOI: 10.7554/elife.51260] [Citation(s) in RCA: 45] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2019] [Accepted: 05/01/2020] [Indexed: 01/15/2023] Open
Abstract
Involvement of dopamine in regulating exploration during decision-making has long been hypothesized, but direct causal evidence in humans is still lacking. Here, we use a combination of computational modeling, pharmacological intervention and functional magnetic resonance imaging to address this issue. Thirty-one healthy male participants performed a restless four-armed bandit task in a within-subjects design under three drug conditions: 150 mg of the dopamine precursor L-dopa, 2 mg of the D2 receptor antagonist haloperidol, and placebo. Choices were best explained by an extension of an established Bayesian learning model accounting for perseveration, directed exploration and random exploration. Modeling revealed attenuated directed exploration under L-dopa, while neural signatures of exploration, exploitation and prediction error were unaffected. Instead, L-dopa attenuated neural representations of overall uncertainty in insula and dorsal anterior cingulate cortex. Our results highlight the computational role of these regions in exploration and suggest that dopamine modulates how this circuit tracks accumulating uncertainty during decision-making.
Collapse
Affiliation(s)
- Karima Chakroun
- Department of Systems Neuroscience, University Medical Center Hamburg-EppendorfHamburgGermany
| | - David Mathar
- Department of Psychology, Biological Psychology, University of CologneCologneGermany
| | - Antonius Wiehler
- Department of Systems Neuroscience, University Medical Center Hamburg-EppendorfHamburgGermany
- Institut du Cerveau et de la Moelle épinière - ICM, Centre de NeuroImagerie de Recherche - CENIR, Sorbonne Universités, Groupe Hospitalier Pitié-SalpêtrièreParisFrance
| | - Florian Ganzer
- German Center for Addiction Research in Childhood and Adolescence, University Medical Center Hamburg-EppendorfHamburgGermany
| | - Jan Peters
- Department of Systems Neuroscience, University Medical Center Hamburg-EppendorfHamburgGermany
- Department of Psychology, Biological Psychology, University of CologneCologneGermany
| |
Collapse
|
40
|
Adams RA, Moutoussis M, Nour MM, Dahoun T, Lewis D, Illingworth B, Veronese M, Mathys C, de Boer L, Guitart-Masip M, Friston KJ, Howes OD, Roiser JP. Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models. Cereb Cortex 2020; 30:3573-3589. [PMID: 32083297 PMCID: PMC7233027 DOI: 10.1093/cercor/bhz327] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 11/18/2019] [Accepted: 12/05/2019] [Indexed: 12/17/2022] Open
Abstract
Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans.
Collapse
Affiliation(s)
- Rick A Adams
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
- Division of Psychiatry, University College London, London W1T 7NF, UK
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
| | - Michael Moutoussis
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
| | - Matthew M Nour
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King’s College London, London SE5 8AF, UK
| | - Tarik Dahoun
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford OX3 7JX, UK
| | - Declan Lewis
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| | - Benjamin Illingworth
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| | - Mattia Veronese
- Centre for Neuroimaging Sciences, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King's College London, London SE5 8AF, UK
| | - Christoph Mathys
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
- Scuola Internazionale Superiore di Studi Avanzati (SISSA), 34136 Trieste, Italy
- Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich and ETH Zurich, 8032 Zurich, Switzerland
| | - Lieke de Boer
- Aging Research Center, Karolinska Institute, 171 65 Stockholm, Sweden
| | - Marc Guitart-Masip
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
- Aging Research Center, Karolinska Institute, 171 65 Stockholm, Sweden
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK
| | - Oliver D Howes
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King’s College London, London SE5 8AF, UK
| | - Jonathan P Roiser
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| |
Collapse
|
41
|
Trempler I, Bürkner PC, El-Sourani N, Binder E, Reker P, Fink GR, Schubotz RI. Impaired context-sensitive adjustment of behaviour in Parkinson's disease patients tested on and off medication: An fMRI study. Neuroimage 2020; 212:116674. [PMID: 32097724 DOI: 10.1016/j.neuroimage.2020.116674] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2019] [Revised: 02/18/2020] [Accepted: 02/19/2020] [Indexed: 10/24/2022] Open
Abstract
The brain's sensitivity to and accentuation of unpredicted over predicted sensory signals plays a fundamental role in learning. According to recent theoretical models of the predictive coding framework, dopamine is responsible for balancing the interplay between bottom-up input and top-down predictions by controlling the precision of surprise signals that guide learning. Using functional MRI, we investigated whether patients with Parkinson's disease (PD) show impaired learning from prediction errors requiring either adaptation or stabilisation of current predictions. Moreover, we were interested in whether deficits in learning over a specific time scale would be accompanied by altered surprise responses in dopamine-related brain structures. To this end, twenty-one PD patients tested on and off dopaminergic medication and twenty-one healthy controls performed a digit prediction paradigm. During the task, violations of sequence-based predictions either signalled the need to update or to stabilise the current prediction and, thus, to react to them or ignore them, respectively. To investigate contextual adaptation to prediction errors, the probability (or its inverse, surprise) of the violations fluctuated across the experiment. When the probability of prediction errors over a specific time scale increased, healthy controls but not PD patients off medication became more flexible, i.e., error rates at violations requiring a motor response decreased in controls but increased in patients. On the neural level, this learning deficit in patients was accompanied by reduced signalling in the substantia nigra and the caudate nucleus. In contrast, differences between the groups regarding the probabilistic modulation of behaviour and neural responses were much less pronounced at prediction errors requiring only stabilisation but no adaptation. Interestingly, dopaminergic medication could neither improve learning from prediction errors nor restore the physiological, neurotypical pattern. Our findings point to a pivotal role of dysfunctions of the substantia nigra and caudate nucleus in deficits in learning from flexibility-demanding prediction errors in PD. Moreover, the data witness poor effects of dopaminergic medication on learning in PD.
Collapse
Affiliation(s)
- Ima Trempler
- Department of Psychology, University of Muenster, 48149, Münster, Germany; Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Muenster, 48149, Münster, Germany.
| | | | - Nadiya El-Sourani
- Department of Psychology, University of Muenster, 48149, Münster, Germany
| | - Ellen Binder
- Faculty of Medicine and University Hospital Cologne, Department of Neurology, 50937, Cologne, Germany; Institute of Neuroscience and Medicine (INM3), Cognitive Neuroscience, Research Centre Jülich, 52425, Jülich, Germany
| | - Paul Reker
- Faculty of Medicine and University Hospital Cologne, Department of Neurology, 50937, Cologne, Germany
| | - Gereon R Fink
- Faculty of Medicine and University Hospital Cologne, Department of Neurology, 50937, Cologne, Germany; Institute of Neuroscience and Medicine (INM3), Cognitive Neuroscience, Research Centre Jülich, 52425, Jülich, Germany
| | - Ricarda I Schubotz
- Department of Psychology, University of Muenster, 48149, Münster, Germany; Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Muenster, 48149, Münster, Germany; Faculty of Medicine and University Hospital Cologne, Department of Neurology, 50937, Cologne, Germany
| |
Collapse
|
42
|
Rubin JE, Vich C, Clapp M, Noneman K, Verstynen T. The credit assignment problem in cortico‐basal ganglia‐thalamic networks: A review, a problem and a possible solution. Eur J Neurosci 2020; 53:2234-2253. [DOI: 10.1111/ejn.14745] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2020] [Revised: 03/23/2020] [Accepted: 03/25/2020] [Indexed: 12/21/2022]
Affiliation(s)
- Jonathan E. Rubin
- Department of Mathematics Center for the Neural Basis of Cognition University of Pittsburgh Pittsburgh PA USA
| | - Catalina Vich
- Department de Matemàtiques i Informàtica Institute of Applied Computing and Community Code Universitat de les Illes Balears Palma Spain
| | - Matthew Clapp
- Carnegie Mellon Neuroscience Institute Carnegie Mellon University Pittsburgh PA USA
| | - Kendra Noneman
- Micron School of Materials Science and Engineering Boise State University Boise ID USA
| | - Timothy Verstynen
- Carnegie Mellon Neuroscience Institute Carnegie Mellon University Pittsburgh PA USA
- Department of Psychology Center for the Neural Basis of Cognition Carnegie Mellon University Pittsburgh PA USA
| |
Collapse
|
43
|
Zack M, St George R, Clark L. Dopaminergic signaling of uncertainty and the aetiology of gambling addiction. Prog Neuropsychopharmacol Biol Psychiatry 2020; 99:109853. [PMID: 31870708 DOI: 10.1016/j.pnpbp.2019.109853] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/27/2019] [Revised: 12/03/2019] [Accepted: 12/20/2019] [Indexed: 11/15/2022]
Abstract
Although there is increasing clinical recognition of behavioral addictions, of which gambling disorder is the prototype example, there is a limited understanding of the psychological properties of (non-substance-related) behaviors that enable them to become 'addictive' in a way that is comparable to drugs of abuse. According to an influential application of reinforcement learning to substance addictions, the direct effects of drugs to release dopamine can create a perpetual escalation of incentive salience. This article focusses on reward uncertainty, which is proposed to be the core feature of gambling that creates the capacity for addiction. We describe the neuro-dynamics of the dopamine response to uncertainty that may allow a similar escalation of incentive salience, and its relevance to behavioral addictions. We review translational evidence from both preclinical animal models and human clinical research, including studies in people with gambling disorder. Further, we describe the evidence for 1) the effects of the omission of expected reward as a stressor and to promote sensitization, 2) the effect of the resolution of reward uncertainty as a source of value, 3) structural characteristics of modern Electronic Gaming Machines (EGMs) in leveraging these mechanisms, 4) analogies to the aberrant salience hypothesis of psychosis for creating and maintaining gambling-related cognitive distortions. This neurobiologically-inspired model has implications for harm profiling of other putative behavioral addictions, as well as offering avenues for enhancing neurological, pharmacological and psychological treatments for gambling disorder, and harm reduction strategies for EGM design.
Collapse
Affiliation(s)
- Martin Zack
- Centre for Addiction and Mental Health, 33 Russell St, Toronto, ON M5S 2S1, Canada; Department of Pharmacology & Toxicology, University of Toronto, 1 King's College Circle, Toronto, ON M5S 1A8, Canada.
| | - Ross St George
- Department of Psychology, University of British Columbia, Okanagan Campus, 3333 University Way, Kelowna, BC V1V 1V7, Canada.
| | - Luke Clark
- Centre for Gambling Research at UBC, Department of Psychology, University of British Columbia, 2136 West Mall, Vancouver, BC V6T 1Z4, Canada.
| |
Collapse
|
44
|
Song MR, Lee SW. Dynamic resource allocation during reinforcement learning accounts for ramping and phasic dopamine activity. Neural Netw 2020; 126:95-107. [PMID: 32203877 DOI: 10.1016/j.neunet.2020.03.005] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2019] [Revised: 01/22/2020] [Accepted: 03/02/2020] [Indexed: 11/29/2022]
Abstract
For an animal to learn about its environment with limited motor and cognitive resources, it should focus its resources on potentially important stimuli. However, too narrow focus is disadvantageous for adaptation to environmental changes. Midbrain dopamine neurons are excited by potentially important stimuli, such as reward-predicting or novel stimuli, and allocate resources to these stimuli by modulating how an animal approaches, exploits, explores, and attends. The current study examined the theoretical possibility that dopamine activity reflects the dynamic allocation of resources for learning. Dopamine activity may transition between two patterns: (1) phasic responses to cues and rewards, and (2) ramping activity arising as the agent approaches the reward. Phasic excitation has been explained by prediction errors generated by experimentally inserted cues. However, when and why dopamine activity transitions between the two patterns remain unknown. By parsimoniously modifying a standard temporal difference (TD) learning model to accommodate a mixed presentation of both experimental and environmental stimuli, we simulated dopamine transitions and compared them with experimental data from four different studies. The results suggested that dopamine transitions from ramping to phasic patterns as the agent focuses its resources on a small number of reward-predicting stimuli, thus leading to task dimensionality reduction. The opposite occurs when the agent re-distributes its resources to adapt to environmental changes, resulting in task dimensionality expansion. This research elucidates the role of dopamine in a broader context, providing a potential explanation for the diverse repertoire of dopamine activity that cannot be explained solely by prediction error.
Collapse
Affiliation(s)
- Minryung R Song
- Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, South Korea
| | - Sang Wan Lee
- Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, South Korea; Program of Brain and Cognitive Engineering, Daejeon, 34141, South Korea; KAIST Institute for Health, Science, and Technology, Daejeon, 34141, South Korea; KAIST Institute for Artificial Intelligence, Daejeon, 34141, South Korea; KAIST Center for Neuroscience-inspired AI, Daejeon, 34141, South Korea.
| |
Collapse
|
45
|
Viviani R, Dommes L, Bosch J, Steffens M, Paul A, Schneider KL, Stingl JC, Beschoner P. Signals of anticipation of reward and of mean reward rates in the human brain. Sci Rep 2020; 10:4287. [PMID: 32152378 PMCID: PMC7062891 DOI: 10.1038/s41598-020-61257-y] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2019] [Accepted: 02/23/2020] [Indexed: 01/14/2023] Open
Abstract
Theoretical models of dopamine function stemming from reinforcement learning theory have emphasized the importance of prediction errors, which signal changes in the expectation of impending rewards. Much less is known about the effects of mean reward rates, which may be of motivational significance due to their role in computing the optimal effort put into exploiting reward opportunities. Here, we used a reinforcement learning model to design three functional neuroimaging studies and disentangle the effects of changes in reward expectations and mean reward rates, showing recruitment of specific regions in the brainstem regardless of prediction errors. While changes in reward expectations activated ventral striatal areas as in previous studies, mean reward rates preferentially modulated the substantia nigra/ventral tegmental area, deep layers of the superior colliculi, and a posterior pontomesencephalic region. These brainstem structures may work together to set motivation and attentional efforts levels according to perceived reward opportunities.
Collapse
Affiliation(s)
- Roberto Viviani
- Institute of Psychology, University of Innsbruck, 6020, Innsbruck, Austria.
- Department of Psychiatry and Psychotherapy III, University of Ulm, 89075, Ulm, Germany.
| | - Lisa Dommes
- Department of Psychiatry and Psychotherapy III, University of Ulm, 89075, Ulm, Germany
| | - Julia Bosch
- Department of Psychiatry and Psychotherapy III, University of Ulm, 89075, Ulm, Germany
| | - Michael Steffens
- Research Division, Federal Institute for Drugs and Medical Devices (BfArM), 53175, Bonn, Germany
| | - Anna Paul
- Research Division, Federal Institute for Drugs and Medical Devices (BfArM), 53175, Bonn, Germany
| | - Katharina L Schneider
- Research Division, Federal Institute for Drugs and Medical Devices (BfArM), 53175, Bonn, Germany
| | - Julia C Stingl
- Institute of Clinical Pharmacology, University Hospital of RWTH Aachen, Aachen, Germany
| | - Petra Beschoner
- Department of Psychosomatic Medicine and Psychotherapy, University of Ulm, 89075, Ulm, Germany
| |
Collapse
|
46
|
McGuigan S, Zhou SH, Brosnan MB, Thyagarajan D, Bellgrove MA, Chong TTJ. Dopamine restores cognitive motivation in Parkinson's disease. Brain 2020; 142:719-732. [PMID: 30689734 DOI: 10.1093/brain/awy341] [Citation(s) in RCA: 49] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2018] [Revised: 11/06/2018] [Accepted: 11/07/2018] [Indexed: 11/14/2022] Open
Abstract
Disorders of motivation, such as apathy, are common in Parkinson's disease, and a key feature of such disorders is a greater aversion to effort. In humans, the experience of cognitive effort is ubiquitous, and cognitive apathy has traditionally been considered distinct and separable from other subtypes. Surprisingly, however, the neurobiology of cognitive motivation is poorly understood. In particular, although dopamine has a well-characterized role in incentivizing physically effortful behaviour, a critical, unresolved issue is whether its facilitatory role generalizes to other domains. Here, we asked how dopamine modulates the willingness of patients with Parkinson's disease to invest cognitive effort in return for reward. We tested 20 patients with idiopathic Parkinson's disease across two counterbalanced sessions-ON and OFF their usual dopaminergic medication-and compared their performance to 20 healthy age-matched controls. We applied a novel task in which we manipulated cognitive effort as the number of rapid serial visual presentation streams to which participants had to attend. After training participants to ceiling performance, we then asked them to choose between a low-effort/low-reward baseline option, and a higher-effort/higher-reward offer. Computational models of choice behaviour revealed four key results. First, patients OFF medication were significantly less cognitively motivated than controls, as manifest by steeper cognitive effort discounting functions in the former group. Second, dopaminergic therapy improved this deficit, such that choices in patients ON medication were indistinguishable from controls. Third, differences in motivation were also accompanied by independent changes in the stochasticity of individuals' decisions, such that dopamine reduced the variability in choice behaviour. Finally, choices on our task correlated uniquely with the subscale of the Dimensional Apathy Scale that specifically indexes cognitive motivation, which suggests a close relationship between our laboratory measure of cognitive effort discounting and subjective reports of day-to-day cognitive apathy. Importantly, participants' choices were not confounded by temporal discounting, probability discounting, physical demand, or varying task performance. These results are the first to reveal the central role of dopamine in overcoming cognitive effort costs. They provide an insight into the computational mechanisms underlying cognitive apathy in Parkinson's disease, and demonstrate its amenability to dopaminergic therapy. More broadly, they offer important empirical support for prominent frameworks proposing a domain-general role for dopamine in value-based decision-making, and provide a critical link between dopamine and multidimensional theories of apathy.
Collapse
Affiliation(s)
- Sara McGuigan
- Monash Institute of Cognitive and Clinical Neurosciences, Monash University, Victoria, Australia
| | - Shou-Han Zhou
- Monash Institute of Cognitive and Clinical Neurosciences, Monash University, Victoria, Australia
| | - Méadhbh B Brosnan
- Monash Institute of Cognitive and Clinical Neurosciences, Monash University, Victoria, Australia
| | | | - Mark A Bellgrove
- Monash Institute of Cognitive and Clinical Neurosciences, Monash University, Victoria, Australia
| | - Trevor T-J Chong
- Monash Institute of Cognitive and Clinical Neurosciences, Monash University, Victoria, Australia.,Department of Neurology, Alfred Health, Melbourne, Victoria, Australia.,Department of Clinical Neurosciences, St Vincent's Hospital, Melbourne, Victoria, Australia
| |
Collapse
|
47
|
Van Slooten JC, Jahfari S, Theeuwes J. Spontaneous eye blink rate predicts individual differences in exploration and exploitation during reinforcement learning. Sci Rep 2019; 9:17436. [PMID: 31758031 PMCID: PMC6874684 DOI: 10.1038/s41598-019-53805-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2019] [Accepted: 10/31/2019] [Indexed: 12/13/2022] Open
Abstract
Spontaneous eye blink rate (sEBR) has been linked to striatal dopamine function and to how individuals make value-based choices after a period of reinforcement learning (RL). While sEBR is thought to reflect how individuals learn from the negative outcomes of their choices, this idea has not been tested explicitly. This study assessed how individual differences in sEBR relate to learning by focusing on the cognitive processes that drive RL. Using Bayesian latent mixture modelling to quantify the mapping between RL behaviour and its underlying cognitive processes, we were able to differentiate low and high sEBR individuals at the level of these cognitive processes. Further inspection of these cognitive processes indicated that sEBR uniquely indexed explore-exploit tendencies during RL: lower sEBR predicted exploitative choices for high valued options, whereas higher sEBR predicted exploration of lower value options. This relationship was additionally supported by a network analysis where, notably, no link was observed between sEBR and how individuals learned from negative outcomes. Our findings challenge the notion that sEBR predicts learning from negative outcomes during RL, and suggest that sEBR predicts individual explore-exploit tendencies. These then influence value sensitivity during choices to support successful performance when facing uncertain reward.
Collapse
Affiliation(s)
- Joanne C Van Slooten
- Department of Experimental and Applied Psychology, Vrije Universiteit, Amsterdam, The Netherlands.
| | - Sara Jahfari
- Spinoza Centre for Neuroimaging, Royal Academy of Sciences, Amsterdam, The Netherlands
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
| | - Jan Theeuwes
- Department of Experimental and Applied Psychology, Vrije Universiteit, Amsterdam, The Netherlands
| |
Collapse
|
48
|
Dopamine-Dependent Loss Aversion during Effort-Based Decision-Making. J Neurosci 2019; 40:661-670. [PMID: 31727795 DOI: 10.1523/jneurosci.1760-19.2019] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2019] [Revised: 10/31/2019] [Accepted: 11/01/2019] [Indexed: 11/21/2022] Open
Abstract
From psychology to economics, there has been substantial interest in how costs (e.g., delay, risk) are represented asymmetrically during decision-making when attempting to gain reward or avoid punishment. For example, in decision-making under risk, individuals show a tendency to prefer to avoid punishment rather than to acquire the equivalent reward (loss aversion). Although the cost of physical effort has recently received significant attention, it remains unclear whether loss aversion exists during effort-based decision-making. On the one hand, loss aversion may be hardwired due to asymmetric evolutionary pressure on losses and gains and therefore exists across decision-making contexts. On the other hand, distinct brain regions are involved with different decision costs, making it questionable whether similar asymmetries exist. Here, we demonstrate that young healthy human participants (females, 16; males, 6) exhibit loss aversion during effort-based decision-making by exerting more physical effort to avoid punishment than to gain a same-size reward. Next, we show that medicated Parkinson's disease (PD) patients (females, 9; males, 9) show a reduction in loss aversion compared with age-matched control subjects (females, 11; males, 9). Behavioral and computational analysis revealed that people with PD exerted similar physical effort in return for a reward but were less willing to produce effort to avoid punishment. Therefore, loss aversion is present during effort-based decision-making and can be modulated by altered dopaminergic state. This finding could have important implications for our understanding of clinical disorders that show a reduced willingness to exert effort in the pursuit of reward.SIGNIFICANCE STATEMENT Loss aversion-preferring to avoid punishment rather than to acquire equivalent reward-is an important concept in decision-making under risk. However, little is known about whether loss aversion also exists during decisions where the cost is physical effort. This is surprising given that motor cost shapes human behavior, and a reduced willingness to exert effort is a characteristic of many clinical disorders. Here, we show that healthy human individuals exert more effort to minimize punishment than to maximize reward (loss aversion). We also demonstrate that medicated Parkinson's disease patients exert similar effort to gain reward but less effort to avoid punishment when compared with healthy age-matched control subjects. This indicates that dopamine-dependent loss aversion is crucial for explaining effort-based decision-making.
Collapse
|
49
|
Attentional blink and putative noninvasive dopamine markers: Two experiments to consolidate possible associations. COGNITIVE AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2019; 19:1444-1457. [PMID: 31396846 PMCID: PMC6861702 DOI: 10.3758/s13415-019-00717-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Adaptive behavioral control involves a balance between top-down persistence and flexible updating of goals under changing demands. According to the metacontrol state model (MSM), this balance emerges from the interaction between the frontal and the striatal dopaminergic system. The attentional blink (AB) task has been argued to tap into the interaction between persistence and flexibility, as it reflects overpersistence—the too-exclusive allocation of attentional resources to the processing of the first of two consecutive targets. Notably, previous studies are inconclusive about the association between the AB and noninvasive proxies of dopamine including the spontaneous eye blink rate (sEBR), which allegedly assesses striatal dopamine levels. We aimed to substantiate and extend previous attempts to predict individual sizes of the AB in two separate experiments with larger sample sizes (N = 71 & N = 65) by means of noninvasive behavioral and physiological proxies of dopamine (DA), such as sEBR and mood measures, which are likely to reflect striatal dopamine levels, and color discrimination, which has been argued to tap into the frontal dopamine levels. Our findings did not confirm the prediction that AB size covaries with sEBR, mood, or color discrimination. The implications of this inconsistency with previous observations are discussed.
Collapse
|
50
|
Kwak S, Jung MW. Distinct roles of striatal direct and indirect pathways in value-based decision making. eLife 2019; 8:46050. [PMID: 31310237 PMCID: PMC6658164 DOI: 10.7554/elife.46050] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2019] [Accepted: 07/09/2019] [Indexed: 12/12/2022] Open
Abstract
The striatum is critically involved in value-based decision making. However, it is unclear how striatal direct and indirect pathways work together to make optimal choices in a dynamic and uncertain environment. Here, we examined the effects of selectively inactivating D1 receptor (D1R)- or D2 receptor (D2R)-expressing dorsal striatal neurons (corresponding to direct- and indirect-pathway neurons, respectively) on mouse choice behavior in a reversal task with progressively increasing reversal frequency and a dynamic two-armed bandit task. Inactivation of either D1R- or D2R-expressing striatal neurons impaired performance in both tasks, but the pattern of altered choice behavior differed between the two animal groups. A reinforcement learning model-based analysis indicated that inactivation of D1R- and D2R-expressing striatal neurons selectively impairs value-dependent action selection and value learning, respectively. Our results suggest differential contributions of striatal direct and indirect pathways to two distinct steps in value-based decision making.
Collapse
Affiliation(s)
- Shinae Kwak
- Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon, Republic of Korea.,Department of Biological Sciences, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Min Whan Jung
- Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon, Republic of Korea.,Department of Biological Sciences, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| |
Collapse
|