1
|
Ihara K, Shikano Y, Kato S, Yagishita S, Tanaka KF, Takata N. A reinforcement learning model with choice traces for a progressive ratio schedule. Front Behav Neurosci 2024; 17:1302842. [PMID: 38268795 PMCID: PMC10806202 DOI: 10.3389/fnbeh.2023.1302842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Accepted: 12/13/2023] [Indexed: 01/26/2024] Open
Abstract
The progressive ratio (PR) lever-press task serves as a benchmark for assessing goal-oriented motivation. However, a well-recognized limitation of the PR task is that only a single data point, known as the breakpoint, is obtained from an entire session as a barometer of motivation. Because the breakpoint is defined as the final ratio of responses achieved in a PR session, variations in choice behavior during the PR task cannot be captured. We addressed this limitation by constructing four reinforcement learning models: a simple Q-learning model, an asymmetric model with two learning rates, a perseverance model with choice traces, and a perseverance model without learning. These models incorporated three behavioral choices: reinforced and non-reinforced lever presses and void magazine nosepokes, because we noticed that male mice performed frequent magazine nosepokes during PR tasks. The best model was the perseverance model, which predicted a gradual reduction in amplitudes of reward prediction errors (RPEs) upon void magazine nosepokes. We confirmed the prediction experimentally with fiber photometry of extracellular dopamine (DA) dynamics in the ventral striatum of male mice using a fluorescent protein (genetically encoded GPCR activation-based DA sensor: GRABDA2m). We verified application of the model by acute intraperitoneal injection of low-dose methamphetamine (METH) before a PR task, which increased the frequency of magazine nosepokes during the PR session without changing the breakpoint. The perseverance model captured behavioral modulation as a result of increased initial action values, which are customarily set to zero and disregarded in reinforcement learning analysis. Our findings suggest that the perseverance model reveals the effects of psychoactive drugs on choice behaviors during PR tasks.
Collapse
Affiliation(s)
- Keiko Ihara
- Division of Brain Sciences, Institute for Advanced Medical Research, Keio University School of Medicine, Tokyo, Japan
| | - Yu Shikano
- Division of Brain Sciences, Institute for Advanced Medical Research, Keio University School of Medicine, Tokyo, Japan
- Department of Biology, Stanford University, Stanford, CA, United States
| | - Sae Kato
- Division of Brain Sciences, Institute for Advanced Medical Research, Keio University School of Medicine, Tokyo, Japan
| | - Sho Yagishita
- Center for Disease Biology and Integrative Medicine, Faculty of Medicine, The University of Tokyo, Tokyo, Japan
| | - Kenji F. Tanaka
- Division of Brain Sciences, Institute for Advanced Medical Research, Keio University School of Medicine, Tokyo, Japan
| | - Norio Takata
- Division of Brain Sciences, Institute for Advanced Medical Research, Keio University School of Medicine, Tokyo, Japan
| |
Collapse
|
2
|
Limanaqi F, Busceti CL, Celli R, Biagioni F, Fornai F. Autophagy as a gateway for the effects of methamphetamine: From neurotransmitter release and synaptic plasticity to psychiatric and neurodegenerative disorders. Prog Neurobiol 2021; 204:102112. [PMID: 34171442 DOI: 10.1016/j.pneurobio.2021.102112] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Revised: 02/27/2021] [Accepted: 06/18/2021] [Indexed: 02/07/2023]
Abstract
As a major eukaryotic cell clearing machinery, autophagy grants cell proteostasis, which is key for neurotransmitter release, synaptic plasticity, and neuronal survival. In line with this, besides neuropathological events, autophagy dysfunctions are bound to synaptic alterations that occur in mental disorders, and early on, in neurodegenerative diseases. This is also the case of methamphetamine (METH) abuse, which leads to psychiatric disturbances and neurotoxicity. While consistently altering the autophagy machinery, METH produces behavioral and neurotoxic effects through molecular and biochemical events that can be recapitulated by autophagy blockade. These consist of altered physiological dopamine (DA) release, abnormal stimulation of DA and glutamate receptors, as well as oxidative, excitotoxic, and neuroinflammatory events. Recent molecular insights suggest that METH early impairs the autophagy machinery, though its functional significance remains to be investigated. Here we discuss evidence suggesting that alterations of DA transmission and autophagy are intermingled within a chain of events underlying behavioral alterations and neurodegenerative phenomena produced by METH. Understanding how METH alters the autophagy machinery is expected to provide novel insights into the neurobiology of METH addiction sharing some features with psychiatric disorders and parkinsonism.
Collapse
Affiliation(s)
- Fiona Limanaqi
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, Via Roma, 55, 56126, Pisa, PI, Italy
| | | | - Roberta Celli
- IRCCS Neuromed, Via Atinense 18, 86077 Pozzilli, IS, Italy
| | | | - Francesco Fornai
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, Via Roma, 55, 56126, Pisa, PI, Italy; IRCCS Neuromed, Via Atinense 18, 86077 Pozzilli, IS, Italy.
| |
Collapse
|
3
|
Abstract
Methamphetamine abuse leads to devastating consequences, including addiction, crime, and death. Despite decades of research, no medication has been approved by the U.S. Food and Drug Administration for the treatment of Methamphetamine Use Disorder. Thus, there is a need for new therapeutic approaches. Animal studies demonstrate that methamphetamine exposure dysregulates forebrain function involving the Group-I metabotropic glutamate receptor subtype 5 (mGlu5), which is predominantly localized to postsynaptic sites. Allosteric modulators of mGlu5 offer a unique opportunity to modulate glutamatergic neurotransmission selectively, thereby potentially ameliorating methamphetamine-induced disruptions. Negative allosteric modulators of mGlu5 attenuate the effects of methamphetamine, including rewarding/reinforcing properties of the drug across animal models, and have shown promising effects in clinical trials for Anxiety Disorder and Major Depressive Disorder. Preclinical studies have also sparked great interest in mGlu5 positive allosteric modulators, which exhibit antipsychotic and anxiolytic properties, and facilitate extinction learning when access to methamphetamine is removed, possibly via the amelioration of methamphetamine-induced cognitive deficits. Clinical research is now needed to elucidate the mechanisms underlying the mGlu5 receptor-related effects of methamphetamine and the contributions of these effects to addictive behaviors. The growing array of mGlu5 allosteric modulators provides excellent tools for this purpose and may offer the prospect of developing tailored and effective medications for Methamphetamine Use Disorder.
Collapse
|
4
|
Robinson MJF, Caplan KA, Knes AS, Rodríguez-Cruz HO, Clibanoff C, Freeland CM. Reward uncertainty attributes incentive value to reward proximal cues, while amphetamine sensitization reverts attention to more predictive reward distal cues. Prog Neuropsychopharmacol Biol Psychiatry 2020; 97:109795. [PMID: 31669548 DOI: 10.1016/j.pnpbp.2019.109795] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/02/2019] [Revised: 10/03/2019] [Accepted: 10/19/2019] [Indexed: 02/05/2023]
Abstract
Slot-machine gambling incorporates numerous audiovisual cues prior to and during reward delivery (e.g. spinning wheels, flashing lights, celebratory sounds). Over time, these cues may motivate playing and even elicit cravings and relapse in those affected by gambling disorder. Animal studies suggest a heightened attraction to these cues despite diminished predictive ability under reward uncertainty, as evidenced by sign-tracking behavior in rats. Repeated amphetamine administration may also enhance the incentive value attributed to cues. Here, we explored the impact of reward uncertainty and prior amphetamine sensitization on the relative attractiveness and conditioned reinforcing properties of serial Pavlovian cues with different degrees of predictive and incentive value in rats. Animals were sensitized through repeated injections of amphetamine (1-4 mg/kg) or saline and then trained in a Pavlovian autoshaping task involving two sequential lever-auditory cue combinations (CS1, CS2) under Certain (100%-1) or Uncertain (50%-1-2-3) reward conditions. Subsequently, we evaluated the impact of acute amphetamine exposure on cue attraction. Our results suggest that Uncertainty alone enhanced attraction towards the reward-proximal CS2. However, combined sensitization and Uncertainty reversed cue preference relative to Uncertainty alone, enhancing attraction towards the more predictive reward-distal CS1. Both cues acquired conditioned reinforcing properties, despite the CS2 being otherwise ignored in all groups besides Uncertainty. However, combined sensitization and Uncertainty increased the reinforcing value of both cues and doubled the amount of interaction with the CS1 lever per presentation. Our results imply competitive mechanisms for attributing incentive value to gambling-related cues between reward uncertainty, prior amphetamine sensitization, and acute amphetamine administration.
Collapse
Affiliation(s)
- Mike J F Robinson
- Department of Psychology, Wesleyan University, 207 High Street, Middletown, CT 06459, USA; Neuroscience & Behavior Program, Wesleyan University, Middletown, CT 06459, USA.
| | - Kian A Caplan
- Neuroscience & Behavior Program, Wesleyan University, Middletown, CT 06459, USA
| | - Anna S Knes
- Department of Psychology, Wesleyan University, 207 High Street, Middletown, CT 06459, USA; Neuroscience & Behavior Program, Wesleyan University, Middletown, CT 06459, USA
| | | | - Callie Clibanoff
- Department of Psychology, Wesleyan University, 207 High Street, Middletown, CT 06459, USA; Neuroscience & Behavior Program, Wesleyan University, Middletown, CT 06459, USA
| | - Charlotte M Freeland
- Neuroscience & Behavior Program, Wesleyan University, Middletown, CT 06459, USA; Department of Biology, Wesleyan University, 52 Lawn Avenue, Middletown, CT 06459, USA
| |
Collapse
|
5
|
Adverse neuropsychiatric development following perinatal brain injury: from a preclinical perspective. Pediatr Res 2019; 85:198-215. [PMID: 30367160 DOI: 10.1038/s41390-018-0222-6] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/17/2018] [Revised: 10/11/2018] [Accepted: 10/15/2018] [Indexed: 02/06/2023]
Abstract
Perinatal brain injury is a leading cause of death and disability in young children. Recent advances in obstetrics, reproductive medicine and neonatal intensive care have resulted in significantly higher survival rates of preterm or sick born neonates, at the price of increased prevalence of neurological, behavioural and psychiatric problems in later life. Therefore, the current focus of experimental research shifts from immediate injury processes to the consequences for brain function in later life. The aetiology of perinatal brain injury is multi-factorial involving maternal and also labour-associated factors, including not only placental insufficiency and hypoxia-ischaemia but also exposure to high oxygen concentrations, maternal infection yielding excess inflammation, genetic factors and stress as important players, all of them associated with adverse long-term neurological outcome. Several animal models addressing these noxious stimuli have been established in the past to unravel the underlying molecular and cellular mechanisms of altered brain development. In spite of substantial efforts to investigate short-term consequences, preclinical evaluation of the long-term sequelae for the development of cognitive and neuropsychiatric disorders have rarely been addressed. This review will summarise and discuss not only current evidence but also requirements for experimental research providing a causal link between insults to the developing brain and long-lasting neurodevelopmental disorders.
Collapse
|
6
|
Xu S, Tu S, Gao J, Liu J, Guo Z, Zhang J, Liu X, Liang J, Huang Y, Han M. Protective and restorative effects of the traditional Chinese medicine Jitai tablet against methamphetamine-induced dopaminergic neurotoxicity. BMC COMPLEMENTARY AND ALTERNATIVE MEDICINE 2018; 18:76. [PMID: 29475448 PMCID: PMC6389157 DOI: 10.1186/s12906-018-2094-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/18/2017] [Accepted: 01/16/2018] [Indexed: 01/09/2023]
Abstract
Background Methamphetamine (METH) is a psychostimulant with high abuse liability that affects the monoamine neurotransmitter systems, particularly the dopamine system. Currently there are no effective medications for the treatment of METH abuse to restore METH-induced dopaminergic dysfunction. The Jitai tablet (JTT), a commercial traditional Chinese medicinal preparation, has been shown to modulate the dopaminergic function both in heroin addicts and in morphine-dependent rats. The purpose of this study was to investigate, in a rodent model, whether JTT can protect against METH-induced neurotoxicity, and/or restore METH-damaged dopaminergic function. Methods Immunohistochemical staining and/or autoradiography staining were used to detect tyrosine hydroxylase (TH) expression in the substantia nigra, and to examine the levels of dopamine transporter (DAT), dopamine D2 receptor (D2R) and TH levels in the striatum. Using a stereotyped behavior rating scale, we evaluated the inhibitory effect of JTT on METH-induced behavioral sensitization. Results Repeated METH administration induced obvious stereotyped behavior and neurotoxicity on the dopaminergic system. Pre-treatment with JTT significantly attenuated METH-induced stereotyped responses, and interdicted METH-induced changes in the levels of DAT, D2R and TH expression. Treatment with JTT after METH administration restored DAT, D2R and TH expression to normal levels. Conclusions Our results indicated that JTT protects against METH-induced neurotoxicity and restores the dopaminergic function, and thus might be a potential treatment for the dopaminergic deficits associated with METH abuse.
Collapse
|
7
|
Fultz EK, Martin DL, Hudson CN, Kippin TE, Szumlinski KK. Methamphetamine-alcohol interactions in murine models of sequential and simultaneous oral drug-taking. Drug Alcohol Depend 2017; 177:178-186. [PMID: 28601731 PMCID: PMC6445265 DOI: 10.1016/j.drugalcdep.2017.03.026] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/07/2016] [Revised: 03/10/2017] [Accepted: 03/13/2017] [Indexed: 11/19/2022]
Abstract
BACKGROUND A high degree of co-morbidity exists between methamphetamine (MA) addiction and alcohol use disorders and both sequential and simultaneous MA-alcohol mixing increases risk for co-abuse. As little preclinical work has focused on the biobehavioral interactions between MA and alcohol within the context of drug-taking behavior, we employed simple murine models of voluntary oral drug consumption to examine how prior histories of either MA- or alcohol-taking influence the intake of the other drug. METHODS In one study, mice with a 10-day history of binge alcohol-drinking [5,10, 20 and 40% (v/v); 2h/day] were trained to self-administer oral MA in an operant-conditioning paradigm (10-40mg/L). In a second study, mice with a 10-day history of limited-access oral MA-drinking (5, 10, 20 and 40mg/L; 2h/day) were presented with alcohol (5-40% v/v; 2h/day) and then a choice between solutions of 20% alcohol, 10mg/L MA or their mix. RESULTS Under operant-conditioning procedures, alcohol-drinking mice exhibited less MA reinforcement overall, than water controls. However, when drug availability was not behaviorally-contingent, alcohol-drinking mice consumed more MA and exhibited greater preference for the 10mg/L MA solution than drug-naïve and combination drug-experienced mice. Conversely, prior MA-drinking history increased alcohol intake across a range of alcohol concentrations. DISCUSSION These exploratory studies indicate the feasibility of employing procedurally simple murine models of sequential and simultaneous oral MA-alcohol mixing of relevance to advancing our biobehavioral understanding of MA-alcohol co-abuse.
Collapse
Affiliation(s)
- Elissa K Fultz
- Department of Psychological and Brain Sciences, University of California Santa Barbara, Santa Barbara, CA, 93106-9660, USA
| | - Douglas L Martin
- Department of Psychological and Brain Sciences, University of California Santa Barbara, Santa Barbara, CA, 93106-9660, USA
| | - Courtney N Hudson
- Department of Psychological and Brain Sciences, University of California Santa Barbara, Santa Barbara, CA, 93106-9660, USA
| | - Tod E Kippin
- Department of Psychological and Brain Sciences, University of California Santa Barbara, Santa Barbara, CA, 93106-9660, USA; Department of Molecular, Cellular and Developmental Biology and the Neuroscience Research Institute, University of California Santa Barbara, Santa Barbara, CA, 93106-9660, USA; Institute for Collaborative Biotechnology, University of California Santa Barbara, Santa Barbara, CA, 93106-9660, USA
| | - Karen K Szumlinski
- Department of Psychological and Brain Sciences, University of California Santa Barbara, Santa Barbara, CA, 93106-9660, USA; Department of Molecular, Cellular and Developmental Biology and the Neuroscience Research Institute, University of California Santa Barbara, Santa Barbara, CA, 93106-9660, USA.
| |
Collapse
|
8
|
Methamphetamine reduces expression of caveolin-1 in the dorsal striatum: Implication for dysregulation of neuronal function. Neuroscience 2016; 328:147-56. [PMID: 27138644 DOI: 10.1016/j.neuroscience.2016.04.039] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2016] [Revised: 03/30/2016] [Accepted: 04/25/2016] [Indexed: 12/15/2022]
Abstract
Role of striatal dopamine D1 receptors (D1Rs) in methamphetamine (Meth) taking and seeking is recognized from contingent Meth self-administration studies. For example, Meth increases levels of D1Rs in the dorsal striatum in animal models of Meth addiction, and blockade of striatal D1Rs decreased responding for Meth and reduced Meth priming-induced drug seeking. However, the mechanism underlying enhanced expression of striatal D1Rs in animals self-administering Meth is unknown and is hypothesized to involve maladaptive intracellular signal transduction mechanism via hyperphosphorylation of extracellular signal-regulated kinase 1/2 (ERK1/2). D1Rs are predominantly localized to detergent-resistant membrane/lipid raft fractions (MLR fraction), and in vitro studies indicate that D1R signaling and recycling is regulated by the MLR-resident protein caveolin-1 (Cav-1), in an endocytotic-dependent manner. Notably, expression of Cav-1 is inversely regulated by ERK1/2 activation, suggesting a signaling interplay among D1Rs, ERK1/2 and Cav-1. We therefore evaluated the effects of extended access Meth self-administration on expression of striatal D1Rs, activated ERK1/2 and Cav-1. We first report that Cav-1 is heavily expressed in neurons located in the dorsal striatum. We also report that extended access Meth produces compulsive-like unregulated intake of the drug, and these behavioral outcomes are associated with enhanced expression of D1Rs, increased activity of ERK1/2, and reduced Cav-1 expression in the dorsal striatum. These data suggest a possible cellular mechanism that involves Cav-1 regulation of D1R expression in response to escalated Meth intake, and how this response of altered D1Rs and enhanced ERK1/2 activation to Meth self-administration contributes to contingent-related processes such as addiction.
Collapse
|
9
|
Rodent models of impulsive–compulsive behaviors in Parkinson's disease: How far have we reached? Neurobiol Dis 2015; 82:561-573. [DOI: 10.1016/j.nbd.2015.08.026] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2015] [Revised: 08/22/2015] [Accepted: 08/25/2015] [Indexed: 01/26/2023] Open
|
10
|
Laćan G, Hadamitzky M, Kuczenski R, Melega WP. Alterations in the striatal dopamine system during intravenous methamphetamine exposure: effects of contingent and noncontingent administration. Synapse 2013; 67:476-88. [PMID: 23417852 DOI: 10.1002/syn.21654] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2012] [Accepted: 02/12/2013] [Indexed: 11/09/2022]
Abstract
The continuing spread of methamphetamine (METH) abuse has stimulated research aimed at understanding consequences of its prolonged exposure. Alterations in nigrostriatal dopamine (DA) system parameters have been characterized in experimental studies after discontinuation of long-term METH but fewer studies have included similar assessments during METH exposure. Here, we report METH plasma pharmacokinetics and striatal DA system alterations in rat after noncontingent and contingent METH administration for 7.5 weeks. Escalating METH exposure was delivered by dynamic infusion (DI) that incorporated a "humanized" plasma METH half life or by intravenous self-administration (IVSA) that included binge intakes. Kinetic modeling of DI and IVSA for 24 h periods during the final week of METH exposure showed that plasma METH levels remained between 0.7 and 1.5 µM. Animals were sacrificed during their last METH administration for autoradiography assessment using [³H]ligands and D2 agonist-induced [³⁵S]GTPγS binding. DA transporter binding was decreased (DI, 34%; IVSA, 15%) while vesicular monoamine transporter binding and substantia nigra DA cell numbers were unchanged. Decreases were measured for D2 receptor (DI and IVSA, 15-20%) and [³⁵S]GTPγS binding (DI, 35%; IVSA, 18%). These similar patterns of DI and IVSA associated decreases in striatal DA markers reflect consequences of cumulative METH exposure and not the drug delivery method. For METH IVSA, individual differences were observed, yet each animal's total intake was similar within and across three 24-h binges. IVSA rodent models may be useful for identifying molecular mechanisms that are associated with METH binges in humans.
Collapse
Affiliation(s)
- Goran Laćan
- Department of Molecular and Medical Pharmacology, David Geffen School of Medicine at UCLA, Los Angeles, California, USA
| | | | | | | |
Collapse
|