1
|
Bouton ME, Maren S, McNally GP. BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING. Physiol Rev 2021; 101:611-681. [PMID: 32970967 PMCID: PMC8428921 DOI: 10.1152/physrev.00016.2020] [Citation(s) in RCA: 165] [Impact Index Per Article: 55.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022] Open
Abstract
This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.
Collapse
Affiliation(s)
- Mark E Bouton
- Department of Psychological Science, University of Vermont, Burlington, Vermont
| | - Stephen Maren
- Department of Psychological and Brain Sciences and Institute for Neuroscience, Texas A&M University, College Station, Texas
| | - Gavan P McNally
- School of Psychology, University of New South Wales, Sydney, Australia
| |
Collapse
|
2
|
Groman SM, Massi B, Mathias SR, Lee D, Taylor JR. Model-Free and Model-Based Influences in Addiction-Related Behaviors. Biol Psychiatry 2019; 85:936-945. [PMID: 30737015 PMCID: PMC6534429 DOI: 10.1016/j.biopsych.2018.12.017] [Citation(s) in RCA: 47] [Impact Index Per Article: 9.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/04/2018] [Revised: 12/19/2018] [Accepted: 12/20/2018] [Indexed: 01/09/2023]
Abstract
BACKGROUND Disruptions in the decision-making processes that guide action selection are a core feature of many psychiatric disorders, including addiction. Decision making is influenced by the goal-directed and habitual systems that can be computationally characterized using model-based and model-free reinforcement learning algorithms, respectively. Recent evidence suggests an imbalance in the influence of these reinforcement learning systems on behavior in individuals with substance dependence, but it is unknown whether these disruptions are a manifestation of chronic drug use and/or are a preexisting risk factor for addiction. METHODS We trained adult male rats on a multistage decision-making task to quantify model-free and model-based processes before and after self-administration of methamphetamine or saline. RESULTS Individual differences in model-free, but not model-based, learning prior to any drug use predicted subsequent methamphetamine self-administration; rats with lower model-free behavior took more methamphetamine than rats with higher model-free behavior. This relationship was selective to model-free updating following a rewarded, but not unrewarded, choice. Both model-free and model-based learning were reduced in rats following methamphetamine self-administration, which was due to a decrement in the ability of rats to use unrewarded outcomes appropriately. Moreover, the magnitude of drug-induced disruptions in model-free learning was not correlated with disruptions in model-based behavior, indicating that drug self-administration independently altered both reinforcement learning strategies. CONCLUSIONS These findings provide direct evidence that model-free and model-based learning mechanisms are involved in select aspects of addiction vulnerability and pathology, and they provide a unique behavioral platform for conducting systems-level analyses of decision making in preclinical models of mental illness.
Collapse
Affiliation(s)
- Stephanie M. Groman
- Department of Psychiatry, Yale University,Correspondence to be direct to: Stephanie M. Groman, Ph.D. (); Jane R. Taylor, Ph.D. (), 34 Park Street, New Haven CT 06515
| | - Bart Massi
- Department of Neuroscience, Yale University
| | | | - Daeyeol Lee
- Department of Psychiatry, Yale University,Department of Neuroscience, Yale University,Department of Psychology, Yale University
| | - Jane R. Taylor
- Department of Psychiatry, Yale University,Department of Psychology, Yale University,Correspondence to be direct to: Stephanie M. Groman, Ph.D. (); Jane R. Taylor, Ph.D. (), 34 Park Street, New Haven CT 06515
| |
Collapse
|
3
|
van Holst RJ, Sescousse G, Janssen LK, Janssen M, Berry AS, Jagust WJ, Cools R. Increased Striatal Dopamine Synthesis Capacity in Gambling Addiction. Biol Psychiatry 2018; 83:1036-1043. [PMID: 28728675 PMCID: PMC6698370 DOI: 10.1016/j.biopsych.2017.06.010] [Citation(s) in RCA: 44] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/20/2017] [Revised: 05/15/2017] [Accepted: 06/01/2017] [Indexed: 02/07/2023]
Abstract
BACKGROUND The hypothesis that dopamine plays an important role in the pathophysiology of pathological gambling is pervasive. However, there is little to no direct evidence for a categorical difference between pathological gamblers and healthy control subjects in terms of dopamine transmission in a drug-free state. Here we provide evidence for this hypothesis by comparing dopamine synthesis capacity in the dorsal and ventral parts of the striatum in 13 pathological gamblers and 15 healthy control subjects. METHODS This was achieved using [18F]fluoro-levo-dihydroxyphenylalanine dynamic positron emission tomography scans and striatal regions of interest that were hand-drawn based on visual inspection of individual structural magnetic resonance imaging scans. RESULTS Our results show that dopamine synthesis capacity was increased in pathological gamblers compared with healthy control subjects. Dopamine synthesis was 16% higher in the caudate body, 17% higher in the dorsal putamen, and 17% higher in the ventral striatum in pathological gamblers compared with control subjects. Moreover, dopamine synthesis capacity in the dorsal putamen and caudate head was positively correlated with gambling distortions in pathological gamblers. CONCLUSIONS Taken together, these results provide empirical evidence for increased striatal dopamine synthesis in pathological gambling.
Collapse
|
4
|
Egervari G, Ciccocioppo R, Jentsch JD, Hurd YL. Shaping vulnerability to addiction - the contribution of behavior, neural circuits and molecular mechanisms. Neurosci Biobehav Rev 2018; 85:117-125. [PMID: 28571877 PMCID: PMC5708151 DOI: 10.1016/j.neubiorev.2017.05.019] [Citation(s) in RCA: 46] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2017] [Revised: 05/11/2017] [Accepted: 05/22/2017] [Indexed: 12/11/2022]
Abstract
Substance use disorders continue to impose increasing medical, financial and emotional burdens on society in the form of morbidity and overdose, family disintegration, loss of employment and crime, while advances in prevention and treatment options remain limited. Importantly, not all individuals exposed to abused substances effectively develop the disease. Genetic factors play a significant role in determining addiction vulnerability and interactions between innate predisposition, environmental factors and personal experiences are also critical. Thus, understanding individual differences that contribute to the initiation of substance use as well as on long-term maladaptations driving compulsive drug use and relapse propensity is of critical importance to reduce this devastating disorder. In this paper, we discuss current topics in the field of addiction regarding individual vulnerability related to behavioral endophenotypes, neural circuits, as well as genetics and epigenetic mechanisms. Expanded knowledge of these factors is of importance to improve and personalize prevention and treatment interventions in the future.
Collapse
Affiliation(s)
- Gabor Egervari
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, 10029 New York, NY, USA; Department of Neuroscience, Icahn School of Medicine at Mount Sinai, 10029 New York, NY, USA
| | - Roberto Ciccocioppo
- School of Pharmacy, Pharmacology Unit, University of Camerino, 62032 Camerino, Italy
| | - J David Jentsch
- Department of Psychology, Binghamton University, 13902 Binghamton, NY, USA
| | - Yasmin L Hurd
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, 10029 New York, NY, USA; Department of Neuroscience, Icahn School of Medicine at Mount Sinai, 10029 New York, NY, USA.
| |
Collapse
|
5
|
Large-Scale Persistent Network Reconfiguration Induced by Ketamine in Anesthetized Monkeys: Relevance to Mood Disorders. Biol Psychiatry 2016; 79:765-775. [PMID: 25837427 DOI: 10.1016/j.biopsych.2015.02.028] [Citation(s) in RCA: 54] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/25/2014] [Revised: 01/13/2015] [Accepted: 02/15/2015] [Indexed: 12/19/2022]
Abstract
BACKGROUND Ketamine is a highly attractive candidate for developing fast-onset antidepressant agents; however, the relevant brain circuits that underlie sustained, efficacious antidepressant effects remain largely unknown. METHODS We used a holistic scheme combining whole-brain resting-state fMRI and graph theoretical analysis to examine the sustained effects on brain networks after administration of a single dose of ketamine and to identify the brain regions and circuits preferentially targeted by ketamine. Topological differences in functional networks of anesthetized macaque monkeys were compared between ketamine (.5 mg/kg) and saline treatment after 18 hours. RESULTS We observed persistent global reconfiguration of small-world properties in response to ketamine intake, accompanied by large-scale downregulation of functional connectivity, most prominently in the orbital prefrontal cortex, the subgenual and posterior cingulate cortices, and the nucleus accumbens. Intriguingly, intrinsic connectivity with the medial prefrontal areas in the reward circuits were selectively downregulated. Global and regional regulations of the brain networks precisely opposed the maladaptive alterations in the depressed brain. CONCLUSIONS Our results demonstrated that local synaptic plasticity triggered by blockade of N-methyl-D-aspartic acid receptors was capable of translating into prolonged network reconfiguration in the distributed cortico-limbic-striatal circuit, providing mechanistic insight into developing specific loci or circuit-targeted, long-term therapeutics.
Collapse
|
6
|
Gullo MJ, Potenza MN. Impulsivity: mechanisms, moderators and implications for addictive behaviors. Addict Behav 2014; 39:1543-1546. [PMID: 25042111 DOI: 10.1016/j.addbeh.2014.06.005] [Citation(s) in RCA: 45] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2014] [Accepted: 06/12/2014] [Indexed: 12/21/2022]
Affiliation(s)
- Matthew J Gullo
- Centre for Youth Substance Abuse Research, The University of Queensland, Australia.
| | - Marc N Potenza
- Department of Psychiatry, Yale University School of Medicine, New Haven, CT, United States; Department of Neurobiology, Yale University School of Medicine, New Haven, CT, United States; Child Study Center, Yale University School of Medicine, New Haven, CT, United States
| |
Collapse
|
7
|
Lominac KD, McKenna CL, Schwartz LM, Ruiz PN, Wroten MG, Miller BW, Holloway JJ, Travis KO, Rajasekar G, Maliniak D, Thompson AB, Urman LE, Phillips TJ, Szumlinski KK. Mesocorticolimbic monoamine correlates of methamphetamine sensitization and motivation. Front Syst Neurosci 2014; 8:70. [PMID: 24847220 PMCID: PMC4019853 DOI: 10.3389/fnsys.2014.00070] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2014] [Accepted: 04/10/2014] [Indexed: 11/16/2022] Open
Abstract
Methamphetamine (MA) is a highly addictive psychomotor stimulant, with life-time prevalence rates of abuse ranging from 5–10% world-wide. Yet, a paucity of research exists regarding MA addiction vulnerability/resiliency and neurobiological mediators of the transition to addiction that might occur upon repeated low-dose MA exposure, more characteristic of early drug use. As stimulant-elicited neuroplasticity within dopamine neurons innervating the nucleus accumbens (NAC) and prefrontal cortex (PFC) is theorized as central for addiction-related behavioral anomalies, we used a multi-disciplinary research approach in mice to examine the interactions between sub-toxic MA dosing, motivation for MA and mesocorticolimbic monoamines. Biochemical studies of C57BL/6J (B6) mice revealed short- (1 day), as well as longer-term (21 days), changes in extracellular dopamine, DAT and/or D2 receptors during withdrawal from 10, once daily, 2 mg/kg MA injections. Follow-up biochemical studies conducted in mice selectively bred for high vs. low MA drinking (respectively, MAHDR vs. MALDR mice), provided novel support for anomalies in mesocorticolimbic dopamine as a correlate of genetic vulnerability to high MA intake. Finally, neuropharmacological targeting of NAC dopamine in MA-treated B6 mice demonstrated a bi-directional regulation of MA-induced place-conditioning. These results extend extant literature for MA neurotoxicity by demonstrating that even subchronic exposure to relatively low MA doses are sufficient to elicit relatively long-lasting changes in mesocorticolimbic dopamine and that drug-induced or idiopathic anomalies in mesocorticolimbic dopamine may underpin vulnerability/resiliency to MA addiction.
Collapse
Affiliation(s)
- Kevin D Lominac
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| | - Courtney L McKenna
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| | - Lisa M Schwartz
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| | - Paige N Ruiz
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| | - Melissa G Wroten
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| | - Bailey W Miller
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| | - John J Holloway
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| | - Katherine O Travis
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| | - Ganesh Rajasekar
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| | - Dan Maliniak
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| | - Andrew B Thompson
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| | - Lawrence E Urman
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| | - Tamara J Phillips
- Behavioral Neuroscience, Methamphetamine Abuse Research Center, Veterans Affairs Medical Center, Oregon Health and Science University Portland, OR, USA
| | - Karen K Szumlinski
- Department of Psychological and Brain Sciences, Neuroscience Research Institute, University of California at Santa Barbara Santa Barbara, CA, USA
| |
Collapse
|
8
|
Jentsch JD, Ashenhurst JR, Cervantes MC, Groman SM, James AS, Pennington ZT. Dissecting impulsivity and its relationships to drug addictions. Ann N Y Acad Sci 2014; 1327:1-26. [PMID: 24654857 DOI: 10.1111/nyas.12388] [Citation(s) in RCA: 145] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
Addictions are often characterized as forms of impulsive behavior. That said, it is often noted that impulsivity is a multidimensional construct, spanning several psychological domains. This review describes the relationship between varieties of impulsivity and addiction-related behaviors, the nature of the causal relationship between the two, and the underlying neurobiological mechanisms that promote impulsive behaviors. We conclude that the available data strongly support the notion that impulsivity is both a risk factor for, and a consequence of, drug and alcohol consumption. While the evidence indicating that subtypes of impulsive behavior are uniquely informative--either biologically or with respect to their relationships to addictions--is convincing, multiple lines of study link distinct subtypes of impulsivity to low dopamine D2 receptor function and perturbed serotonergic transmission, revealing shared mechanisms between the subtypes. Therefore, a common biological framework involving monoaminergic transmitters in key frontostriatal circuits may link multiple forms of impulsivity to drug self-administration and addiction-related behaviors. Further dissection of these relationships is needed before the next phase of genetic and genomic discovery will be able to reveal the biological sources of the vulnerability for addiction indexed by impulsivity.
Collapse
Affiliation(s)
- J David Jentsch
- Department of Psychology, University of California Los Angeles, Los Angeles, California
| | | | | | | | | | | |
Collapse
|
9
|
Groman SM, Morales AM, Lee B, London ED, Jentsch JD. Methamphetamine-induced increases in putamen gray matter associate with inhibitory control. Psychopharmacology (Berl) 2013; 229:527-38. [PMID: 23748383 PMCID: PMC3770792 DOI: 10.1007/s00213-013-3159-9] [Citation(s) in RCA: 43] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/11/2013] [Accepted: 05/18/2013] [Indexed: 10/26/2022]
Abstract
RATIONALE Problematic drug use is associated with difficulty in exerting self-control over behaviors, and this difficulty may be a consequence of atypical morphometric characteristics that are exhibited by drug-experienced individuals. The extent to which these structural abnormalities result from drug use or reflect neurobiological risk factors that predate drug use, however, is unknown. OBJECTIVES The purpose of this study is to determine how methamphetamine affects corticostriatal structure and how drug-induced changes relate to alterations in inhibitory control. METHODS Structural magnetic resonance images and positron emission tomography (PET) scans, assessing dopamine D₂-like receptor and transporter availability, were acquired in monkeys trained to acquire, retain, and reverse three-choice visual discrimination problems before and after exposure to an escalating dose regimen of methamphetamine (or saline, as a control). Voxel-based morphometry was used to compare changes in corticostriatal gray matter between methamphetamine- and saline-exposed monkeys. The change in gray matter before and after the dosing regimen was compared to the change in the behavioral performance and in dopaminergic markers measured with PET. RESULTS Methamphetamine exposure, compared to saline, increased gray matter within the right putamen. These changes were positively correlated with changes in performance of methamphetamine-exposed monkeys in the reversal phase, and were negatively correlated with alterations in D₂-like receptor and DAT availability. CONCLUSIONS The results provide the first evidence that exposure to a methamphetamine dosing regimen that resembles human use alters the structural integrity of the striatum and that gray-matter abnormalities detected in human methamphetamine users are due, at least in part, to the pharmacological effects of drug experience.
Collapse
Affiliation(s)
| | - Angelica M. Morales
- Department of Psychiatry & Bio-behavioral Sciences, University of California, Los Angeles
| | - Buyean Lee
- Department of Psychiatry & Bio-behavioral Sciences, University of California, Los Angeles
| | - Edythe D. London
- Department of Psychiatry & Bio-behavioral Sciences, University of California, Los Angeles,Department of Medical and Molecular Pharmacology, University of California, Los Angeles
| | - James David Jentsch
- Department of Psychology, University of California, Los Angeles,Department of Psychiatry & Bio-behavioral Sciences, University of California, Los Angeles,Correspondence should be sent to: J. David Jentsch () UCLA Department of Psychology PO Box 951563 Los Angeles, CA 90095-1563
| |
Collapse
|