1
|
Kwon M, Choi H, Park H, Ahn WY, Jung YC. Neural correlates of model-based behavior in internet gaming disorder and alcohol use disorder. J Behav Addict 2024; 13:236-249. [PMID: 38460004 PMCID: PMC10988400 DOI: 10.1556/2006.2024.00006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/10/2023] [Revised: 12/26/2023] [Accepted: 02/08/2024] [Indexed: 03/11/2024] Open
Abstract
Background An imbalance between model-based and model-free decision-making systems is a common feature in addictive disorders. However, little is known about whether similar decision-making deficits appear in internet gaming disorder (IGD). This study compared neurocognitive features associated with model-based and model-free systems in IGD and alcohol use disorder (AUD). Method Participants diagnosed with IGD (n = 22) and AUD (n = 22), and healthy controls (n = 30) performed the two-stage task inside the functional magnetic resonance imaging (fMRI) scanner. We used computational modeling and hierarchical Bayesian analysis to provide a mechanistic account of their choice behavior. Then, we performed a model-based fMRI analysis and functional connectivity analysis to identify neural correlates of the decision-making processes in each group. Results The computational modeling results showed similar levels of model-based behavior in the IGD and AUD groups. However, we observed distinct neural correlates of the model-based reward prediction error (RPE) between the two groups. The IGD group exhibited insula-specific activation associated with model-based RPE, while the AUD group showed prefrontal activation, particularly in the orbitofrontal cortex and superior frontal gyrus. Furthermore, individuals with IGD demonstrated hyper-connectivity between the insula and brain regions in the salience network in the context of model-based RPE. Discussion and Conclusions The findings suggest potential differences in the neurobiological mechanisms underlying model-based behavior in IGD and AUD, albeit shared cognitive features observed in computational modeling analysis. As the first neuroimaging study to compare IGD and AUD in terms of the model-based system, this study provides novel insights into distinct decision-making processes in IGD.
Collapse
Affiliation(s)
- Mina Kwon
- Department of Psychology, Seoul National University, Seoul, South Korea
| | - Hangnyoung Choi
- Department of Psychiatry, Yonsei University College of Medicine, Seoul, South Korea
- Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, South Korea
| | - Harhim Park
- Department of Psychology, Seoul National University, Seoul, South Korea
| | - Woo-Young Ahn
- Department of Psychology, Seoul National University, Seoul, South Korea
- Department of Brain and Cognitive Sciences, Seoul National University, Seoul, South Korea
- AI Institute, Seoul National University, Seoul, South Korea
| | - Young-Chul Jung
- Department of Psychiatry, Yonsei University College of Medicine, Seoul, South Korea
- Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, South Korea
- Institute for Innovation in Digital Healthcare, Yonsei University, Seoul, South Korea
| |
Collapse
|
2
|
Gutierrez-Castellanos N, Sarra D, Godinho BS, Mainen ZF. Maturation of cortical input to dorsal raphe nucleus increases behavioral persistence in mice. eLife 2024; 13:e93485. [PMID: 38477558 PMCID: PMC10994666 DOI: 10.7554/elife.93485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Accepted: 02/28/2024] [Indexed: 03/14/2024] Open
Abstract
The ability to persist toward a desired objective is a fundamental aspect of behavioral control whose impairment is implicated in several behavioral disorders. One of the prominent features of behavioral persistence is that its maturation occurs relatively late in development. This is presumed to echo the developmental time course of a corresponding circuit within late-maturing parts of the brain, such as the prefrontal cortex, but the specific identity of the responsible circuits is unknown. Here, we used a genetic approach to describe the maturation of the projection from layer 5 neurons of the neocortex to the dorsal raphe nucleus in mice. Using optogenetic-assisted circuit mapping, we show that this projection undergoes a dramatic increase in synaptic potency between postnatal weeks 3 and 8, corresponding to the transition from juvenile to adult. We then show that this period corresponds to an increase in the behavioral persistence that mice exhibit in a foraging task. Finally, we used a genetic targeting strategy that primarily affected neurons in the medial prefrontal cortex, to selectively ablate this pathway in adulthood and show that mice revert to a behavioral phenotype similar to juveniles. These results suggest that frontal cortical to dorsal raphe input is a critical anatomical and functional substrate of the development and manifestation of behavioral persistence.
Collapse
Affiliation(s)
| | - Dario Sarra
- Champalimaud Research, Champalimaud FoundationLisbonPortugal
- Nuffield Department of Clinical Neurosciences, University of OxfordOxfordUnited Kingdom
| | - Beatriz S Godinho
- Champalimaud Research, Champalimaud FoundationLisbonPortugal
- Nuffield Department of Clinical Neurosciences, University of OxfordOxfordUnited Kingdom
| | | |
Collapse
|
3
|
Scholz V, Waltmann M, Herzog N, Reiter A, Horstmann A, Deserno L. Cortical Grey Matter Mediates Increases in Model-Based Control and Learning from Positive Feedback from Adolescence to Adulthood. J Neurosci 2023; 43:2178-2189. [PMID: 36823039 PMCID: PMC10039741 DOI: 10.1523/jneurosci.1418-22.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Revised: 12/20/2022] [Accepted: 01/13/2023] [Indexed: 02/25/2023] Open
Abstract
Cognition and brain structure undergo significant maturation from adolescence into adulthood. Model-based (MB) control is known to increase across development, which is mediated by cognitive abilities. Here, we asked two questions unaddressed in previous developmental studies. First, what are the brain structural correlates of age-related increases in MB control? Second, how are age-related increases in MB control from adolescence to adulthood influenced by motivational context? A human developmental sample (n = 103; age, 12-50, male/female, 55:48) completed structural MRI and an established task to capture MB control. The task was modified with respect to outcome valence by including (1) reward and punishment blocks to manipulate the motivational context and (2) an additional choice test to assess learning from positive versus negative feedback. After replicating that an age-dependent increase in MB control is mediated by cognitive abilities, we demonstrate first-time evidence that gray matter density (GMD) in the parietal cortex mediates the increase of MB control with age. Although motivational context did not relate to age-related changes in MB control, learning from positive feedback improved with age. Meanwhile, negative feedback learning showed no age effects. We present a first report that an age-related increase in positive feedback learning was mediated by reduced GMD in the parietal, medial, and dorsolateral prefrontal cortex. Our findings indicate that brain maturation, putatively reflected in lower GMD, in distinct and partially overlapping brain regions could lead to a more efficient brain organization and might thus be a key developmental step toward age-related increases in planning and value-based choice.SIGNIFICANCE STATEMENT Changes in model-based decision-making are paralleled by extensive maturation in cognition and brain structure across development. Still, to date the neuroanatomical underpinnings of these changes remain unclear. Here, we demonstrate for the first time that parietal GMD mediates age-dependent increases in model-based control. Age-related increases in positive feedback learning were mediated by reduced GMD in the parietal, medial, and dorsolateral prefrontal cortex. A manipulation of motivational context did not have an impact on age-related changes in model-based control. These findings highlight that brain maturation in distinct and overlapping cortical regions constitutes a key developmental step toward improved value-based choices.
Collapse
Affiliation(s)
- Vanessa Scholz
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, 97080 Würzburg, Germany
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, 6525 GD Nijmegen, The Netherlands
| | - Maria Waltmann
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, 97080 Würzburg, Germany
- Max Planck Institute for Cognition and Neuroscience, D-04103 Leipzig, Germany
| | - Nadine Herzog
- Max Planck Institute for Cognition and Neuroscience, D-04103 Leipzig, Germany
- Integrated Research and Treatment Center AdiposityDiseases, Leipzig University Medical Center, 04103 Leipzig, Germany
| | - Andrea Reiter
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, 97080 Würzburg, Germany
- Collaborative Research Center-940 Volition and Cognitive Control, Faculty of Psychology, Technical University Dresden, 01069 Dresden, Germany
| | - Annette Horstmann
- Max Planck Institute for Cognition and Neuroscience, D-04103 Leipzig, Germany
- Integrated Research and Treatment Center AdiposityDiseases, Leipzig University Medical Center, 04103 Leipzig, Germany
- Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, 00014 Helsinki, Finland
| | - Lorenz Deserno
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, 97080 Würzburg, Germany
- Max Planck Institute for Cognition and Neuroscience, D-04103 Leipzig, Germany
- Integrated Research and Treatment Center AdiposityDiseases, Leipzig University Medical Center, 04103 Leipzig, Germany
- Department of Psychiatry and Psychotherapy, University Hospital Carl Gustav Carus, Technical University Dresden, 01069 Dresden, Germany
| |
Collapse
|
4
|
Fronto-striatal structures related with model-based control as an endophenotype for obsessive-compulsive disorder. Sci Rep 2021; 11:11951. [PMID: 34099768 PMCID: PMC8185095 DOI: 10.1038/s41598-021-91179-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2020] [Accepted: 02/08/2021] [Indexed: 11/18/2022] Open
Abstract
Recent theories suggest a shift from model-based goal-directed to model-free habitual decision-making in obsessive–compulsive disorder (OCD). However, it is yet unclear, whether this shift in the decision process is heritable. We investigated 32 patients with OCD, 27 unaffected siblings (SIBs) and 31 healthy controls (HCs) using the two-step task. We computed behavioral and reaction time analyses and fitted a computational model to assess the balance between model-based and model-free control. 80 subjects also underwent structural imaging. We observed a significant ordered effect for the shift towards model-free control in the direction OCD > SIB > HC in our computational parameter of interest. However less directed analyses revealed no shift towards model-free control in OCDs. Nonetheless, we found evidence for reduced model-based control in OCDs compared to HCs and SIBs via 2nd stage reaction time analyses. In this measure SIBs also showed higher levels of model-based control than HCs. Across all subjects these effects were associated with the surface area of the left medial/right dorsolateral prefrontal cortex. Moreover, correlations between bilateral putamen/right caudate volumes and these effects varied as a function of group: they were negative in SIBs and OCDs, but positive in HCs. Associations between fronto-striatal regions and model-based reaction time effects point to a potential endophenotype for OCD.
Collapse
|
5
|
Chen H, Mojtahedzadeh N, Belanger MJ, Nebe S, Kuitunen-Paul S, Sebold M, Garbusow M, Huys QJM, Heinz A, Rapp MA, Smolka MN. Model-Based and Model-Free Control Predicts Alcohol Consumption Developmental Trajectory in Young Adults: A 3-Year Prospective Study. Biol Psychiatry 2021; 89:980-989. [PMID: 33771349 DOI: 10.1016/j.biopsych.2021.01.009] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/20/2020] [Revised: 12/21/2020] [Accepted: 01/17/2021] [Indexed: 11/15/2022]
Abstract
BACKGROUND A shift from goal-directed toward habitual control has been associated with alcohol dependence. Whether such a shift predisposes to risky drinking is not yet clear. We investigated how goal-directed and habitual control at age 18 predict alcohol use trajectories over the course of 3 years. METHODS Goal-directed and habitual control, as informed by model-based (MB) and model-free (MF) learning, were assessed with a two-step sequential decision-making task during functional magnetic resonance imaging in 146 healthy 18-year-old men. Three-year alcohol use developmental trajectories were based on either a consumption score from the self-reported Alcohol Use Disorders Identification Test (assessed every 6 months) or an interview-based binge drinking score (grams of alcohol/occasion; assessed every year). We applied a latent growth curve model to examine how MB and MF control predicted the drinking trajectory. RESULTS Drinking behavior was best characterized by a linear trajectory. MB behavioral control was negatively associated with the development of the binge drinking score; MF reward prediction error blood oxygen level-dependent signals in the ventromedial prefrontal cortex and the ventral striatum predicted a higher starting point and steeper increase of the Alcohol Use Disorders Identification Test consumption score over time, respectively. CONCLUSIONS We found that MB behavioral control was associated with the binge drinking trajectory, while the MF reward prediction error signal was closely linked to the consumption score development. These findings support the idea that unbalanced MB and MF control might be an important individual vulnerability in predisposing to risky drinking behavior.
Collapse
Affiliation(s)
- Hao Chen
- Department of Psychiatry, Technische Universität Dresden, Dresden, Germany; Neuroimaging Center, Technische Universität Dresden, Dresden, Germany
| | - Negin Mojtahedzadeh
- Department of Psychiatry, Technische Universität Dresden, Dresden, Germany; Neuroimaging Center, Technische Universität Dresden, Dresden, Germany
| | - Matthew J Belanger
- Department of Psychiatry, Technische Universität Dresden, Dresden, Germany; Neuroimaging Center, Technische Universität Dresden, Dresden, Germany
| | - Stephan Nebe
- Department of Psychiatry, Technische Universität Dresden, Dresden, Germany; Neuroimaging Center, Technische Universität Dresden, Dresden, Germany; Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zurich, Switzerland
| | - Sören Kuitunen-Paul
- Institute of Clinical Psychology and Psychotherapy, Technische Universität Dresden, Dresden, Germany; Department of Child and Adolescent Psychiatry, Technische Universität Dresden, Dresden, Germany
| | - Miriam Sebold
- Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Maria Garbusow
- Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Quentin J M Huys
- Division of Psychiatry, University College London, London, United Kingdom; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, United Kingdom
| | - Andreas Heinz
- Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Michael A Rapp
- Area of Excellence Cognitive Sciences, University of Potsdam, Potsdam, Germany
| | - Michael N Smolka
- Department of Psychiatry, Technische Universität Dresden, Dresden, Germany; Neuroimaging Center, Technische Universität Dresden, Dresden, Germany.
| |
Collapse
|
6
|
Kuitunen-Paul S, Roessner V, Basedow LA, Golub Y. Beyond the tip of the iceberg: A narrative review to identify research gaps on comorbid psychiatric disorders in adolescents with methamphetamine use disorder or chronic methamphetamine use. Subst Abus 2020; 42:13-32. [PMID: 32870121 DOI: 10.1080/08897077.2020.1806183] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
BACKGROUND Methamphetamine use disorder (MUD) frequently begins in adolescence, often accompanied by other psychiatric or mental disorders. Up to now, no comprehensive review about MUD and comorbid disorders in adolescents is available. We thus aimed to review the literature on comorbid mental disorders and MUD in adolescents in order to identify future research topics. Method: A PubMed search was conducted in July 2019. Relevant comorbidities were defined as attention-deficit disorder with/without hyperactivity, anxiety disorders, depression, eating disorders, post-traumatic stress disorder, psychosis, borderline personality disorder, conduct disorder and antisocial personality disorder, as well as other substance use disorders. For each comorbidity, we summarized prevalence rates, findings on comorbidity mechanisms, and recommended treatment options, if applicable. Results: Few articles focused on MUD in adolescents. Prevalence rates differed largely between comorbid disorders, with tobacco use disorder, conduct disorder, post-traumatic stress disorder, anxiety disorders, and attention-deficit disorders being the most prevalent comorbidities while eating disorders were rare. Examined onset patterns and comorbidity mechanisms indicated three groups of comorbidities: preexisting disorders self-medicated with methamphetamine, disorders induced by chronic methamphetamine use, and disorders arising due to risk factors shared with MUD. Reviewed comorbidities were frequently associated with worse treatment outcomes. Conclusions: The limited evidence is in stark contrast to the presumably high prevalence and relevance of comorbid mental disorders in adolescents with MUD. Suggestions for future research topics, informed by adult findings, include genetic vulnerabilities, biological changes, and consequences of different use patterns. Surprisingly few MUD treatment programs explicitly integrate comorbid mental disorder modules.
Collapse
Affiliation(s)
- Sören Kuitunen-Paul
- Research Group Stress and Addiction, Department of Child and Adolescent Psychiatry, Faculty of Medicine, Technische Universität Dresden, Dresden, Germany
| | - Veit Roessner
- Research Group Stress and Addiction, Department of Child and Adolescent Psychiatry, Faculty of Medicine, Technische Universität Dresden, Dresden, Germany
| | - Lukas A Basedow
- Research Group Stress and Addiction, Department of Child and Adolescent Psychiatry, Faculty of Medicine, Technische Universität Dresden, Dresden, Germany
| | - Yulia Golub
- Research Group Stress and Addiction, Department of Child and Adolescent Psychiatry, Faculty of Medicine, Technische Universität Dresden, Dresden, Germany
| |
Collapse
|
7
|
Berghäuser J, Bensmann W, Zink N, Endrass T, Beste C, Stock AK. Alcohol Hangover Does Not Alter the Application of Model-Based and Model-Free Learning Strategies. J Clin Med 2020; 9:jcm9051453. [PMID: 32414137 PMCID: PMC7290484 DOI: 10.3390/jcm9051453] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2020] [Revised: 05/06/2020] [Accepted: 05/07/2020] [Indexed: 12/14/2022] Open
Abstract
Frequent alcohol binges shift behavior from goal-directed to habitual processing modes. This shift in reward-associated learning strategies plays a key role in the development and maintenance of alcohol use disorders and seems to persist during (early stages of) sobriety in at-risk drinkers. Yet still, it has remained unclear whether this phenomenon might be associated with alcohol hangover and thus also be found in social drinkers. In an experimental crossover design, n = 25 healthy young male participants performed a two-step decision-making task once sober and once hungover (i.e., when reaching sobriety after consuming 2.6 g of alcohol per estimated liter of total body water). This task allows the separation of effortful model-based and computationally less demanding model-free learning strategies. The experimental induction of alcohol hangover was successful, but we found no significant hangover effects on model-based and model-free learning scores, the balance between model-free and model-based valuation (ω), or perseveration tendencies (π). Bayesian analyses provided positive evidence for the null hypothesis for all measures except π (anecdotal evidence for the null hypothesis). Taken together, alcohol hangover, which results from a single binge drinking episode, does not impair the application of effortful and computationally costly model-based learning strategies and/or increase model-free learning strategies. This supports the notion that the behavioral deficits observed in at-risk drinkers are most likely not caused by the immediate aftereffects of individual binge drinking events.
Collapse
Affiliation(s)
- Julia Berghäuser
- Chair of Addiction Research, Institute for Clinical Psychology and Psychotherapy, Faculty of Psychology TU Dresden, Chemnitzer Str. 46, 01062 Dresden, Germany; (J.B.); (T.E.)
| | - Wiebke Bensmann
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Faculty of Medicine, TU Dresden, Fetscherstr. 74, 01307 Dresden, Germany; (W.B.); (N.Z.); (C.B.)
| | - Nicolas Zink
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Faculty of Medicine, TU Dresden, Fetscherstr. 74, 01307 Dresden, Germany; (W.B.); (N.Z.); (C.B.)
| | - Tanja Endrass
- Chair of Addiction Research, Institute for Clinical Psychology and Psychotherapy, Faculty of Psychology TU Dresden, Chemnitzer Str. 46, 01062 Dresden, Germany; (J.B.); (T.E.)
| | - Christian Beste
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Faculty of Medicine, TU Dresden, Fetscherstr. 74, 01307 Dresden, Germany; (W.B.); (N.Z.); (C.B.)
| | - Ann-Kathrin Stock
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Faculty of Medicine, TU Dresden, Fetscherstr. 74, 01307 Dresden, Germany; (W.B.); (N.Z.); (C.B.)
- Correspondence:
| |
Collapse
|
8
|
Groefsema MM, Engels RC, Voon V, Schellekens AF, Luijten M, Sescousse G. Brain responses to anticipating and receiving beer: Comparing light, at-risk, and dependent alcohol users. Addict Biol 2020; 25:e12766. [PMID: 31066137 PMCID: PMC7187239 DOI: 10.1111/adb.12766] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2018] [Revised: 02/26/2019] [Accepted: 03/30/2019] [Indexed: 12/15/2022]
Abstract
Impaired brain processing of alcohol‐related rewards has been suggested to play a central role in alcohol use disorder. Yet, evidence remains inconsistent and mainly originates from studies in which participants passively observe alcohol cues or taste alcohol. Here, we designed a protocol in which beer consumption was predicted by incentive cues and contingent on instrumental action closer to real life situations. We predicted that anticipating and receiving beer (compared with water) would elicit activity in the brain reward network and that this activity would correlate with drinking level across participants. The sample consisted of 150 beer‐drinking males, aged 18 to 25 years. Three groups were defined based on alcohol use disorders identification test (AUDIT) scores: light drinkers (n = 39), at‐risk drinkers (n = 64), and dependent drinkers (n = 47). fMRI measures were obtained while participants engaged in the beer incentive delay task involving beer‐ and water‐predicting cues followed by real sips of beer or water. During anticipation, outcome notification and delivery of beer compared with water, higher activity was found in a reward‐related brain network including the dorsal medial prefrontal cortex, orbitofrontal cortex, and amygdala. Yet, no activity was observed in the striatum, and no differences were found between the groups. Our results reveal that anticipating, obtaining, and tasting beer activates parts of the brain reward network, but that these brain responses do not differentiate between different drinking levels.
Collapse
Affiliation(s)
- Martine M. Groefsema
- Executive BoardRadboud University, Behavioural Science Institute Nijmegen The Netherlands
| | | | - Valerie Voon
- Cambridge University, Behavioural and Clinical Neuroscience Institute Cambridge United Kingdom
| | | | - Maartje Luijten
- Executive BoardRadboud University, Behavioural Science Institute Nijmegen The Netherlands
| | - Guillaume Sescousse
- Radboud University, Donders Institute for Brain, Cognition and Behaviour Nijmegen The Netherlands
- Centre de Recherche en Neurosciences de Lyon, INSERM U1028, CNRS UMR5292, PSYR2 Team Lyon France
- CH Le Vinatier, Service Universitaire d'Addictologie Bron France
| |
Collapse
|
9
|
Hogarth L. Addiction is driven by excessive goal-directed drug choice under negative affect: translational critique of habit and compulsion theory. Neuropsychopharmacology 2020; 45:720-735. [PMID: 31905368 PMCID: PMC7265389 DOI: 10.1038/s41386-020-0600-8] [Citation(s) in RCA: 119] [Impact Index Per Article: 29.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/05/2019] [Revised: 12/09/2019] [Accepted: 12/18/2019] [Indexed: 01/21/2023]
Abstract
Drug addiction may be a goal-directed choice driven by excessive drug value in negative affective states, a habit driven by strong stimulus-response associations, or a compulsion driven by insensitivity to costs imposed on drug seeking. Laboratory animal and human evidence for these three theories is evaluated. Excessive goal theory is supported by dependence severity being associated with greater drug choice/economic demand. Drug choice is demonstrably goal-directed (driven by the expected value of the drug) and can be augmented by stress/negative mood induction and withdrawal-effects amplified in those with psychiatric symptoms and drug use coping motives. Furthermore, psychiatric symptoms confer risk of dependence, and coping motives mediate this risk. Habit theory of addiction has weaker support. Habitual behaviour seen in drug-exposed animals often does not occur in complex decision scenarios, or where responding is rewarded, so habit is unlikely to explain most human addictive behaviour where these conditions apply. Furthermore, most human studies have not found greater propensity to habitual behaviour in drug users or as a function of dependence severity, and the minority that have can be explained by task disengagement producing impaired explicit contingency knowledge. Compulsion theory of addiction also has weak support. The persistence of punished drug seeking in animals is better explained by greater drug value (evinced by the association with economic demand) than by insensitivity to costs. Furthermore, human studies have provided weak evidence that propensity to discount cost imposed on drug seeking is associated with dependence severity. These data suggest that human addiction is primarily driven by excessive goal-directed drug choice under negative affect, and less by habit or compulsion. Addiction is pathological because negative states powerfully increase expected drug value acutely outweighing abstinence goals.
Collapse
Affiliation(s)
- Lee Hogarth
- School of Psychology, University of Exeter, Washington Singer Building, Perry Road, Exeter, EX4 4QG, UK.
| |
Collapse
|
10
|
Abstract
Compulsive behaviors (e.g., addiction) can be viewed as an aberrant decision process where inflexible reactions automatically evoked by stimuli (habit) take control over decision making to the detriment of a more flexible (goal-oriented) behavioral learning system. These behaviors are thought to arise from learning algorithms known as “model-based” and “model-free” reinforcement learning. Gambling disorder, a form of addiction without the confound of neurotoxic effects of drugs, showed impaired goal-directed control but the way in which problem gamblers (PG) orchestrate model-based and model-free strategies has not been evaluated. Forty-nine PG and 33 healthy participants (CP) completed a two-step sequential choice task for which model-based and model-free learning have distinct and identifiable trial-by-trial learning signatures. The influence of common psychopathological comorbidities on those two forms of learning were investigated. PG showed impaired model-based learning, particularly after unrewarded outcomes. In addition, PG exhibited faster reaction times than CP following unrewarded decisions. Troubled mood, higher impulsivity (i.e., positive and negative urgency) and current and chronic stress reported via questionnaires did not account for those results. These findings demonstrate specific reinforcement learning and decision-making deficits in behavioral addiction that advances our understanding and may be important dimensions for designing effective interventions.
Collapse
|
11
|
Ottenheimer DJ, Wang K, Haimbaugh A, Janak PH, Richard JM. Recruitment and disruption of ventral pallidal cue encoding during alcohol seeking. Eur J Neurosci 2019; 50:3428-3444. [PMID: 31338915 DOI: 10.1111/ejn.14527] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2019] [Revised: 06/14/2019] [Accepted: 07/15/2019] [Indexed: 01/18/2023]
Abstract
A critical area of inquiry in the neurobiology of alcohol abuse is the mechanism by which cues gain the ability to elicit alcohol use. Previously, we found that cue-evoked activity in rat ventral pallidum robustly encodes the value of sucrose cues trained under both Pavlovian and instrumental contingencies, despite a stronger relationship between cue-evoked activity and behavioral latency after instrumental training (Richard et al., 2018, Elife, 7, e33107). Here, we assessed: (a) ventral pallidal representations of Pavlovian versus instrumental cues trained with alcohol reward, and (b) the impact of non-associative alcohol exposure on ventral pallidal representations of sucrose cues. Decoding of cue identity based on ventral pallidum firing was blunted for the Pavlovian alcohol cue in comparison to both the instrumental cue trained with alcohol and either cue type trained with sucrose. Further, non-associative alcohol exposure had opposing effects on ventral pallidal encoding of sucrose cues trained on instrumental versus Pavlovian associations, enhancing decoding accuracy for an instrumental discriminative stimulus and reducing decoding accuracy for a Pavlovian conditioned stimulus. These findings suggest that alcohol exposure can drive biased engagement of specific reward-related signals in the ventral pallidum.
Collapse
Affiliation(s)
- David J Ottenheimer
- Solomon H. Snyder Department of Neuroscience, Johns Hopkins School of Medicine, Johns Hopkins University, Baltimore, MD, USA
| | - Karen Wang
- Department of Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD, USA
| | - Alexandria Haimbaugh
- Department of Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD, USA
| | - Patricia H Janak
- Solomon H. Snyder Department of Neuroscience, Johns Hopkins School of Medicine, Johns Hopkins University, Baltimore, MD, USA.,Department of Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD, USA.,Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD, USA
| | - Jocelyn M Richard
- Department of Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD, USA.,Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
12
|
Ognibene D, Fiore VG, Gu X. Addiction beyond pharmacological effects: The role of environment complexity and bounded rationality. Neural Netw 2019; 116:269-278. [PMID: 31125913 PMCID: PMC6581592 DOI: 10.1016/j.neunet.2019.04.022] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2018] [Revised: 04/06/2019] [Accepted: 04/25/2019] [Indexed: 02/03/2023]
Abstract
Several decision-making vulnerabilities have been identified as underlying causes for addictive behaviours, or the repeated execution of stereotyped actions despite their adverse consequences. These vulnerabilities are mostly associated with brain alterations caused by the consumption of substances of abuse. However, addiction can also happen in the absence of a pharmacological component, such as seen in pathological gambling and videogaming. We use a new reinforcement learning model to highlight a previously neglected vulnerability that we suggest interacts with those already identified, whilst playing a prominent role in non-pharmacological forms of addiction. Specifically, we show that a dual-learning system (i.e. combining model-based and model-free) can be vulnerable to highly rewarding, but suboptimal actions, that are followed by a complex ramification of stochastic adverse effects. This phenomenon is caused by the overload of the capabilities of an agent, as time and cognitive resources required for exploration, deliberation, situation recognition, and habit formation, all increase as a function of the depth and richness of detail of an environment. Furthermore, the cognitive overload can be aggravated due to alterations (e.g. caused by stress) in the bounded rationality, i.e. the limited amount of resources available for the model-based component, in turn increasing the agent's chances to develop or maintain addictive behaviours. Our study demonstrates that, independent of drug consumption, addictive behaviours can arise in the interaction between the environmental complexity and the biologically finite resources available to explore and represent it.
Collapse
Affiliation(s)
- Dimitri Ognibene
- School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK; ETIC, Universitat Pompeu Fabra, Barcelona, Spain.
| | - Vincenzo G Fiore
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Xiaosi Gu
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA; Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA; The Mental Illness Research, Education, and Clinical Center (MIRECC VISN 2) at the James J. Peter Veterans Affairs Medical Center, Bronx, NY, USA
| |
Collapse
|
13
|
Heller AS, Ezie CEC, Otto AR, Timpano KR. Model-based learning and individual differences in depression: The moderating role of stress. Behav Res Ther 2018; 111:19-26. [PMID: 30273768 DOI: 10.1016/j.brat.2018.09.007] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2017] [Revised: 09/19/2018] [Accepted: 09/24/2018] [Indexed: 11/19/2022]
Abstract
Inflexible decision-making has been proposed as a transdiagnostic risk factor for mood disorders. Evidence suggests that inflexible decision-making may emerge only when individuals are experiencing increased negative affect or stress. 151 participants completed symptom measures of depression and anxiety, followed by a two-stage decision-making task that distinguishes between habitual and goal-directed choice. An experimental manipulation to induce stress was introduced halfway through the task. Individuals with higher depression levels became less model-based after the manipulation than those with lower depression levels. There was no relationship between trait anxiety and the impact of the manipulation on decision-making. Controlling for main effects of anxiety did not attenuate the association between depression and impact of stress. Anhedonia was associated with the impact of the manipulation on model-based decision-making. These results suggest that risk for depression is associated with reflexive decision-making, but these effects may only emerge under conditions of stress.
Collapse
Affiliation(s)
- Aaron S Heller
- Department of Psychology, University of Miami, Coral Gables, FL, 33146, USA.
| | - C E Chiemeka Ezie
- Department of Psychology, University of Miami, Coral Gables, FL, 33146, USA
| | - A Ross Otto
- Department of Psychology, McGill University, 2001 McGill College Avenue, Montréal, QC, H3A 1G1, Canada
| | - Kiara R Timpano
- Department of Psychology, University of Miami, Coral Gables, FL, 33146, USA
| |
Collapse
|
14
|
Hogarth L, Lam‐Cassettari C, Pacitti H, Currah T, Mahlberg J, Hartley L, Moustafa A. Intact goal‐directed control in treatment‐seeking drug users indexed by outcome‐devaluation and Pavlovian to instrumental transfer: critique of habit theory. Eur J Neurosci 2018; 50:2513-2525. [DOI: 10.1111/ejn.13961] [Citation(s) in RCA: 36] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2018] [Revised: 04/16/2018] [Accepted: 04/17/2018] [Indexed: 12/17/2022]
Affiliation(s)
- Lee Hogarth
- School of Psychology University of Exeter Exeter UK
- School of Psychology University of New South Wales Sydney NSW Australia
| | - Christa Lam‐Cassettari
- MARCS Institute for Brain, Behaviour and Development Western Sydney University Sydney NSW Australia
| | - Helena Pacitti
- School of Psychology University of New South Wales Sydney NSW Australia
| | - Tara Currah
- School of Psychology University of Exeter Exeter UK
| | - Justin Mahlberg
- School of Social Sciences and Psychology Western Sydney University Sydney NSW Australia
| | | | - Ahmed Moustafa
- MARCS Institute for Brain, Behaviour and Development Western Sydney University Sydney NSW Australia
- School of Social Sciences and Psychology Western Sydney University Sydney NSW Australia
| |
Collapse
|
15
|
Behavioral and Neural Signatures of Reduced Updating of Alternative Options in Alcohol-Dependent Patients during Flexible Decision-Making. J Neurosci 2017; 36:10935-10948. [PMID: 27798176 DOI: 10.1523/jneurosci.4322-15.2016] [Citation(s) in RCA: 56] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2015] [Accepted: 08/14/2016] [Indexed: 01/09/2023] Open
Abstract
Addicted individuals continue substance use despite the knowledge of harmful consequences and often report having no choice but to consume. Computational psychiatry accounts have linked this clinical observation to difficulties in making flexible and goal-directed decisions in dynamic environments via consideration of potential alternative choices. To probe this in alcohol-dependent patients (n = 43) versus healthy volunteers (n = 35), human participants performed an anticorrelated decision-making task during functional neuroimaging. Via computational modeling, we investigated behavioral and neural signatures of inference regarding the alternative option. While healthy control subjects exploited the anticorrelated structure of the task to guide decision-making, alcohol-dependent patients were relatively better explained by a model-free strategy due to reduced inference on the alternative option after punishment. Whereas model-free prediction error signals were preserved, alcohol-dependent patients exhibited blunted medial prefrontal signatures of inference on the alternative option. This reduction was associated with patients' behavioral deficit in updating the alternative choice option and their obsessive-compulsive drinking habits. All results remained significant when adjusting for potential confounders (e.g., neuropsychological measures and gray matter density). A disturbed integration of alternative choice options implemented by the medial prefrontal cortex appears to be one important explanation for the puzzling question of why addicted individuals continue drug consumption despite negative consequences. SIGNIFICANCE STATEMENT In addiction, patients maintain substance use despite devastating consequences and often report having no choice but to consume. These clinical observations have been theoretically linked to disturbed mechanisms of inference, for example, to difficulties when learning statistical regularities of the environmental structure to guide decisions. Using computational modeling, we demonstrate disturbed inference on alternative choice options in alcohol addiction. Patients neglecting "what might have happened" was accompanied by blunted coding of inference regarding alternative choice options in the medial prefrontal cortex. An impaired integration of alternative choice options implemented by the medial prefrontal cortex might contribute to ongoing drug consumption in the face of evident negative consequences.
Collapse
|
16
|
Reiter A, Heinz A, Deserno L. Linking social context and addiction neuroscience: a computational psychiatry approach. Nat Rev Neurosci 2017. [PMID: 28626229 DOI: 10.1038/nrn.2017.67] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Affiliation(s)
- Andrea Reiter
- Department for Lifespan Developmental Neuroscience, Technical University Dresden, 01062 Dresden, Germany; and at the Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany
| | - Andreas Heinz
- Department of Psychiatry and Psychotherapy, Charité Universitätsmedizin Berlin, 10117 Berlin, Germany
| | - Lorenz Deserno
- Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University of Leipzig, 04103 Leipzig, Germany; and at the Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany
| |
Collapse
|
17
|
Ekhtiari H, Victor TA, Paulus MP. Aberrant decision-making and drug addiction — how strong is the evidence? Curr Opin Behav Sci 2017. [DOI: 10.1016/j.cobeha.2016.09.002] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
|
18
|
Sjoerds Z, Dietrich A, Deserno L, de Wit S, Villringer A, Heinze HJ, Schlagenhauf F, Horstmann A. Slips of Action and Sequential Decisions: A Cross-Validation Study of Tasks Assessing Habitual and Goal-Directed Action Control. Front Behav Neurosci 2016; 10:234. [PMID: 28066200 PMCID: PMC5167743 DOI: 10.3389/fnbeh.2016.00234] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2016] [Accepted: 11/28/2016] [Indexed: 11/25/2022] Open
Abstract
Instrumental learning and decision-making rely on two parallel systems: a goal-directed and a habitual system. In the past decade, several paradigms have been developed to study these systems in animals and humans by means of e.g., overtraining, devaluation procedures and sequential decision-making. These different paradigms are thought to measure the same constructs, but cross-validation has rarely been investigated. In this study we compared two widely used paradigms that assess aspects of goal-directed and habitual behavior. We correlated parameters from a two-step sequential decision-making task that assesses model-based (MB) and model-free (MF) learning with a slips-of-action paradigm that assesses the ability to suppress cue-triggered, learnt responses when the outcome has been devalued and is therefore no longer desirable. MB control during the two-step task showed a very moderately positive correlation with goal-directed devaluation sensitivity, whereas MF control did not show any associations. Interestingly, parameter estimates of MB and goal-directed behavior in the two tasks were positively correlated with higher-order cognitive measures (e.g., visual short-term memory). These cognitive measures seemed to (at least partly) mediate the association between MB control during sequential decision-making and goal-directed behavior after instructed devaluation. This study provides moderate support for a common framework to describe the propensity towards goal-directed behavior as measured with two frequently used tasks. However, we have to caution that the amount of shared variance between the goal-directed and MB system in both tasks was rather low, suggesting that each task does also pick up distinct aspects of goal-directed behavior. Further investigation of the commonalities and differences between the MF and habit systems as measured with these, and other, tasks is needed. Also, a follow-up cross-validation on the neural systems driving these constructs across different paradigms would promote the definition and operationalization of measures of instrumental learning and decision-making in humans.
Collapse
Affiliation(s)
- Zsuzsika Sjoerds
- Department of Neurology, Max-Planck Institute for Human Cognitive and Brain Sciences Leipzig, Germany
| | - Anja Dietrich
- Department of Neurology, Max-Planck Institute for Human Cognitive and Brain Sciences Leipzig, Germany
| | - Lorenz Deserno
- Department of Neurology, Max-Planck Institute for Human Cognitive and Brain SciencesLeipzig, Germany; Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité - Universitätsmedizin BerlinBerlin, Germany; Department of Neurology, Otto-von-Guericke UniversityMagdeburg, Germany
| | - Sanne de Wit
- Department of Clinical Psychology, University of AmsterdamAmsterdam, Netherlands; Amsterdam Brain and Cognition, University of AmsterdamAmsterdam, Netherlands
| | - Arno Villringer
- Department of Neurology, Max-Planck Institute for Human Cognitive and Brain SciencesLeipzig, Germany; Mind and Brain Institute, Charité and Humboldt UniversityBerlin, Germany
| | - Hans-Jochen Heinze
- Department of Neurology, Otto-von-Guericke UniversityMagdeburg, Germany; Department of Behavioral Neurology, Leibniz Institute for NeurobiologyMagdeburg, Germany
| | - Florian Schlagenhauf
- Department of Neurology, Max-Planck Institute for Human Cognitive and Brain SciencesLeipzig, Germany; Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité - Universitätsmedizin BerlinBerlin, Germany
| | - Annette Horstmann
- Department of Neurology, Max-Planck Institute for Human Cognitive and Brain SciencesLeipzig, Germany; Integrated Research and Treatment Center Adiposity Diseases, Leipzig University Medical CenterLeipzig, Germany
| |
Collapse
|
19
|
Reiter AMF, Koch SP, Schröger E, Hinrichs H, Heinze HJ, Deserno L, Schlagenhauf F. The Feedback-related Negativity Codes Components of Abstract Inference during Reward-based Decision-making. J Cogn Neurosci 2016; 28:1127-38. [DOI: 10.1162/jocn_a_00957] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
Abstract
Behavioral control is influenced not only by learning from the choices made and the rewards obtained but also by “what might have happened,” that is, inference about unchosen options and their fictive outcomes. Substantial progress has been made in understanding the neural signatures of direct learning from choices that are actually made and their associated rewards via reward prediction errors (RPEs). However, electrophysiological correlates of abstract inference in decision-making are less clear. One seminal theory suggests that the so-called feedback-related negativity (FRN), an ERP peaking 200–300 msec after a feedback stimulus at frontocentral sites of the scalp, codes RPEs. Hitherto, the FRN has been predominantly related to a so-called “model-free” RPE: The difference between the observed outcome and what had been expected. Here, by means of computational modeling of choice behavior, we show that individuals employ abstract, “double-update” inference on the task structure by concurrently tracking values of chosen stimuli (associated with observed outcomes) and unchosen stimuli (linked to fictive outcomes). In a parametric analysis, model-free RPEs as well as their modification because of abstract inference were regressed against single-trial FRN amplitudes. We demonstrate that components related to abstract inference uniquely explain variance in the FRN beyond model-free RPEs. These findings advance our understanding of the FRN and its role in behavioral adaptation. This might further the investigation of disturbed abstract inference, as proposed, for example, for psychiatric disorders, and its underlying neural correlates.
Collapse
Affiliation(s)
- Andrea M. F. Reiter
- 1Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- 2University of Leipzig
- 3Technische Universität Dresden
| | | | | | - Hermann Hinrichs
- 5Leibniz Institute for Neurobiology, Magdeburg, Germany
- 6Otto-von-Guericke University Magdeburg
| | - Hans-Jochen Heinze
- 5Leibniz Institute for Neurobiology, Magdeburg, Germany
- 6Otto-von-Guericke University Magdeburg
| | - Lorenz Deserno
- 1Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- 4Charité-Universitätsmedizin Berlin
- 6Otto-von-Guericke University Magdeburg
| | - Florian Schlagenhauf
- 1Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- 4Charité-Universitätsmedizin Berlin
| |
Collapse
|