1
|
Vahedi J, Mundorf A, Bellebaum C, Peterburs J. Emotional cues reduce Pavlovian interference in feedback-based go and nogo learning. PSYCHOLOGICAL RESEARCH 2024; 88:1212-1230. [PMID: 38483574 PMCID: PMC11142951 DOI: 10.1007/s00426-024-01946-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Accepted: 02/26/2024] [Indexed: 06/02/2024]
Abstract
It is easier to execute a response in the promise of a reward and withhold a response in the promise of a punishment than vice versa, due to a conflict between cue-related Pavlovian and outcome-related instrumental action tendencies in the reverse conditions. This robust learning asymmetry in go and nogo learning is referred to as the Pavlovian bias. Interestingly, it is similar to motivational tendencies reported for affective facial expressions, i.e., facilitation of approach to a smile and withdrawal from a frown. The present study investigated whether and how learning from emotional faces instead of abstract stimuli modulates the Pavlovian bias in reinforcement learning. To this end, 137 healthy adult participants performed an orthogonalized Go/Nogo task that fully decoupled action (go/nogo) and outcome valence (win points/avoid losing points). Three groups of participants were tested with either emotional facial cues whose affective valence was either congruent (CON) or incongruent (INC) to the required instrumental response, or with neutral facial cues (NEU). Relative to NEU, the Pavlovian bias was reduced in both CON and INC, though still present under all learning conditions. Importantly, only for CON, the reduction of the Pavlovian bias effect was adaptive by improving learning performance in one of the conflict conditions. In contrast, the reduction of the Pavlovian bias in INC was completely driven by decreased learning performance in non-conflict conditions. These results suggest a potential role of arousal/salience in Pavlovian-instrumental regulation and cue-action congruency in the adaptability of goal-directed behavior. Implications for clinical application are discussed.
Collapse
Affiliation(s)
- Julian Vahedi
- Faculty of Mathematics and Natural Sciences, Heinrich Heine University Düsseldorf, Düsseldorf, Germany.
| | - Annakarina Mundorf
- Institute for Systems Medicine, Department of Human Medicine, MSH Medical School, Hamburg, Germany
| | - Christian Bellebaum
- Faculty of Mathematics and Natural Sciences, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Jutta Peterburs
- Institute for Systems Medicine, Department of Human Medicine, MSH Medical School, Hamburg, Germany
| |
Collapse
|
2
|
Pedersen R, Johansson J, Nordin K, Rieckmann A, Wåhlin A, Nyberg L, Bäckman L, Salami A. Dopamine D1-Receptor Organization Contributes to Functional Brain Architecture. J Neurosci 2024; 44:e0621232024. [PMID: 38302439 PMCID: PMC10941071 DOI: 10.1523/jneurosci.0621-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Revised: 12/01/2023] [Accepted: 01/21/2024] [Indexed: 02/03/2024] Open
Abstract
Recent work has recognized a gradient-like organization in cortical function, spanning from primary sensory to transmodal cortices. It has been suggested that this axis is aligned with regional differences in neurotransmitter expression. Given the abundance of dopamine D1-receptors (D1DR), and its importance for modulation and neural gain, we tested the hypothesis that D1DR organization is aligned with functional architecture, and that inter-regional relationships in D1DR co-expression modulate functional cross talk. Using the world's largest dopamine D1DR-PET and MRI database (N = 180%, 50% female), we demonstrate that D1DR organization follows a unimodal-transmodal hierarchy, expressing a high spatial correspondence to the principal gradient of functional connectivity. We also demonstrate that individual differences in D1DR density between unimodal and transmodal regions are associated with functional differentiation of the apices in the cortical hierarchy. Finally, we show that spatial co-expression of D1DR primarily modulates couplings within, but not between, functional networks. Together, our results show that D1DR co-expression provides a biomolecular layer to the functional organization of the brain.
Collapse
Affiliation(s)
- Robin Pedersen
- Department of Integrative Medical Biology, Umeå University, Umeå S-90197, Sweden
- Wallenberg Center for Molecular Medicine (WCMM), Umeå University, Umeå S-90197, Sweden
- Umeå Center for Functional Brain Imaging (UFBI), Umeå University, Umeå S-90197, Sweden
| | - Jarkko Johansson
- Department of Integrative Medical Biology, Umeå University, Umeå S-90197, Sweden
- Wallenberg Center for Molecular Medicine (WCMM), Umeå University, Umeå S-90197, Sweden
- Umeå Center for Functional Brain Imaging (UFBI), Umeå University, Umeå S-90197, Sweden
| | - Kristin Nordin
- Umeå Center for Functional Brain Imaging (UFBI), Umeå University, Umeå S-90197, Sweden
- Aging Research Center, Karolinska Institutet & Stockholm University, Stockholm S-17165, Sweden
| | - Anna Rieckmann
- Umeå Center for Functional Brain Imaging (UFBI), Umeå University, Umeå S-90197, Sweden
- Department of Radiation Sciences, Umeå University, Umeå S-90197, Sweden
- Max-Planck-Institut für Sozialrecht und Sozialpolitik, Munich 80799, Germany
| | - Anders Wåhlin
- Department of Integrative Medical Biology, Umeå University, Umeå S-90197, Sweden
- Umeå Center for Functional Brain Imaging (UFBI), Umeå University, Umeå S-90197, Sweden
| | - Lars Nyberg
- Department of Integrative Medical Biology, Umeå University, Umeå S-90197, Sweden
- Wallenberg Center for Molecular Medicine (WCMM), Umeå University, Umeå S-90197, Sweden
- Umeå Center for Functional Brain Imaging (UFBI), Umeå University, Umeå S-90197, Sweden
- Department of Radiation Sciences, Umeå University, Umeå S-90197, Sweden
| | - Lars Bäckman
- Aging Research Center, Karolinska Institutet & Stockholm University, Stockholm S-17165, Sweden
| | - Alireza Salami
- Department of Integrative Medical Biology, Umeå University, Umeå S-90197, Sweden
- Wallenberg Center for Molecular Medicine (WCMM), Umeå University, Umeå S-90197, Sweden
- Umeå Center for Functional Brain Imaging (UFBI), Umeå University, Umeå S-90197, Sweden
- Aging Research Center, Karolinska Institutet & Stockholm University, Stockholm S-17165, Sweden
| |
Collapse
|
3
|
Algermissen J, Swart JC, Scheeringa R, Cools R, den Ouden HEM. Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases. Nat Commun 2024; 15:19. [PMID: 38168089 PMCID: PMC10762147 DOI: 10.1038/s41467-023-44632-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 12/22/2023] [Indexed: 01/05/2024] Open
Abstract
Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.
Collapse
Affiliation(s)
- Johannes Algermissen
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| | - Jennifer C Swart
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - René Scheeringa
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, Essen, Germany
| | - Roshan Cools
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Department of Psychiatry, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Hanneke E M den Ouden
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| |
Collapse
|
4
|
Conn KA, Alexander S, Burne THJ, Kesby JP. Antagonism of D2 receptors via raclopride ameliorates amphetamine-induced associative learning deficits in male mice. Behav Brain Res 2023; 454:114649. [PMID: 37643667 DOI: 10.1016/j.bbr.2023.114649] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 08/09/2023] [Accepted: 08/25/2023] [Indexed: 08/31/2023]
Abstract
Dopamine levels in the dorsomedial striatum (DMS) are highly dynamic and are thought to underly the encoding of action-outcome associations. Although it is known that amphetamine disrupts the learning that is required for goal-directed action, the role of D1 and D2 receptors in this process has not been established. In this study, we examined the role of D1 and D2 receptor antagonists on learning in response to amphetamine. We used the outcome-specific devaluation task to examine goal-directed action in male C57BL6/J mice treated systemically with either a D1 antagonist (SCH-23990; 0.01 mg/kg) or a D2 antagonist (raclopride; 0.5 mg/kg) and then administered amphetamine (1 mg/kg). The mice were injected repeatedly throughout the instrumental training phase of the task to assess the impact on the learning of action-outcomes, and the subsequent choice test assessing performance of goal-directed action was conducted drug free. Effects of chronic drug administration on locomotor behaviour was assessed before and after the choice test. Treatment during learning with either amphetamine, or the D1 or D2 antagonists, impaired the subsequent performance of goal-directed action. The amphetamine-induced impairment in goal-directed action was reversed in mice treated with raclopride, but not when treated with SCH-23990. By contrast, amphetamine-induced hyperactivity was reversed in mice treated with SCH-23990, but not in mice treated with raclopride. Taken together, these data support the role of a balance of dopamine receptor signalling after amphetamine treatment. While overall D1 receptor availability is necessary to promote learning, in a state of elevated dopamine, modifying D2 receptor function can ameliorate learning deficits.
Collapse
Affiliation(s)
- Kyna-Anne Conn
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia
| | - Suzy Alexander
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia
| | - Thomas H J Burne
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia; Queensland Centre for Mental Health Research, Wacol, QLD 4076, Australia
| | - James P Kesby
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia; Queensland Centre for Mental Health Research, Wacol, QLD 4076, Australia.
| |
Collapse
|
5
|
Saeedpour S, Hossein MM, Deroy O, Bahrami B. Interindividual differences in Pavlovian influence on learning are consistent. ROYAL SOCIETY OPEN SCIENCE 2023; 10:230447. [PMID: 37736528 PMCID: PMC10509574 DOI: 10.1098/rsos.230447] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Accepted: 08/29/2023] [Indexed: 09/23/2023]
Abstract
Pavlovian influences impair instrumental learning. It is easier to learn to approach reward-predictive signals and avoid punishment-predictive cues than their contrary. Whether the interindividual variability in this Pavlovian influence is consistent across time has been examined by a number of recent studies and met with mixed results. Here we introduce an open-source, web-based instance of a well-established Go-NoGo paradigm for measuring Pavlovian influence. We closely replicated the previous laboratory-based results. Moreover, the interindividual differences in Pavlovian influence were consistent across a two-week time window at the level of (i) raw measures of learning (i.e. performance accuracy), (ii) linear, descriptive estimates of Pavlovian bias (test-retest reliability: 0.40), and (iii) parameters obtained from reinforcement learning model fitting and model selection (test-retest reliability: 0.25). Nonetheless, the correlations reported here are still lower than the standards (i.e. 0.7) employed in psychometrics and self-reported measures. Our results provide support for trusting Pavlovian bias as a relatively stable individual characteristic and for using its measure in the computational understanding of human mental health.
Collapse
Affiliation(s)
- Sepehr Saeedpour
- Department of Electrical and Computer Engineering, University of Tehran, Tehran, Iran
| | | | - Ophelia Deroy
- Faculty of Philosophy, Ludwig Maximilian University, Munich, Germany
- Munich Center for Neuroscience, Ludwig Maximilian University, Munich, Germany
- School of Advanced Study, University of London, London, UK
| | - Bahador Bahrami
- Faculty of General Psychology and Education, Ludwig Maximilian University, Munich, Germany
| |
Collapse
|
6
|
Taherianfard M, Ahmadijokani S. The passive avoidance task ameliorate the toxic effects of bisphenol A on dopamine D1 receptor density in hippocampus, amygdala, and cerebellum of male rats. Brain Behav 2023; 13:e2942. [PMID: 36879399 PMCID: PMC10097143 DOI: 10.1002/brb3.2942] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/27/2021] [Revised: 03/21/2022] [Accepted: 03/23/2022] [Indexed: 03/08/2023] Open
Abstract
INTRODUCTION Dopamine D1 receptor seems to play a role in mediating plasticity. Therefore, the present study aimed to investigate the effects of passive avoidance tasks postexposed to BPA on dopamine D1 receptor density in the hippocampus, amygdala, and cerebellum of male rats. METHODS Thirty-five male Sprague-Dawley rats weighing 220.300 g, in standard light-dark 12 h light/12 h dark were used in the present study; water and food were ad libitum. Animals were divided into six groups. Administration of BPA 5 and 50 mg/kg/day were gavaged for 15 days. Learning and memory assessment were done by a shuttle box after 15 days of BPA administration. The density of the dopamine D1 receptor was investigated using an immunohistochemistry (IH) procedure. For determining the color difference in IH sections, Image Analyzer software was used. The data were analyzed by one-way ANOVA followed by Tukey's as a post hoc test. RESULTS The data showed that BPA in both doses could significantly increase the density of dopamine D1 receptors in the hippocampus, amygdala, and cerebellum of male rats; learning in rats postexposed to BPA improves dopamine D1 receptor density significantly in three brain structures. DISCUSSION According to the results, passive avoidance learning and memory can improve the density of dopamine D1 receptors in the hippocampus, amygdala, and cerebellum of male rats.
Collapse
Affiliation(s)
- Mahnaz Taherianfard
- Physiology Division of Basic Science Department, School of Veterinary Medicine, Shiraz University, Shiraz, Iran
| | - Saiedeh Ahmadijokani
- Physiology Division of Basic Science Department, School of Veterinary Medicine, Shiraz University, Shiraz, Iran
| |
Collapse
|
7
|
Hird E, Beierholm U, De Boer L, Axelsson J, Beckman L, Guitart-Masip M. Dopamine and reward-related vigor in younger and older adults. Neurobiol Aging 2022; 118:34-43. [DOI: 10.1016/j.neurobiolaging.2022.06.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2021] [Revised: 06/10/2022] [Accepted: 06/12/2022] [Indexed: 10/18/2022]
|
8
|
Luciana M, Collins PF. Is Adolescence a Sensitive Period for the Development of Incentive-Reward Motivation? Curr Top Behav Neurosci 2021; 53:79-99. [PMID: 34784026 DOI: 10.1007/7854_2021_275] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
Abstract
Human adolescence is broadly construed as a time of heightened risk-taking and a vulnerability period for the emergence of psychopathology. These tendencies have been attributed to the age-related development of neural systems that mediate incentive motivation and other aspects of reward processing as well as individual difference factors that interact with ongoing development. Here, we describe the adolescent development of incentive motivation, which we view as an inherently positive developmental progression, and its associated neural mechanisms. We consider challenges in applying the sensitive period concept to these maturational events and discuss future directions that may help to clarify mechanisms of change.
Collapse
Affiliation(s)
- Monica Luciana
- Department of Psychology, University of Minnesota, Minneapolis, MN, USA.
| | - Paul F Collins
- Department of Psychology, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
9
|
Yokokura M, Takebasashi K, Takao A, Nakaizumi K, Yoshikawa E, Futatsubashi M, Suzuki K, Nakamura K, Yamasue H, Ouchi Y. In vivo imaging of dopamine D1 receptor and activated microglia in attention-deficit/hyperactivity disorder: a positron emission tomography study. Mol Psychiatry 2021; 26:4958-4967. [PMID: 32439845 DOI: 10.1038/s41380-020-0784-7] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/10/2020] [Revised: 05/05/2020] [Accepted: 05/11/2020] [Indexed: 02/07/2023]
Abstract
Alterations in the cortical dopamine system and microglial activation have been implicated in the pathophysiology of attention-deficit/hyperactivity disorder (ADHD), one of neurodevelopmental disorders that can be conventionally treated with a dopamine enhancer (methylphenidate) albeit unsatisfactorily. Here, we investigated the contributions of the dopamine D1 receptor (D1R) and activated microglia and their interactions to the clinical severities in ADHD individuals using positron emission tomography (PET). Twenty-four psychotropic-naïve ADHD individuals and 24 age- and sex-matched typically developing (TD) subjects underwent PET measurements with [11C]SCH23390 for the D1R and [11C](R)PK11195 for activated microglia as well as assessments of clinical symptoms and cognitive functions. The ADHD individuals showed decreased D1R in the anterior cingulate cortex (ACC) and increased activated microglia in the dorsolateral prefrontal cortex (DLPFC) and orbitofrontal cortex (OFC) compared with the TD subjects. The decreased D1R in the ACC was associated with severe hyperactivity in the participants with ADHD. Microglial activation in the DLPFC were associated with deficits in processing speed and attentional ability, and that in the OFC was correlated with lower processing speed in the ADHD individuals. Furthermore, positive correlations between the D1R and activated microglia in both the DLPFC and the OFC were found to be significantly specific to the ADHD group and not to the TD group. The current findings suggest that microglial activation and the D1R reduction as well as their aberrant interactions underpin the neurophysiological mechanism of ADHD and indicate these biomolecular changes as a novel therapeutic target.
Collapse
Affiliation(s)
- Masamichi Yokokura
- Department of Psychiatry, Hamamatsu University School of Medicine, Hamamatsu, Japan
| | - Kiyokazu Takebasashi
- Department of Psychiatry, Hamamatsu University School of Medicine, Hamamatsu, Japan
| | | | - Kyoko Nakaizumi
- Department of Psychiatry, Hamamatsu University School of Medicine, Hamamatsu, Japan
| | - Etsuji Yoshikawa
- Central Research Laboratory, Hamamatsu Photonics K.K, Hamamatsu, Japan
| | - Masami Futatsubashi
- Global Strategic Challenge Center, Hamamatsu Photonics K.K, Hamamatsu, Japan.,Hamamatsu PET Imaging Center, Hamamatsu Medical Photonics Foundation, Hamamatsu, Japan
| | - Katsuaki Suzuki
- Department of Biofunctional Imaging, Hamamatsu University School of Medicine, Hamamatsu, Japan
| | | | - Hidenori Yamasue
- Department of Psychiatry, Hamamatsu University School of Medicine, Hamamatsu, Japan
| | - Yasuomi Ouchi
- Department of Biofunctional Imaging, Hamamatsu University School of Medicine, Hamamatsu, Japan.
| |
Collapse
|
10
|
Richter A, de Boer L, Guitart-Masip M, Behnisch G, Seidenbecher CI, Schott BH. Motivational learning biases are differentially modulated by genetic determinants of striatal and prefrontal dopamine function. J Neural Transm (Vienna) 2021; 128:1705-1720. [PMID: 34302222 PMCID: PMC8536632 DOI: 10.1007/s00702-021-02382-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Accepted: 07/04/2021] [Indexed: 01/20/2023]
Abstract
Dopaminergic neurotransmission plays a pivotal role in appetitively motivated behavior in mammals, including humans. Notably, action and valence are not independent in motivated tasks, and it is particularly difficult for humans to learn the inhibition of an action to obtain a reward. We have previously observed that the carriers of the DRD2/ANKK1 TaqIA A1 allele, that has been associated with reduced striatal dopamine D2 receptor expression, showed a diminished learning performance when required to learn response inhibition to obtain rewards, a finding that was replicated in two independent cohorts. With our present study, we followed two aims: first, we aimed to replicate our finding on the DRD2/ANKK1 TaqIA polymorphism in a third independent cohort (N = 99) and to investigate the nature of the genetic effects more closely using trial-by-trial behavioral analysis and computational modeling in the combined dataset (N = 281). Second, we aimed to assess a potentially modulatory role of prefrontal dopamine availability, using the widely studied COMT Val108/158Met polymorphism as a proxy. We first report a replication of the above mentioned finding. Interestingly, after combining all three cohorts, exploratory analyses regarding the COMT Val108/158Met polymorphism suggest that homozygotes for the Met allele, which has been linked to higher prefrontal dopaminergic tone, show a lower learning bias. Our results corroborate the importance of genetic variability of the dopaminergic system in individual learning differences of action-valence interaction and, furthermore, suggest that motivational learning biases are differentially modulated by genetic determinants of striatal and prefrontal dopamine function.
Collapse
Affiliation(s)
- Anni Richter
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118, Magdeburg, Germany.
| | - Lieke de Boer
- Ageing Research Centre, Karolinska Institute, Stockholm, Sweden
- Max Planck Institute for Human Development, Center for Lifespan Psychology, Berlin, Germany
| | - Marc Guitart-Masip
- Ageing Research Centre, Karolinska Institute, Stockholm, Sweden
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
| | - Gusalija Behnisch
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118, Magdeburg, Germany
| | - Constanze I Seidenbecher
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118, Magdeburg, Germany
- Center for Behavioral Brain Sciences, Magdeburg, Germany
| | - Björn H Schott
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118, Magdeburg, Germany
- Center for Behavioral Brain Sciences, Magdeburg, Germany
- Department of Psychiatry and Psychotherapy, University Medicine Göttingen, Göttingen, Germany
- Department of Neurology, University of Magdeburg, Magdeburg, Germany
- German Center for Neurodegenerative Diseases (DZNE), Göttingen, Germany
| |
Collapse
|
11
|
Zhang R, Manza P, Tomasi D, Kim SW, Shokri-Kojori E, Demiral SB, Kroll DS, Feldman DE, McPherson KL, Biesecker CL, Wang GJ, Volkow ND. Dopamine D1 and D2 receptors are distinctly associated with rest-activity rhythms and drug reward. J Clin Invest 2021; 131:e149722. [PMID: 34264865 DOI: 10.1172/jci149722] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2021] [Accepted: 07/14/2021] [Indexed: 11/17/2022] Open
Abstract
BACKGROUND Certain components of rest-activity rhythms such as greater eveningness (delayed phase), physical inactivity (blunted amplitude) and shift work (irregularity) are associated with increased risk for drug use. Dopaminergic (DA) signaling has been hypothesized to mediate the associations, though clinical evidence is lacking. METHODS We examined associations between rhythm components and striatal D1 (D1R) and D2/3 receptor (D2/3R) availability in 32 healthy adults (12 female, age: 42.40±12.22) and its relationship to drug reward. Rest-activity rhythms were assessed by one-week actigraphy combined with self-reports. [11C]NNC112 and [11C]raclopride Positron Emission Tomography (PET) scans were conducted to measure D1R and D2/3R availability, respectively. Additionally, self-reported drug-rewarding effects of 60 mg oral methylphenidate were assessed. RESULTS We found that delayed rhythm was associated with higher D1R availability in caudate, which was not attributable to sleep loss or 'social jet lag', whereas physical inactivity was associated with higher D2/3R availability in nucleus accumbens (NAc). Delayed rest-activity rhythm, higher caudate D1R and NAc D2/3R availability were associated with greater sensitivity to the rewarding effects of methylphenidate. CONCLUSION These findings reveal specific components of rest-activity rhythms associated with striatal D1R, D2/3R availability and drug-rewarding effects. Personalized interventions that target rest-activity rhythms may help prevent and treat substance use disorders. TRIAL REGISTRATION ClinicalTrials.gov: NCT03190954FUNDING. This work was accomplished with support from the National Institute on Alcohol Abuse and Alcoholism (ZIAAA000550).
Collapse
Affiliation(s)
- Rui Zhang
- National Institute on Alcohol Abuse and Alcoholism, NIH, Bethesda, United States of America
| | - Peter Manza
- National Institute on Alcohol Abuse and Alcoholism, NIH, Bethesda, United States of America
| | - Dardo Tomasi
- National Institute on Alcohol Abuse and Alcoholism, NIH, Bethesda, United States of America
| | - Sung Won Kim
- National Institute on Alcohol Abuse and Alcoholism, NIH, Bethesda, United States of America
| | - Ehsan Shokri-Kojori
- National Institute on Alcohol Abuse and Alcoholism, NIH, Bethesda, United States of America
| | - Sukru B Demiral
- National Institute on Alcohol Abuse and Alcoholism, NIH, Bethesda, United States of America
| | - Danielle S Kroll
- National Institute on Alcohol Abuse and Alcoholism, NIH, Bethesda, United States of America
| | - Dana E Feldman
- National Institute on Alcohol Abuse and Alcoholism, NIH, Bethesda, United States of America
| | - Katherine L McPherson
- National Institute on Alcohol Abuse and Alcoholism, NIH, Bethesda, United States of America
| | - Catherine L Biesecker
- National Institute on Alcohol Abuse and Alcoholism, NIH, Bethesda, United States of America
| | - Gene-Jack Wang
- National Institute on Alcohol Abuse and Alcoholism, NIH, Bethesda, United States of America
| | - Nora D Volkow
- National Institute on Drug Abuse, NIH, Bethesda, United States of America
| |
Collapse
|
12
|
Moutoussis M, Garzón B, Neufeld S, Bach DR, Rigoli F, Goodyer I, Bullmore E, Guitart-Masip M, Dolan RJ. Decision-making ability, psychopathology, and brain connectivity. Neuron 2021; 109:2025-2040.e7. [PMID: 34019810 PMCID: PMC8221811 DOI: 10.1016/j.neuron.2021.04.019] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2020] [Revised: 02/16/2021] [Accepted: 04/19/2021] [Indexed: 12/11/2022]
Abstract
Decision-making is a cognitive process of central importance for the quality of our lives. Here, we ask whether a common factor underpins our diverse decision-making abilities. We obtained 32 decision-making measures from 830 young people and identified a common factor that we call "decision acuity," which was distinct from IQ and reflected a generic decision-making ability. Decision acuity was decreased in those with aberrant thinking and low general social functioning. Crucially, decision acuity and IQ had dissociable brain signatures, in terms of their associated neural networks of resting-state functional connectivity. Decision acuity was reliably measured, and its relationship with functional connectivity was also stable when measured in the same individuals 18 months later. Thus, our behavioral and brain data identify a new cognitive construct that underpins decision-making ability across multiple domains. This construct may be important for understanding mental health, particularly regarding poor social function and aberrant thought patterns.
Collapse
Affiliation(s)
- Michael Moutoussis
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK.
| | - Benjamín Garzón
- Aging Research Centre, Karolinska Institute, Stockholm, Sweden
| | - Sharon Neufeld
- Department of Psychiatry, University of Cambridge, Cambridge CB2 0SZ, UK
| | - Dominik R Bach
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK; Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, Psychiatric Hospital, University of Zurich, 8032 Zurich, Switzerland
| | | | - Ian Goodyer
- Department of Psychiatry, University of Cambridge, Cambridge CB2 0SZ, UK
| | - Edward Bullmore
- Department of Psychiatry, University of Cambridge, Cambridge CB2 0SZ, UK
| | - Marc Guitart-Masip
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK; Aging Research Centre, Karolinska Institute, Stockholm, Sweden
| | - Raymond J Dolan
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
| |
Collapse
|
13
|
Cataldi S, Stanley AT, Miniaci MC, Sulzer D. Interpreting the role of the striatum during multiple phases of motor learning. FEBS J 2021; 289:2263-2281. [PMID: 33977645 DOI: 10.1111/febs.15908] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2021] [Revised: 03/28/2021] [Accepted: 04/30/2021] [Indexed: 01/11/2023]
Abstract
The synaptic pathways in the striatum are central to basal ganglia functions including motor control, learning and organization, action selection, acquisition of motor skills, cognitive function, and emotion. Here, we review the role of the striatum and its connections in motor learning and performance. The development of new techniques to record neuronal activity and animal models of motor disorders using neurotoxin, pharmacological, and genetic manipulations are revealing pathways that underlie motor performance and motor learning, as well as how they are altered by pathophysiological mechanisms. We discuss approaches that can be used to analyze complex motor skills, particularly in rodents, and identify specific questions central to understanding how striatal circuits mediate motor learning.
Collapse
Affiliation(s)
- Stefano Cataldi
- Departments of Psychiatry, Neurology, Pharmacology, Biology, Columbia University, New York, NY, USA.,Division of Molecular Therapeutics, New York State Psychiatric Institute, NY, USA
| | - Adrien T Stanley
- Departments of Psychiatry, Neurology, Pharmacology, Biology, Columbia University, New York, NY, USA.,Division of Molecular Therapeutics, New York State Psychiatric Institute, NY, USA
| | | | - David Sulzer
- Departments of Psychiatry, Neurology, Pharmacology, Biology, Columbia University, New York, NY, USA.,Division of Molecular Therapeutics, New York State Psychiatric Institute, NY, USA
| |
Collapse
|
14
|
Gershman SJ, Guitart-Masip M, Cavanagh JF. Neural signatures of arbitration between Pavlovian and instrumental action selection. PLoS Comput Biol 2021; 17:e1008553. [PMID: 33566831 PMCID: PMC7901778 DOI: 10.1371/journal.pcbi.1008553] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2020] [Revised: 02/23/2021] [Accepted: 11/23/2020] [Indexed: 01/10/2023] Open
Abstract
Pavlovian associations drive approach towards reward-predictive cues, and avoidance of punishment-predictive cues. These associations “misbehave” when they conflict with correct instrumental behavior. This raises the question of how Pavlovian and instrumental influences on behavior are arbitrated. We test a computational theory according to which Pavlovian influence will be stronger when inferred controllability of outcomes is low. Using a model-based analysis of a Go/NoGo task with human subjects, we show that theta-band oscillatory power in frontal cortex tracks inferred controllability, and that these inferences predict Pavlovian action biases. Functional MRI data revealed an inferior frontal gyrus correlate of action probability and a ventromedial prefrontal correlate of outcome valence, both of which were modulated by inferred controllability. Using a combination of computational modeling, neuroimaging (both EEG and fMRI), and behavioral analysis, we present evidence for a dual-process architecture in which Pavlovian and instrumental action values are adaptively combined through a Bayesian arbitration mechanism. Building on prior research, we find neural signatures of this arbitration mechanism in frontal cortex. In particular, we show that trial-by-trial changes in Pavlovian influences on action can be predicted by our computational model, and are reflected in midfrontal theta power, as well as inferior frontal and ventromedial prefrontal cortex fMRI responses.
Collapse
Affiliation(s)
- Samuel J Gershman
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, Massachusetts, United States of America.,Center for Brains, Minds and Machines, MIT, Cambridge, Massachusetts, United States of America
| | - Marc Guitart-Masip
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London, United Kingdom.,Aging Research Center, Karolinska Institute, Solna, Sweden
| | - James F Cavanagh
- Department of Psychology, University of New Mexico, Albuquerque, New Mexico, United States of America
| |
Collapse
|
15
|
Ereira S, Pujol M, Guitart-Masip M, Dolan RJ, Kurth-Nelson Z. Overcoming Pavlovian bias in semantic space. Sci Rep 2021; 11:3416. [PMID: 33564034 PMCID: PMC7873193 DOI: 10.1038/s41598-021-82889-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Accepted: 01/25/2021] [Indexed: 01/25/2023] Open
Abstract
Action is invigorated in the presence of reward-predicting stimuli and inhibited in the presence of punishment-predicting stimuli. Although valuable as a heuristic, this Pavlovian bias can also lead to maladaptive behaviour and is implicated in addiction. Here we explore whether Pavlovian bias can be overcome through training. Across five experiments, we find that Pavlovian bias is resistant to unlearning under most task configurations. However, we demonstrate that when subjects engage in instrumental learning in a verbal semantic space, as opposed to a motoric space, not only do they exhibit the typical Pavlovian bias, but this Pavlovian bias diminishes with training. Our results suggest that learning within the semantic space is necessary, but not sufficient, for subjects to unlearn their Pavlovian bias, and that other task features, such as gamification and spaced stimulus presentation may also be necessary. In summary, we show that Pavlovian bias, whilst robust, is susceptible to change with experience, but only under specific environmental conditions.
Collapse
Affiliation(s)
- Sam Ereira
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, UCL, London, WC1B 5EH, UK.
- Wellcome Centre for Human Neuroimaging, UCL, London, WC1N 3BG, UK.
| | - Marine Pujol
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, UCL, London, WC1B 5EH, UK
- Sorbonne Université, Paris, France
| | - Marc Guitart-Masip
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, UCL, London, WC1B 5EH, UK
- Aging Research Centre, Karolinska Institute, 171 65, Stockholm, Sweden
| | - Raymond J Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, UCL, London, WC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, UCL, London, WC1N 3BG, UK
| | - Zeb Kurth-Nelson
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, UCL, London, WC1B 5EH, UK
- DeepMind, London, N1C 4AG, UK
| |
Collapse
|
16
|
Garzón B, Lövdén M, de Boer L, Axelsson J, Riklund K, Bäckman L, Nyberg L, Guitart-Masip M. Role of dopamine and gray matter density in aging effects and individual differences of functional connectomes. Brain Struct Funct 2021; 226:743-758. [PMID: 33423111 PMCID: PMC7981334 DOI: 10.1007/s00429-020-02205-4] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2020] [Accepted: 12/16/2020] [Indexed: 12/01/2022]
Abstract
With increasing age, functional connectomes become dissimilar across normal individuals, reflecting heterogenous aging effects on functional connectivity (FC). We investigated the distribution of these effects across the connectome and their relationship with age-related differences in dopamine (DA) D1 receptor availability and gray matter density (GMD). With this aim, we determined aging effects on mean and interindividual variance of FC using fMRI in 30 younger and 30 older healthy subjects and mapped the contribution of each connection to the patterns of age-related similarity loss. Aging effects on mean FC accounted mainly for the dissimilarity between connectomes of younger and older adults, and were related, across brain regions, to aging effects on DA D1 receptor availability. Aging effects on the variance of FC indicated a global increase in variance with advancing age, explained connectome dissimilarity among older subjects and were related to aging effects on variance of GMD. The relationship between aging and the similarity of connectomes can thus be partly explained by age differences in DA modulation and gray matter structure.
Collapse
Affiliation(s)
- Benjamín Garzón
- Aging Research Center, Karolinska Institutet, Stockholm, Sweden. .,Aging Research Center, Tomtebodavägen 18A, 171 65, Solna, Sweden.
| | - Martin Lövdén
- Aging Research Center, Karolinska Institutet, Stockholm, Sweden
| | - Lieke de Boer
- Aging Research Center, Karolinska Institutet, Stockholm, Sweden
| | - Jan Axelsson
- Department of Radiation Sciences, Umeå University, Umeå, Sweden.,Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden
| | - Katrine Riklund
- Department of Radiation Sciences, Umeå University, Umeå, Sweden.,Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden
| | - Lars Bäckman
- Aging Research Center, Karolinska Institutet, Stockholm, Sweden
| | - Lars Nyberg
- Department of Radiation Sciences, Umeå University, Umeå, Sweden.,Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden.,Department of Integrative Medical Biology, Umeå University, Umeå, Sweden
| | - Marc Guitart-Masip
- Aging Research Center, Karolinska Institutet, Stockholm, Sweden.,Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
| |
Collapse
|
17
|
Cumming P, Gründer G, Brinson Z, Wong DF. Applications, Advances, and Limitations of Molecular Imaging of Brain Receptors. Mol Imaging 2021. [DOI: 10.1016/b978-0-12-816386-3.00063-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022] Open
|
18
|
van Nuland AJ, Helmich RC, Dirkx MF, Zach H, Toni I, Cools R, den Ouden HEM. Effects of dopamine on reinforcement learning in Parkinson's disease depend on motor phenotype. Brain 2020; 143:3422-3434. [PMID: 33147621 PMCID: PMC7719026 DOI: 10.1093/brain/awaa335] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2019] [Revised: 07/10/2020] [Accepted: 08/06/2020] [Indexed: 01/16/2023] Open
Abstract
Parkinson's disease is clinically defined by bradykinesia, along with rigidity and tremor. However, the severity of these motor signs is greatly variable between individuals, particularly the presence or absence of tremor. This variability in tremor relates to variation in cognitive/motivational impairment, as well as the spatial distribution of neurodegeneration in the midbrain and dopamine depletion in the striatum. Here we ask whether interindividual heterogeneity in tremor symptoms could account for the puzzlingly large variability in the effects of dopaminergic medication on reinforcement learning, a fundamental cognitive function known to rely on dopamine. Given that tremor-dominant and non-tremor Parkinson's disease patients have different dopaminergic phenotypes, we hypothesized that effects of dopaminergic medication on reinforcement learning differ between tremor-dominant and non-tremor patients. Forty-three tremor-dominant and 20 non-tremor patients with Parkinson's disease were recruited to be tested both OFF and ON dopaminergic medication (200/50 mg levodopa-benserazide), while 22 age-matched control subjects were recruited to be tested twice OFF medication. Participants performed a reinforcement learning task designed to dissociate effects on learning rate from effects on motivational choice (i.e. the tendency to 'Go/NoGo' in the face of reward/threat of punishment). In non-tremor patients, dopaminergic medication improved reward-based choice, replicating previous studies. In contrast, in tremor-dominant patients, dopaminergic medication improved learning from punishment. Formal modelling showed divergent computational effects of dopaminergic medication as a function of Parkinson's disease motor phenotype, with a modulation of motivational choice bias and learning rate in non-tremor and tremor patients, respectively. This finding establishes a novel cognitive/motivational difference between tremor and non-tremor Parkinson's disease patients, and highlights the importance of considering motor phenotype in future work.
Collapse
Affiliation(s)
- Annelies J van Nuland
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, 6500 HB Nijmegen, The Netherlands
| | - Rick C Helmich
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, 6500 HB Nijmegen, The Netherlands
- Radboud University Medical Centre, Donders Institute for Brain, Cognition and Behaviour, Department of Neurology, 6500 HB Nijmegen, The Netherlands
| | - Michiel F Dirkx
- Radboud University Medical Centre, Donders Institute for Brain, Cognition and Behaviour, Department of Neurology, 6500 HB Nijmegen, The Netherlands
| | - Heidemarie Zach
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, 6500 HB Nijmegen, The Netherlands
- Radboud University Medical Centre, Donders Institute for Brain, Cognition and Behaviour, Department of Neurology, 6500 HB Nijmegen, The Netherlands
- Department of Neurology, Medical University Vienna, Vienna, Austria
| | - Ivan Toni
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, 6500 HB Nijmegen, The Netherlands
| | - Roshan Cools
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, 6500 HB Nijmegen, The Netherlands
- Radboud University Medical Center, Donders Institute for Brain, Cognition and Behaviour, Department of Psychiatry, Nijmegen, The Netherlands
| | - Hanneke E M den Ouden
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, 6500 HB Nijmegen, The Netherlands
| |
Collapse
|
19
|
Betts MJ, Richter A, de Boer L, Tegelbeckers J, Perosa V, Baumann V, Chowdhury R, Dolan RJ, Seidenbecher C, Schott BH, Düzel E, Guitart-Masip M, Krauel K. Learning in anticipation of reward and punishment: perspectives across the human lifespan. Neurobiol Aging 2020; 96:49-57. [DOI: 10.1016/j.neurobiolaging.2020.08.011] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2020] [Revised: 07/13/2020] [Accepted: 08/19/2020] [Indexed: 12/13/2022]
|
20
|
Raab HA, Hartley CA. Adolescents exhibit reduced Pavlovian biases on instrumental learning. Sci Rep 2020; 10:15770. [PMID: 32978451 PMCID: PMC7519144 DOI: 10.1038/s41598-020-72628-w] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2020] [Accepted: 09/03/2020] [Indexed: 02/07/2023] Open
Abstract
Multiple learning systems allow individuals to flexibly respond to opportunities and challenges present in the environment. An evolutionarily conserved "Pavlovian" learning mechanism couples valence and action, promoting a tendency to approach cues associated with reward and to inhibit action in the face of anticipated punishment. Although this default response system may be adaptive, these hard-wired reactions can hinder the ability to learn flexible "instrumental" actions in pursuit of a goal. Such constraints on behavioral flexibility have been studied extensively in adults. However, the extent to which these valence-specific response tendencies bias instrumental learning across development remains poorly characterized. Here, we show that while Pavlovian response biases constrain flexible action learning in children and adults, these biases are attenuated in adolescents. This adolescent-specific reduction in Pavlovian bias may promote unbiased exploration of approach and avoidance responses, facilitating the discovery of rewarding behavior in the many novel contexts that adolescents encounter.
Collapse
Affiliation(s)
- Hillary A Raab
- Department of Psychology, New York University, New York, NY, USA
| | - Catherine A Hartley
- Department of Psychology, New York University, New York, NY, USA.
- Center for Neural Science, New York University, New York, NY, USA.
| |
Collapse
|
21
|
Adams RA, Moutoussis M, Nour MM, Dahoun T, Lewis D, Illingworth B, Veronese M, Mathys C, de Boer L, Guitart-Masip M, Friston KJ, Howes OD, Roiser JP. Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models. Cereb Cortex 2020; 30:3573-3589. [PMID: 32083297 PMCID: PMC7233027 DOI: 10.1093/cercor/bhz327] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 11/18/2019] [Accepted: 12/05/2019] [Indexed: 12/17/2022] Open
Abstract
Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans.
Collapse
Affiliation(s)
- Rick A Adams
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
- Division of Psychiatry, University College London, London W1T 7NF, UK
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
| | - Michael Moutoussis
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
| | - Matthew M Nour
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King’s College London, London SE5 8AF, UK
| | - Tarik Dahoun
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford OX3 7JX, UK
| | - Declan Lewis
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| | - Benjamin Illingworth
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| | - Mattia Veronese
- Centre for Neuroimaging Sciences, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King's College London, London SE5 8AF, UK
| | - Christoph Mathys
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
- Scuola Internazionale Superiore di Studi Avanzati (SISSA), 34136 Trieste, Italy
- Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich and ETH Zurich, 8032 Zurich, Switzerland
| | - Lieke de Boer
- Aging Research Center, Karolinska Institute, 171 65 Stockholm, Sweden
| | - Marc Guitart-Masip
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
- Aging Research Center, Karolinska Institute, 171 65 Stockholm, Sweden
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK
| | - Oliver D Howes
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King’s College London, London SE5 8AF, UK
| | - Jonathan P Roiser
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| |
Collapse
|
22
|
Soutschek A, Kozak R, de Martinis N, Howe W, Burke CJ, Fehr E, Jetter A, Tobler PN. Activation of D1 receptors affects human reactivity and flexibility to valued cues. Neuropsychopharmacology 2020; 45:780-785. [PMID: 31962344 PMCID: PMC7075935 DOI: 10.1038/s41386-020-0617-z] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/13/2019] [Revised: 12/02/2019] [Accepted: 01/09/2020] [Indexed: 12/13/2022]
Abstract
Reward-predicting cues motivate goal-directed behavior, but in unstable environments humans must also be able to flexibly update cue-reward associations. While the capacity of reward cues to trigger motivation ('reactivity') as well as flexibility in cue-reward associations have been linked to the neurotransmitter dopamine in humans, the specific contribution of the dopamine D1 receptor family to these behaviors remained elusive. To fill this gap, we conducted a randomized, placebo-controlled, double-blind pharmacological study testing the impact of three different doses of a novel D1 agonist (relative to placebo) on reactivity to reward-predicting cues (Pavlovian-to-instrumental transfer) and flexibility of cue-outcome associations (reversal learning). We observed that the impact of the D1 agonist crucially depended on baseline working memory functioning, which has been identified as a proxy for baseline dopamine synthesis capacity. Specifically, increasing D1 receptor stimulation strengthened Pavlovian-to-instrumental transfer in individuals with high baseline working memory capacity. In contrast, higher doses of the D1 agonist improved reversal learning only in individuals with low baseline working memory functioning. Our findings suggest a crucial and baseline-dependent role of D1 receptor activation in controlling both cue reactivity and the flexibility of cue-reward associations.
Collapse
Affiliation(s)
- Alexander Soutschek
- Department of Psychology, Ludwig Maximilian University Munich, Munich, Germany. .,Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zurich, Switzerland.
| | - Rouba Kozak
- 0000 0004 0447 7762grid.419849.9Takeda Pharmaceuticals International, Cambridge, MA USA
| | | | - William Howe
- 0000 0001 0694 4940grid.438526.eSchool of Neuroscience, Virginia Tech, Blacksburg, VA USA
| | - Christopher J. Burke
- 0000 0004 1937 0650grid.7400.3Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zurich, Switzerland
| | - Ernst Fehr
- 0000 0004 1937 0650grid.7400.3Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zurich, Switzerland ,Neuroscience Center Zurich, University of Zurich, Swiss Federal Institute of Technology Zurich, Zurich, Switzerland
| | - Alexander Jetter
- Department of Clinical Pharmacology and Toxicology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Philippe N. Tobler
- 0000 0004 1937 0650grid.7400.3Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zurich, Switzerland ,Neuroscience Center Zurich, University of Zurich, Swiss Federal Institute of Technology Zurich, Zurich, Switzerland
| |
Collapse
|
23
|
Olsson A, Knapska E, Lindström B. The neural and computational systems of social learning. Nat Rev Neurosci 2020; 21:197-212. [PMID: 32221497 DOI: 10.1038/s41583-020-0276-4] [Citation(s) in RCA: 119] [Impact Index Per Article: 29.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/07/2020] [Indexed: 01/10/2023]
Abstract
Learning the value of stimuli and actions from others - social learning - adaptively contributes to individual survival and plays a key role in cultural evolution. We review research across species targeting the neural and computational systems of social learning in both the aversive and appetitive domains. Social learning generally follows the same principles as self-experienced value-based learning, including computations of prediction errors and is implemented in brain circuits activated across task domains together with regions processing social information. We integrate neural and computational perspectives of social learning with an understanding of behaviour of varying complexity, from basic threat avoidance to complex social learning strategies and cultural phenomena.
Collapse
Affiliation(s)
- Andreas Olsson
- Department of Clinical Neuroscience, Division of Psychology, Karolinska Institutet, Solna, Sweden.
| | - Ewelina Knapska
- Laboratory of Emotions' Neurobiology, Centre of Excellence for Neural Plasticity and Brain Disorders (BRAINCITY), Nencki Institute of Experimental Biology, Polish Academy of Sciences, Warsaw, Poland
| | - Björn Lindström
- Department of Clinical Neuroscience, Division of Psychology, Karolinska Institutet, Solna, Sweden.,Department of Psychology, University of Amsterdam, Amsterdam, Netherlands
| |
Collapse
|
24
|
Silva PA, Trigo S, Marques CI, Cardoso GC, Soares MC. Experimental evidence for a role of dopamine in avian personality traits. J Exp Biol 2020; 223:jeb216499. [PMID: 31953366 DOI: 10.1242/jeb.216499] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2019] [Accepted: 01/10/2020] [Indexed: 11/20/2022]
Abstract
There is increasing interest in the genetic and physiological bases of behavioural differences among individuals, namely animal personality. One particular dopamine (DA) receptor gene (the dopamine receptor D4 gene) has been used as candidate gene to explain personality differences, but with mixed results. Here, we used an alternative approach, exogenously manipulating the dopaminergic system and testing for effects on personality assays in a social bird species, the common waxbill (Estrilda astrild). We treated birds with agonists and antagonists for DA receptors of both D1 and D2 receptor pathways (the latter includes the D4 receptor) and found that short-term manipulation of DA signalling had an immediate effect on personality-related behaviours. In an assay of social responses (mirror test), manipulation of D2 receptor pathways reduced time spent looking at the social stimulus (mirror image). Blocking D2 receptors reduced motor activity in this social assay, while treatment with a D2 receptor agonist augmented activity in this social assay but reduced activity in a non-social behavioural assay. Also, in the non-social assay, treatment with the D1 receptor antagonist markedly increased time spent at the feeder. These results show distinct and context-specific effects of the dopaminergic pathways on waxbill personality traits. Our results also suggest that experimental manipulation of DA signalling can disrupt a behavioural correlation (more active individuals being less attentive to mirror image) that is habitually observed as part of a behavioural syndrome in waxbills. We discuss our results in the context of animal personality, and the role of the DA system in reward and social behaviour.
Collapse
Affiliation(s)
- Paulo A Silva
- CIBIO-InBIO, Centro de Investigação em Biodiversidade e Recursos Genéticos da Universidade do Porto, Vairão 4485-661, Portugal
| | - Sandra Trigo
- CIBIO-InBIO, Centro de Investigação em Biodiversidade e Recursos Genéticos da Universidade do Porto, Vairão 4485-661, Portugal
| | - Cristiana I Marques
- CIBIO-InBIO, Centro de Investigação em Biodiversidade e Recursos Genéticos da Universidade do Porto, Vairão 4485-661, Portugal
| | - Gonçalo C Cardoso
- CIBIO-InBIO, Centro de Investigação em Biodiversidade e Recursos Genéticos da Universidade do Porto, Vairão 4485-661, Portugal
- Behavioural Ecology Group, Department of Biology, University of Copenhagen, 2100 Copenhagen Ø, Denmark
| | - Marta C Soares
- CIBIO-InBIO, Centro de Investigação em Biodiversidade e Recursos Genéticos da Universidade do Porto, Vairão 4485-661, Portugal
| |
Collapse
|
25
|
Perosa V, de Boer L, Ziegler G, Apostolova I, Buchert R, Metzger C, Amthauer H, Guitart-Masip M, Düzel E, Betts MJ. The Role of the Striatum in Learning to Orthogonalize Action and Valence: A Combined PET and 7 T MRI Aging Study. Cereb Cortex 2020; 30:3340-3351. [PMID: 31897476 DOI: 10.1093/cercor/bhz313] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022] Open
Abstract
Pavlovian biases influence instrumental learning by coupling reward seeking with action invigoration and punishment avoidance with action suppression. Using a probabilistic go/no-go task designed to orthogonalize action (go/no-go) and valence (reward/punishment), recent studies have shown that the interaction between the two is dependent on the striatum and its key neuromodulator dopamine. Using this task, we sought to identify how structural and neuromodulatory age-related differences in the striatum may influence Pavlovian biases and instrumental learning in 25 young and 31 older adults. Computational modeling revealed a significant age-related reduction in reward and punishment sensitivity and marked (albeit not significant) reduction in learning rate and lapse rate (irreducible noise). Voxel-based morphometry analysis using 7 Tesla MRI images showed that individual differences in learning rate in older adults were related to the volume of the caudate nucleus. In contrast, dopamine synthesis capacity in the dorsal striatum, assessed using [18F]-DOPA positron emission tomography in 22 of these older adults, was not associated with learning performance and did not moderate the relationship between caudate volume and learning rate. This multiparametric approach suggests that age-related differences in striatal volume may influence learning proficiency in old age.
Collapse
Affiliation(s)
- Valentina Perosa
- Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke-University Magdeburg, Leipzigerstr. 44, 39120, Magdeburg, Germany.,German Center for Neurodegenerative Diseases (DZNE), Leipzigerstr. 44 39120, Magdeburg, Germany.,Department of Neurology, Otto-von-Guericke University, Leipzigerstr. 44, 39120, Magdeburg, Germany
| | - Lieke de Boer
- Ageing Research Centre, Karolinska Institute, SE-11330 Stockholm, Sweden
| | - Gabriel Ziegler
- Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke-University Magdeburg, Leipzigerstr. 44, 39120, Magdeburg, Germany.,German Center for Neurodegenerative Diseases (DZNE), Leipzigerstr. 44 39120, Magdeburg, Germany
| | - Ivayla Apostolova
- Department of Radiology and Nuclear Medicine, University Hospital Hamburg-Eppendorf, Germany
| | - Ralph Buchert
- Department of Radiology and Nuclear Medicine, University Hospital Hamburg-Eppendorf, Germany.,Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, UK
| | - Coraline Metzger
- Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke-University Magdeburg, Leipzigerstr. 44, 39120, Magdeburg, Germany.,German Center for Neurodegenerative Diseases (DZNE), Leipzigerstr. 44 39120, Magdeburg, Germany
| | - Holger Amthauer
- Department of Nuclear Medicine, Charité-Universitätsmedizin Berlin, Augustenburger Platz 1, 13353 Berlin, Germany
| | - Marc Guitart-Masip
- Ageing Research Centre, Karolinska Institute, SE-11330 Stockholm, Sweden.,Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, UK
| | - Emrah Düzel
- Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke-University Magdeburg, Leipzigerstr. 44, 39120, Magdeburg, Germany.,German Center for Neurodegenerative Diseases (DZNE), Leipzigerstr. 44 39120, Magdeburg, Germany.,Institute of Cognitive Neuroscience, University College London, 17 Queen Square, London, UK
| | - Matthew J Betts
- Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke-University Magdeburg, Leipzigerstr. 44, 39120, Magdeburg, Germany.,German Center for Neurodegenerative Diseases (DZNE), Leipzigerstr. 44 39120, Magdeburg, Germany
| |
Collapse
|