1
|
Vahedi J, Mundorf A, Bellebaum C, Peterburs J. Emotional cues reduce Pavlovian interference in feedback-based go and nogo learning. PSYCHOLOGICAL RESEARCH 2024; 88:1212-1230. [PMID: 38483574 PMCID: PMC11142951 DOI: 10.1007/s00426-024-01946-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Accepted: 02/26/2024] [Indexed: 06/02/2024]
Abstract
It is easier to execute a response in the promise of a reward and withhold a response in the promise of a punishment than vice versa, due to a conflict between cue-related Pavlovian and outcome-related instrumental action tendencies in the reverse conditions. This robust learning asymmetry in go and nogo learning is referred to as the Pavlovian bias. Interestingly, it is similar to motivational tendencies reported for affective facial expressions, i.e., facilitation of approach to a smile and withdrawal from a frown. The present study investigated whether and how learning from emotional faces instead of abstract stimuli modulates the Pavlovian bias in reinforcement learning. To this end, 137 healthy adult participants performed an orthogonalized Go/Nogo task that fully decoupled action (go/nogo) and outcome valence (win points/avoid losing points). Three groups of participants were tested with either emotional facial cues whose affective valence was either congruent (CON) or incongruent (INC) to the required instrumental response, or with neutral facial cues (NEU). Relative to NEU, the Pavlovian bias was reduced in both CON and INC, though still present under all learning conditions. Importantly, only for CON, the reduction of the Pavlovian bias effect was adaptive by improving learning performance in one of the conflict conditions. In contrast, the reduction of the Pavlovian bias in INC was completely driven by decreased learning performance in non-conflict conditions. These results suggest a potential role of arousal/salience in Pavlovian-instrumental regulation and cue-action congruency in the adaptability of goal-directed behavior. Implications for clinical application are discussed.
Collapse
Affiliation(s)
- Julian Vahedi
- Faculty of Mathematics and Natural Sciences, Heinrich Heine University Düsseldorf, Düsseldorf, Germany.
| | - Annakarina Mundorf
- Institute for Systems Medicine, Department of Human Medicine, MSH Medical School, Hamburg, Germany
| | - Christian Bellebaum
- Faculty of Mathematics and Natural Sciences, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Jutta Peterburs
- Institute for Systems Medicine, Department of Human Medicine, MSH Medical School, Hamburg, Germany
| |
Collapse
|
2
|
Park H, Doh H, Lee E, Park H, Ahn WY. The neurocognitive role of working memory load when Pavlovian motivational control affects instrumental learning. PLoS Comput Biol 2023; 19:e1011692. [PMID: 38064498 PMCID: PMC10732416 DOI: 10.1371/journal.pcbi.1011692] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Revised: 12/20/2023] [Accepted: 11/15/2023] [Indexed: 12/21/2023] Open
Abstract
Research suggests that a fast, capacity-limited working memory (WM) system and a slow, incremental reinforcement learning (RL) system jointly contribute to instrumental learning. Thus, situations that strain WM resources alter instrumental learning: under WM loads, learning becomes slow and incremental, the reliance on computationally efficient learning increases, and action selection becomes more random. It is also suggested that Pavlovian learning influences people's behavior during instrumental learning by providing hard-wired instinctive responses including approach to reward predictors and avoidance of punishment predictors. However, it remains unknown how constraints on WM resources affect instrumental learning under Pavlovian influence. Thus, we conducted a functional magnetic resonance imaging (fMRI) study (N = 49) in which participants completed an instrumental learning task with Pavlovian-instrumental conflict (the orthogonalized go/no-go task) both with and without extra WM load. Behavioral and computational modeling analyses revealed that WM load reduced the learning rate and increased random choice, without affecting Pavlovian bias. Model-based fMRI analysis revealed that WM load strengthened RPE signaling in the striatum. Moreover, under WM load, the striatum showed weakened connectivity with the ventromedial and dorsolateral prefrontal cortex when computing reward expectations. These results suggest that the limitation of cognitive resources by WM load promotes slow and incremental learning through the weakened cooperation between WM and RL; such limitation also makes action selection more random, but it does not directly affect the balance between instrumental and Pavlovian systems.
Collapse
Affiliation(s)
- Heesun Park
- Department of Psychology, Seoul National University, Seoul, Korea
| | - Hoyoung Doh
- Department of Psychology, Seoul National University, Seoul, Korea
| | - Eunhwi Lee
- Department of Psychology, Seoul National University, Seoul, Korea
| | - Harhim Park
- Department of Psychology, Seoul National University, Seoul, Korea
| | - Woo-Young Ahn
- Department of Psychology, Seoul National University, Seoul, Korea
- Department of Brain and Cognitive Sciences, Seoul National University, Seoul, Korea
| |
Collapse
|
3
|
Stolz C, Pickering AD, Mueller EM. Dissociable feedback valence effects on frontal midline theta during reward gain versus threat avoidance learning. Psychophysiology 2022; 60:e14235. [PMID: 36529988 DOI: 10.1111/psyp.14235] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Revised: 10/17/2022] [Accepted: 11/17/2022] [Indexed: 12/23/2022]
Abstract
While frontal midline theta (FMθ) has been associated with threat processing, with cognitive control in the context of anxiety, and with reinforcement learning, most reinforcement learning studies on FMθ have used reward rather than threat-related stimuli as reinforcer. Accordingly, the role of FMθ in threat-related reinforcement learning is largely unknown. Here, n = 23 human participants underwent one reward-, and one punishment-, based reversal learning task, which differed only with regard to the kind of reinforcers that feedback was tied to (i.e., monetary gain vs. loud noise burst, respectively). In addition to single-trial EEG, we assessed single-trial feedback expectations based on both a reinforcement learning computational model and trial-by-trial subjective feedback expectation ratings. While participants' performance and feedback expectations were comparable between the reward and punishment tasks, FMθ was more reliably amplified to negative vs. positive feedback in the reward vs. punishment task. Regressions with feedback valence, computationally derived, and self-reported expectations as predictors and FMθ as criterion further revealed that trial-by-trial variations in FMθ specifically relate to reward-related feedback-valence and not to threat-related feedback or to violated expectations/prediction errors. These findings suggest that FMθ as measured in reinforcement learning tasks may be less sensitive to the processing of events with direct relevance for fear and anxiety.
Collapse
Affiliation(s)
- Christopher Stolz
- Department of Psychology University of Marburg Marburg Germany
- Leibniz Institute for Neurobiology (LIN) Magdeburg Germany
- Department of Psychology Goldsmiths, University of London London UK
| | | | - Erik M. Mueller
- Department of Psychology University of Marburg Marburg Germany
| |
Collapse
|
4
|
Richter A, de Boer L, Guitart-Masip M, Behnisch G, Seidenbecher CI, Schott BH. Motivational learning biases are differentially modulated by genetic determinants of striatal and prefrontal dopamine function. J Neural Transm (Vienna) 2021; 128:1705-1720. [PMID: 34302222 PMCID: PMC8536632 DOI: 10.1007/s00702-021-02382-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Accepted: 07/04/2021] [Indexed: 01/20/2023]
Abstract
Dopaminergic neurotransmission plays a pivotal role in appetitively motivated behavior in mammals, including humans. Notably, action and valence are not independent in motivated tasks, and it is particularly difficult for humans to learn the inhibition of an action to obtain a reward. We have previously observed that the carriers of the DRD2/ANKK1 TaqIA A1 allele, that has been associated with reduced striatal dopamine D2 receptor expression, showed a diminished learning performance when required to learn response inhibition to obtain rewards, a finding that was replicated in two independent cohorts. With our present study, we followed two aims: first, we aimed to replicate our finding on the DRD2/ANKK1 TaqIA polymorphism in a third independent cohort (N = 99) and to investigate the nature of the genetic effects more closely using trial-by-trial behavioral analysis and computational modeling in the combined dataset (N = 281). Second, we aimed to assess a potentially modulatory role of prefrontal dopamine availability, using the widely studied COMT Val108/158Met polymorphism as a proxy. We first report a replication of the above mentioned finding. Interestingly, after combining all three cohorts, exploratory analyses regarding the COMT Val108/158Met polymorphism suggest that homozygotes for the Met allele, which has been linked to higher prefrontal dopaminergic tone, show a lower learning bias. Our results corroborate the importance of genetic variability of the dopaminergic system in individual learning differences of action-valence interaction and, furthermore, suggest that motivational learning biases are differentially modulated by genetic determinants of striatal and prefrontal dopamine function.
Collapse
Affiliation(s)
- Anni Richter
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118, Magdeburg, Germany.
| | - Lieke de Boer
- Ageing Research Centre, Karolinska Institute, Stockholm, Sweden
- Max Planck Institute for Human Development, Center for Lifespan Psychology, Berlin, Germany
| | - Marc Guitart-Masip
- Ageing Research Centre, Karolinska Institute, Stockholm, Sweden
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
| | - Gusalija Behnisch
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118, Magdeburg, Germany
| | - Constanze I Seidenbecher
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118, Magdeburg, Germany
- Center for Behavioral Brain Sciences, Magdeburg, Germany
| | - Björn H Schott
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118, Magdeburg, Germany
- Center for Behavioral Brain Sciences, Magdeburg, Germany
- Department of Psychiatry and Psychotherapy, University Medicine Göttingen, Göttingen, Germany
- Department of Neurology, University of Magdeburg, Magdeburg, Germany
- German Center for Neurodegenerative Diseases (DZNE), Göttingen, Germany
| |
Collapse
|
5
|
Byrne A, Hewitt D, Henderson J, Newton-Fenner A, Roberts H, Tyson-Carr J, Fallon N, Giesbrecht T, Stancak A. Investigating the effect of losses and gains on effortful engagement during an incentivized Go/NoGo task through anticipatory cortical oscillatory changes. Psychophysiology 2021; 59:e13897. [PMID: 34251684 DOI: 10.1111/psyp.13897] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2021] [Revised: 05/02/2021] [Accepted: 06/09/2021] [Indexed: 11/29/2022]
Abstract
Losses usually have greater subjective value (SV) than gains of equal nominal value but often cause a relative deterioration in effortful performance. Since losses and gains induce differing approach/avoidance behavioral tendencies, we explored whether incentive type interacted with approach/avoidance motor-sets. Alpha- and beta-band event-related desynchronization (ERD) was hypothesized to be weakest when participants expected a loss and prepared an inhibitory motor-set, and strongest when participants expected a gain and prepared an active motor-set. It was also hypothesized that effort would modulate reward and motor-set-related cortical activation patterns. Participants completed a cued Go/NoGo task while expecting a reward (+10p), avoiding a loss (-10p), or receiving no incentive (0p); and while expecting a NoGo cue with a probability of either .75 or .25. Pre-movement alpha- and beta-band EEG power was analyzed using the ERD method, and the SV of effort was evaluated using a cognitive effort discounting task. Gains incentivized faster RTs and stronger preparatory alpha band ERD compared to loss and no incentive conditions, while inhibitory motor-sets resulted in significantly weaker alpha-band ERD. However, there was no interaction between incentive and motor-sets. Participants were more willing to expend effort in losses compared to gain trials, although the SV of effort was not associated with ERD patterns or RTs. Results suggest that incentive and approach/avoidance motor tendencies modulate cortical activations prior to a speeded RT movement independently, and are not associated with the economic value of effort. The present results favor attentional explanations of the effect of incentive modality on effort.
Collapse
Affiliation(s)
- Adam Byrne
- Department of Psychological Sciences, University of Liverpool, Liverpool, UK.,Institute for Risk and Uncertainty, University of Liverpool, Liverpool, UK
| | - Danielle Hewitt
- Department of Psychological Sciences, University of Liverpool, Liverpool, UK
| | - Jessica Henderson
- Department of Psychological Sciences, University of Liverpool, Liverpool, UK
| | - Alice Newton-Fenner
- Department of Psychological Sciences, University of Liverpool, Liverpool, UK.,Institute for Risk and Uncertainty, University of Liverpool, Liverpool, UK
| | - Hannah Roberts
- Department of Psychological Sciences, University of Liverpool, Liverpool, UK
| | - John Tyson-Carr
- Department of Psychological Sciences, University of Liverpool, Liverpool, UK
| | - Nick Fallon
- Department of Psychological Sciences, University of Liverpool, Liverpool, UK
| | | | - Andrej Stancak
- Department of Psychological Sciences, University of Liverpool, Liverpool, UK.,Institute for Risk and Uncertainty, University of Liverpool, Liverpool, UK
| |
Collapse
|
6
|
Ereira S, Pujol M, Guitart-Masip M, Dolan RJ, Kurth-Nelson Z. Overcoming Pavlovian bias in semantic space. Sci Rep 2021; 11:3416. [PMID: 33564034 PMCID: PMC7873193 DOI: 10.1038/s41598-021-82889-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Accepted: 01/25/2021] [Indexed: 01/25/2023] Open
Abstract
Action is invigorated in the presence of reward-predicting stimuli and inhibited in the presence of punishment-predicting stimuli. Although valuable as a heuristic, this Pavlovian bias can also lead to maladaptive behaviour and is implicated in addiction. Here we explore whether Pavlovian bias can be overcome through training. Across five experiments, we find that Pavlovian bias is resistant to unlearning under most task configurations. However, we demonstrate that when subjects engage in instrumental learning in a verbal semantic space, as opposed to a motoric space, not only do they exhibit the typical Pavlovian bias, but this Pavlovian bias diminishes with training. Our results suggest that learning within the semantic space is necessary, but not sufficient, for subjects to unlearn their Pavlovian bias, and that other task features, such as gamification and spaced stimulus presentation may also be necessary. In summary, we show that Pavlovian bias, whilst robust, is susceptible to change with experience, but only under specific environmental conditions.
Collapse
Affiliation(s)
- Sam Ereira
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, UCL, London, WC1B 5EH, UK.
- Wellcome Centre for Human Neuroimaging, UCL, London, WC1N 3BG, UK.
| | - Marine Pujol
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, UCL, London, WC1B 5EH, UK
- Sorbonne Université, Paris, France
| | - Marc Guitart-Masip
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, UCL, London, WC1B 5EH, UK
- Aging Research Centre, Karolinska Institute, 171 65, Stockholm, Sweden
| | - Raymond J Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, UCL, London, WC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, UCL, London, WC1N 3BG, UK
| | - Zeb Kurth-Nelson
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, UCL, London, WC1B 5EH, UK
- DeepMind, London, N1C 4AG, UK
| |
Collapse
|
7
|
Betts MJ, Richter A, de Boer L, Tegelbeckers J, Perosa V, Baumann V, Chowdhury R, Dolan RJ, Seidenbecher C, Schott BH, Düzel E, Guitart-Masip M, Krauel K. Learning in anticipation of reward and punishment: perspectives across the human lifespan. Neurobiol Aging 2020; 96:49-57. [DOI: 10.1016/j.neurobiolaging.2020.08.011] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2020] [Revised: 07/13/2020] [Accepted: 08/19/2020] [Indexed: 12/13/2022]
|
8
|
Perosa V, de Boer L, Ziegler G, Apostolova I, Buchert R, Metzger C, Amthauer H, Guitart-Masip M, Düzel E, Betts MJ. The Role of the Striatum in Learning to Orthogonalize Action and Valence: A Combined PET and 7 T MRI Aging Study. Cereb Cortex 2020; 30:3340-3351. [PMID: 31897476 DOI: 10.1093/cercor/bhz313] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022] Open
Abstract
Pavlovian biases influence instrumental learning by coupling reward seeking with action invigoration and punishment avoidance with action suppression. Using a probabilistic go/no-go task designed to orthogonalize action (go/no-go) and valence (reward/punishment), recent studies have shown that the interaction between the two is dependent on the striatum and its key neuromodulator dopamine. Using this task, we sought to identify how structural and neuromodulatory age-related differences in the striatum may influence Pavlovian biases and instrumental learning in 25 young and 31 older adults. Computational modeling revealed a significant age-related reduction in reward and punishment sensitivity and marked (albeit not significant) reduction in learning rate and lapse rate (irreducible noise). Voxel-based morphometry analysis using 7 Tesla MRI images showed that individual differences in learning rate in older adults were related to the volume of the caudate nucleus. In contrast, dopamine synthesis capacity in the dorsal striatum, assessed using [18F]-DOPA positron emission tomography in 22 of these older adults, was not associated with learning performance and did not moderate the relationship between caudate volume and learning rate. This multiparametric approach suggests that age-related differences in striatal volume may influence learning proficiency in old age.
Collapse
Affiliation(s)
- Valentina Perosa
- Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke-University Magdeburg, Leipzigerstr. 44, 39120, Magdeburg, Germany.,German Center for Neurodegenerative Diseases (DZNE), Leipzigerstr. 44 39120, Magdeburg, Germany.,Department of Neurology, Otto-von-Guericke University, Leipzigerstr. 44, 39120, Magdeburg, Germany
| | - Lieke de Boer
- Ageing Research Centre, Karolinska Institute, SE-11330 Stockholm, Sweden
| | - Gabriel Ziegler
- Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke-University Magdeburg, Leipzigerstr. 44, 39120, Magdeburg, Germany.,German Center for Neurodegenerative Diseases (DZNE), Leipzigerstr. 44 39120, Magdeburg, Germany
| | - Ivayla Apostolova
- Department of Radiology and Nuclear Medicine, University Hospital Hamburg-Eppendorf, Germany
| | - Ralph Buchert
- Department of Radiology and Nuclear Medicine, University Hospital Hamburg-Eppendorf, Germany.,Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, UK
| | - Coraline Metzger
- Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke-University Magdeburg, Leipzigerstr. 44, 39120, Magdeburg, Germany.,German Center for Neurodegenerative Diseases (DZNE), Leipzigerstr. 44 39120, Magdeburg, Germany
| | - Holger Amthauer
- Department of Nuclear Medicine, Charité-Universitätsmedizin Berlin, Augustenburger Platz 1, 13353 Berlin, Germany
| | - Marc Guitart-Masip
- Ageing Research Centre, Karolinska Institute, SE-11330 Stockholm, Sweden.,Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, UK
| | - Emrah Düzel
- Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke-University Magdeburg, Leipzigerstr. 44, 39120, Magdeburg, Germany.,German Center for Neurodegenerative Diseases (DZNE), Leipzigerstr. 44 39120, Magdeburg, Germany.,Institute of Cognitive Neuroscience, University College London, 17 Queen Square, London, UK
| | - Matthew J Betts
- Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke-University Magdeburg, Leipzigerstr. 44, 39120, Magdeburg, Germany.,German Center for Neurodegenerative Diseases (DZNE), Leipzigerstr. 44 39120, Magdeburg, Germany
| |
Collapse
|
9
|
Hoofs V, Boehler CN, Krebs RM. Biasing Actions by Incentive Valence in an Approach/Avoidance Task. COLLABRA-PSYCHOLOGY 2019. [DOI: 10.1525/collabra.205] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/29/2023]
Abstract
The present study investigates interactions between incentive valence and action, which mirror well-known valence-action biases in the emotional domain. In three joystick experiments, incentive valence (win/loss) and action type (approach/avoid) were signaled by distinct orthogonal stimulus features. By combining several design aspects, i.e., the use of bi-directional joystick movements, the inclusion of no-incentive baseline trials, and cue-locked versus target-locked valence and action signals, we tried to bridge between paradigms used in the emotional and motivational domain, and to understand previous, partly inconsistent results. In the first task variant (Experiment 1), we observed performance benefits for compatible mappings (win-approach; loss-avoid) relative to incompatible ones (loss-approach; win-avoid) when valence and action signals were target-locked, consistent with a fairly automatic response activation that can benefit or impair task performance. In contrast, cue-locked valence signals led to response facilitation (relative to a no-incentive baseline) more or less independent of actual valence (win/loss) and action type (approach/avoid), which is reminiscent of general facilitation effects of incentive cues across diverse cognitive tasks. Slight design variations did not change this main result pattern, indicating that it was neither driven by the close proximity between target and performance feedback (Experiment 2), nor by mere temporal coincidence of valence and action signals (Experiment 3), but rather by differences between preparatory (cued) and immediate (non-cued) effects of incentive valence. The present study provides novel insights regarding the nature of valence-action biases in the motivational domain and helps to integrate previous, partly inconsistent findings across domains.
Collapse
Affiliation(s)
- Vincent Hoofs
- Department of Experimental Psychology, Ghent University, Ghent, BE
| | - C. Nico Boehler
- Department of Experimental Psychology, Ghent University, Ghent, BE
| | - Ruth M. Krebs
- Department of Experimental Psychology, Ghent University, Ghent, BE
| |
Collapse
|
10
|
Dorsal striatal dopamine D1 receptor availability predicts an instrumental bias in action learning. Proc Natl Acad Sci U S A 2018; 116:261-270. [PMID: 30563856 PMCID: PMC6320523 DOI: 10.1073/pnas.1816704116] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/01/2023] Open
Abstract
The brain’s dopaminergic pathways are crucially important for adaptive behavior. They are thought to enable us to approach rewards and stay away from punishments. During learning, dopaminergic reward prediction errors are thought to reinforce previously rewarded actions, so they become easier to repeat. This dopaminergic activity could lead to a systematic bias by which rewarded actions are more readily learned than rewarded inactions. We present two findings. First, dopamine receptors in cortex, dorsal striatum, and nucleus accumbens provide distinct sources of variance in the human brain. Second, the boost in an individual’s learning rate from previously rewarded actions is dependent on the dopamine receptor density in dorsal striatum, a central structure in the dopaminergic circuit. Learning to act to obtain reward and inhibit to avoid punishment is easier compared with learning the opposite contingencies. This coupling of action and valence is often thought of as a Pavlovian bias, although recent research has shown it may also emerge through instrumental mechanisms. We measured this learning bias with a rewarded go/no-go task in 60 adults of different ages. Using computational modeling, we characterized the bias as being instrumental. To assess the role of endogenous dopamine (DA) in the expression of this bias, we quantified DA D1 receptor availability using positron emission tomography (PET) with the radioligand [11C]SCH23390. Using principal-component analysis on the binding potentials in a number of cortical and striatal regions of interest, we demonstrated that cortical, dorsal striatal, and ventral striatal areas provide independent sources of variance in DA D1 receptor availability. Interindividual variation in the dorsal striatal component was related to the strength of the instrumental bias during learning. These data suggest at least three anatomical sources of variance in DA D1 receptor availability separable using PET in humans, and we provide evidence that human dorsal striatal DA D1 receptors are involved in the modulation of instrumental learning biases.
Collapse
|
11
|
Hoofs V, Carsten T, Boehler CN, Krebs RM. Interactions between incentive valence and action information in a cued approach–avoidance task. PSYCHOLOGICAL RESEARCH 2018; 83:13-25. [DOI: 10.1007/s00426-018-0975-x] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2017] [Accepted: 01/02/2018] [Indexed: 11/29/2022]
|
12
|
Richter A, Barman A, Wüstenberg T, Soch J, Schanze D, Deibele A, Behnisch G, Assmann A, Klein M, Zenker M, Seidenbecher C, Schott BH. Behavioral and Neural Manifestations of Reward Memory in Carriers of Low-Expressing versus High-Expressing Genetic Variants of the Dopamine D2 Receptor. Front Psychol 2017; 8:654. [PMID: 28507526 PMCID: PMC5410587 DOI: 10.3389/fpsyg.2017.00654] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2016] [Accepted: 04/12/2017] [Indexed: 12/13/2022] Open
Abstract
Dopamine is critically important in the neural manifestation of motivated behavior, and alterations in the human dopaminergic system have been implicated in the etiology of motivation-related psychiatric disorders, most prominently addiction. Patients with chronic addiction exhibit reduced dopamine D2 receptor (DRD2) availability in the striatum, and the DRD2 TaqIA (rs1800497) and C957T (rs6277) genetic polymorphisms have previously been linked to individual differences in striatal dopamine metabolism and clinical risk for alcohol and nicotine dependence. Here, we investigated the hypothesis that the variants of these polymorphisms would show increased reward-related memory formation, which has previously been shown to jointly engage the mesolimbic dopaminergic system and the hippocampus, as a potential intermediate phenotype for addiction memory. To this end, we performed functional magnetic resonance imaging (fMRI) in 62 young, healthy individuals genotyped for DRD2 TaqIA and C957T variants. Participants performed an incentive delay task, followed by a recognition memory task 24 h later. We observed effects of both genotypes on the overall recognition performance with carriers of low-expressing variants, namely TaqIA A1 carriers and C957T C homozygotes, showing better performance than the other genotype groups. In addition to the better memory performance, C957T C homozygotes also exhibited a response bias for cues predicting monetary reward. At the neural level, the C957T polymorphism was associated with a genotype-related modulation of right hippocampal and striatal fMRI responses predictive of subsequent recognition confidence for reward-predicting items. Our results indicate that genetic variations associated with DRD2 expression affect explicit memory, specifically for rewarded stimuli. We suggest that the relatively better memory for rewarded stimuli in carriers of low-expressing DRD2 variants may reflect an intermediate phenotype of addiction memory.
Collapse
Affiliation(s)
- Anni Richter
- Leibniz Institute for NeurobiologyMagdeburg, Germany
| | | | - Torsten Wüstenberg
- Department of Psychiatry and Psychotherapy, Charité University HospitalBerlin, Germany
| | - Joram Soch
- Leibniz Institute for NeurobiologyMagdeburg, Germany
| | - Denny Schanze
- Institute of Human Genetics, Otto von Guericke UniversityMagdeburg, Germany
| | - Anna Deibele
- Leibniz Institute for NeurobiologyMagdeburg, Germany
| | | | - Anne Assmann
- Leibniz Institute for NeurobiologyMagdeburg, Germany.,Department of Neurology, University of MagdeburgMagdeburg, Germany
| | - Marieke Klein
- Leibniz Institute for NeurobiologyMagdeburg, Germany
| | - Martin Zenker
- Institute of Human Genetics, Otto von Guericke UniversityMagdeburg, Germany
| | - Constanze Seidenbecher
- Leibniz Institute for NeurobiologyMagdeburg, Germany.,Center for Behavioral Brain SciencesMagdeburg, Germany
| | - Björn H Schott
- Leibniz Institute for NeurobiologyMagdeburg, Germany.,Department of Psychiatry and Psychotherapy, Charité University HospitalBerlin, Germany.,Department of Neurology, University of MagdeburgMagdeburg, Germany.,Center for Behavioral Brain SciencesMagdeburg, Germany
| |
Collapse
|
13
|
Smith CT, Dang LC, Buckholtz JW, Tetreault AM, Cowan RL, Kessler RM, Zald DH. The impact of common dopamine D2 receptor gene polymorphisms on D2/3 receptor availability: C957T as a key determinant in putamen and ventral striatum. Transl Psychiatry 2017; 7:e1091. [PMID: 28398340 PMCID: PMC5416688 DOI: 10.1038/tp.2017.45] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/03/2016] [Revised: 12/02/2016] [Accepted: 01/17/2017] [Indexed: 12/20/2022] Open
Abstract
Dopamine function is broadly implicated in multiple neuropsychiatric conditions believed to have a genetic basis. Although a few positron emission tomography (PET) studies have investigated the impact of single-nucleotide polymorphisms (SNPs) in the dopamine D2 receptor gene (DRD2) on D2/3 receptor availability (binding potential, BPND), these studies have often been limited by small sample size. Furthermore, the most commonly studied SNP in D2/3 BPND (Taq1A) is not located in the DRD2 gene itself, suggesting that its linkage with other DRD2 SNPs may explain previous PET findings. Here, in the largest PET genetic study to date (n=84), we tested for effects of the C957T and -141C Ins/Del SNPs (located within DRD2) as well as Taq1A on BPND of the high-affinity D2 receptor tracer 18F-Fallypride. In a whole-brain voxelwise analysis, we found a positive linear effect of C957T T allele status on striatal BPND bilaterally. The multilocus genetic scores containing C957T and one or both of the other SNPs produced qualitatively similar striatal results to C957T alone. The number of C957T T alleles predicted BPND in anatomically defined putamen and ventral striatum (but not caudate) regions of interest, suggesting some regional specificity of effects in the striatum. By contrast, no significant effects arose in cortical regions. Taken together, our data support the critical role of C957T in striatal D2/3 receptor availability. This work has implications for a number of psychiatric conditions in which dopamine signaling and variation in C957T status have been implicated, including schizophrenia and substance use disorders.
Collapse
Affiliation(s)
- C T Smith
- Department of Psychology, Vanderbilt University, Nashville, TN, USA,Department of Psychology, Vanderbilt University, PMB 407817, 2301 Vanderbilt Place, Nashville, TN 37240-7817, USA. E-mail:
| | - L C Dang
- Department of Psychology, Vanderbilt University, Nashville, TN, USA
| | - J W Buckholtz
- Department of Psychology, Harvard University, Cambridge, MA, USA,Department of Psychiatry, Massachusetts General Hospital, Boston, MA, USA
| | - A M Tetreault
- Department of Psychology, Vanderbilt University, Nashville, TN, USA
| | - R L Cowan
- Department of Psychology, Vanderbilt University, Nashville, TN, USA,Department of Psychiatry, Vanderbilt University School of Medicine, Nashville, TN, USA
| | - R M Kessler
- Department of Radiology, UAB School of Medicine, Birmingham, AL, USA
| | - D H Zald
- Department of Psychology, Vanderbilt University, Nashville, TN, USA,Department of Psychiatry, Vanderbilt University School of Medicine, Nashville, TN, USA
| |
Collapse
|
14
|
Schlund MW, Treacher K, Preston O, Magee SK, Richman DM, Brewer AT, Cameron G, Dymond S. “Watch out!”: Effects of instructed threat and avoidance on human free-operant approach-avoidance behavior. J Exp Anal Behav 2017; 107:101-122. [DOI: 10.1002/jeab.238] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2016] [Accepted: 12/15/2016] [Indexed: 11/10/2022]
Affiliation(s)
| | - Kay Treacher
- Department of Behavior Analysis; University of North Texas
| | - Oli Preston
- Department of Psychology; Swansea University; United Kingdom
| | - Sandy K. Magee
- Department of Behavior Analysis; University of North Texas
| | - David M. Richman
- Department of Educational Psychology and Leadership; Texas Tech University
| | - Adam T. Brewer
- Department of Psychology and Liberal Arts; Florida Institute of Technology
| | - Gemma Cameron
- Department of Psychology; Swansea University; United Kingdom
| | - Simon Dymond
- Department of Psychology; Swansea University; United Kingdom
- Department of Psychology; Reykjavík University; Iceland
| |
Collapse
|
15
|
Acute effects of cocaine and cannabis on reversal learning as a function of COMT and DRD2 genotype. Psychopharmacology (Berl) 2016; 233:199-211. [PMID: 26572896 PMCID: PMC4700084 DOI: 10.1007/s00213-015-4141-5] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/20/2015] [Accepted: 09/20/2015] [Indexed: 12/27/2022]
Abstract
RATIONALE Long-term cannabis and cocaine use has been associated with impairments in reversal learning. However, how acute cannabis and cocaine administration affect reversal learning in humans is not known. OBJECTIVE In this study, we aimed to establish the acute effects of administration of cannabis and cocaine on valence-dependent reversal learning as a function of DRD2 Taq1A (rs1800497) and COMT Val108/158Met (rs4680) genotype. METHODS A double-blind placebo-controlled randomized 3-way crossover design was used. Sixty-one regular poly-drug users completed a deterministic reversal learning task under the influence of cocaine, cannabis, and placebo that enabled assessment of both reward- and punishment-based reversal learning. RESULTS Proportion correct on the reversal learning task was increased by cocaine, but decreased by cannabis. Effects of cocaine depended on the DRD2 genotype, as increases in proportion correct were seen only in the A1 carriers, and not in the A2/A2 homozygotes. COMT genotype did not modulate drug-induced effects on reversal learning. CONCLUSIONS These data indicate that acute administration of cannabis and cocaine has opposite effects on reversal learning. The effects of cocaine, but not cannabis, depend on interindividual genetic differences in the dopamine D2 receptor gene.
Collapse
|
16
|
Schevernels H, Bombeke K, Krebs RM, Boehler CN. Preparing for (valenced) action: The role of differential effort in the orthogonalized go/no-go task. Psychophysiology 2015; 53:186-97. [DOI: 10.1111/psyp.12558] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2015] [Accepted: 08/24/2015] [Indexed: 12/01/2022]
Affiliation(s)
- Hanne Schevernels
- Department of Experimental Psychology; Ghent University; Ghent Belgium
| | - Klaas Bombeke
- Department of Experimental Psychology; Ghent University; Ghent Belgium
| | - Ruth M. Krebs
- Department of Experimental Psychology; Ghent University; Ghent Belgium
| | - C. Nico Boehler
- Department of Experimental Psychology; Ghent University; Ghent Belgium
| |
Collapse
|
17
|
Laricchiuta D. Editorial: Individual differences: from neurobiological bases to new insight on approach and avoidance behavior. Front Syst Neurosci 2015; 9:125. [PMID: 26441557 PMCID: PMC4569743 DOI: 10.3389/fnsys.2015.00125] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2015] [Accepted: 08/26/2015] [Indexed: 01/12/2023] Open
Affiliation(s)
- Daniela Laricchiuta
- IRCCS Fondazione Santa Lucia Rome, Italy ; Department of Dynamic and Clinical Psychology, Faculty of Medicine and Psychology, University "Sapienza" of Rome Rome, Italy
| |
Collapse
|