1
|
Algermissen J, den Ouden HEM. Pupil dilation reflects effortful action invigoration in overcoming aversive Pavlovian biases. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2024; 24:720-739. [PMID: 38773022 PMCID: PMC11233311 DOI: 10.3758/s13415-024-01191-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 04/22/2024] [Indexed: 05/23/2024]
Abstract
"Pavlovian" or "motivational" biases describe the phenomenon that the valence of prospective outcomes modulates action invigoration: Reward prospect invigorates action, whereas punishment prospect suppresses it. The adaptive role of these biases in decision-making is still unclear. One idea is that they constitute a fast-and-frugal decision strategy in situations characterized by high arousal, e.g., in presence of a predator, which demand a quick response. In this pre-registered study (N = 35), we tested whether such a situation-induced via subliminally presented angry versus neutral faces-leads to increased reliance on Pavlovian biases. We measured trial-by-trial arousal by tracking pupil diameter while participants performed an orthogonalized Motivational Go/NoGo Task. Pavlovian biases were present in responses, reaction times, and even gaze, with lower gaze dispersion under aversive cues reflecting "freezing of gaze." The subliminally presented faces did not affect responses, reaction times, or pupil diameter, suggesting that the arousal manipulation was ineffective. However, pupil dilations reflected facets of bias suppression, specifically the physical (but not cognitive) effort needed to overcome aversive inhibition: Particularly strong and sustained dilations occurred when participants managed to perform Go responses to aversive cues. Conversely, no such dilations occurred when they managed to inhibit responses to Win cues. These results suggest that pupil diameter does not reflect response conflict per se nor the inhibition of prepotent responses, but specifically effortful action invigoration as needed to overcome aversive inhibition. We discuss our results in the context of the "value of work" theory of striatal dopamine.
Collapse
Affiliation(s)
- Johannes Algermissen
- Donders Institute for Brain, Radboud University, Cognition, and Behaviour, Thomas van Aquinostraat 4, 6526 GD, Nijmegen, The Netherlands.
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Hanneke E M den Ouden
- Donders Institute for Brain, Radboud University, Cognition, and Behaviour, Thomas van Aquinostraat 4, 6526 GD, Nijmegen, The Netherlands.
| |
Collapse
|
2
|
Nunez MD, Fernandez K, Srinivasan R, Vandekerckhove J. A tutorial on fitting joint models of M/EEG and behavior to understand cognition. Behav Res Methods 2024:10.3758/s13428-023-02331-x. [PMID: 38409458 DOI: 10.3758/s13428-023-02331-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/21/2023] [Indexed: 02/28/2024]
Abstract
We present motivation and practical steps necessary to find parameter estimates of joint models of behavior and neural electrophysiological data. This tutorial is written for researchers wishing to build joint models of human behavior and scalp and intracranial electroencephalographic (EEG) or magnetoencephalographic (MEG) data, and more specifically those researchers who seek to understand human cognition. Although these techniques could easily be applied to animal models, the focus of this tutorial is on human participants. Joint modeling of M/EEG and behavior requires some knowledge of existing computational and cognitive theories, M/EEG artifact correction, M/EEG analysis techniques, cognitive modeling, and programming for statistical modeling implementation. This paper seeks to give an introduction to these techniques as they apply to estimating parameters from neurocognitive models of M/EEG and human behavior, and to evaluate model results and compare models. Due to our research and knowledge on the subject matter, our examples in this paper will focus on testing specific hypotheses in human decision-making theory. However, most of the motivation and discussion of this paper applies across many modeling procedures and applications. We provide Python (and linked R) code examples in the tutorial and appendix. Readers are encouraged to try the exercises at the end of the document.
Collapse
Affiliation(s)
- Michael D Nunez
- Psychological Methods, University of Amsterdam, Amsterdam, The Netherlands.
| | - Kianté Fernandez
- Department of Psychology, University of California, Los Angeles, CA, USA
| | - Ramesh Srinivasan
- Department of Cognitive Sciences, University of California, Irvine, CA, USA
- Department of Biomedical Engineering, University of California, Irvine, CA, USA
- Institute of Mathematical Behavioral Sciences, University of California, Irvine, CA, USA
| | - Joachim Vandekerckhove
- Department of Cognitive Sciences, University of California, Irvine, CA, USA
- Institute of Mathematical Behavioral Sciences, University of California, Irvine, CA, USA
- Department of Statistics, University of California, Irvine, CA, USA
| |
Collapse
|
3
|
Algermissen J, Swart JC, Scheeringa R, Cools R, den Ouden HEM. Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases. Nat Commun 2024; 15:19. [PMID: 38168089 PMCID: PMC10762147 DOI: 10.1038/s41467-023-44632-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 12/22/2023] [Indexed: 01/05/2024] Open
Abstract
Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.
Collapse
Affiliation(s)
- Johannes Algermissen
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| | - Jennifer C Swart
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - René Scheeringa
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, Essen, Germany
| | - Roshan Cools
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Department of Psychiatry, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Hanneke E M den Ouden
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| |
Collapse
|
4
|
Park H, Doh H, Lee E, Park H, Ahn WY. The neurocognitive role of working memory load when Pavlovian motivational control affects instrumental learning. PLoS Comput Biol 2023; 19:e1011692. [PMID: 38064498 PMCID: PMC10732416 DOI: 10.1371/journal.pcbi.1011692] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Revised: 12/20/2023] [Accepted: 11/15/2023] [Indexed: 12/21/2023] Open
Abstract
Research suggests that a fast, capacity-limited working memory (WM) system and a slow, incremental reinforcement learning (RL) system jointly contribute to instrumental learning. Thus, situations that strain WM resources alter instrumental learning: under WM loads, learning becomes slow and incremental, the reliance on computationally efficient learning increases, and action selection becomes more random. It is also suggested that Pavlovian learning influences people's behavior during instrumental learning by providing hard-wired instinctive responses including approach to reward predictors and avoidance of punishment predictors. However, it remains unknown how constraints on WM resources affect instrumental learning under Pavlovian influence. Thus, we conducted a functional magnetic resonance imaging (fMRI) study (N = 49) in which participants completed an instrumental learning task with Pavlovian-instrumental conflict (the orthogonalized go/no-go task) both with and without extra WM load. Behavioral and computational modeling analyses revealed that WM load reduced the learning rate and increased random choice, without affecting Pavlovian bias. Model-based fMRI analysis revealed that WM load strengthened RPE signaling in the striatum. Moreover, under WM load, the striatum showed weakened connectivity with the ventromedial and dorsolateral prefrontal cortex when computing reward expectations. These results suggest that the limitation of cognitive resources by WM load promotes slow and incremental learning through the weakened cooperation between WM and RL; such limitation also makes action selection more random, but it does not directly affect the balance between instrumental and Pavlovian systems.
Collapse
Affiliation(s)
- Heesun Park
- Department of Psychology, Seoul National University, Seoul, Korea
| | - Hoyoung Doh
- Department of Psychology, Seoul National University, Seoul, Korea
| | - Eunhwi Lee
- Department of Psychology, Seoul National University, Seoul, Korea
| | - Harhim Park
- Department of Psychology, Seoul National University, Seoul, Korea
| | - Woo-Young Ahn
- Department of Psychology, Seoul National University, Seoul, Korea
- Department of Brain and Cognitive Sciences, Seoul National University, Seoul, Korea
| |
Collapse
|
5
|
Queirazza F, Steele JD, Krishnadas R, Cavanagh J, Philiastides MG. Functional Magnetic Resonance Imaging Signatures of Pavlovian and Instrumental Valuation Systems during a Modified Orthogonalized Go/No-go Task. J Cogn Neurosci 2023; 35:2089-2109. [PMID: 37788326 DOI: 10.1162/jocn_a_02062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/05/2023]
Abstract
Motivational (i.e., Pavlovian) values interfere with instrumental responding and can lead to suboptimal decision-making. In humans, task-based neuroimaging studies have only recently started illuminating the functional neuroanatomy of Pavlovian biasing of instrumental control. To provide a mechanistic understanding of the neural dynamics underlying the Pavlovian and instrumental valuation systems, analysis of neuroimaging data has been informed by computational modeling of conditioned behavior. Nonetheless, because of collinearities in Pavlovian and instrumental predictions, previous research failed to tease out hemodynamic activity that is parametrically and dynamically modulated by coexistent Pavlovian and instrumental value expectations. Moreover, neural correlates of Pavlovian to instrumental transfer effects have so far only been identified in extinction (i.e., in the absence of learning). In this study, we devised a modified version of the orthogonalized go/no-go paradigm, which introduced Pavlovian-only catch trials to better disambiguate trial-by-trial Pavlovian and instrumental predictions in both sexes. We found that hemodynamic activity in the ventromedial pFC covaried uniquely with the model-derived Pavlovian value expectations. Notably, modulation of neural activity encoding for instrumental predictions in the supplementary motor cortex was linked to successful action selection in conflict conditions. Furthermore, hemodynamic activity in regions pertaining to the limbic system and medial pFC was correlated with synergistic Pavlovian and instrumental predictions and improved conditioned behavior during congruent trials. Altogether, our results provide new insights into the functional neuroanatomy of decision-making and corroborate the validity of our variant of the orthogonalized go/no-go task as a behavioral assay of the Pavlovian and instrumental valuation systems.
Collapse
|
6
|
Sedlinská T, Bolte L, Melsæter E, Mittner M, Csifcsák G. Transcranial direct-current stimulation enhances Pavlovian tendencies during intermittent loss of control. Front Psychiatry 2023; 14:1164208. [PMID: 37229391 PMCID: PMC10203223 DOI: 10.3389/fpsyt.2023.1164208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/12/2023] [Accepted: 04/13/2023] [Indexed: 05/27/2023] Open
Abstract
Introduction Pavlovian bias is an innate motivational tendency to approach rewards and remain passive in the face of punishment. The relative reliance on Pavlovian valuation has been found to increase when the perceived control over environmental reinforcers is compromised, leading to behavior resembling learned helplessness (LH). Methods Sixty healthy young adults underwent a Go-NoGo reinforcement learning task and received anodal high-definition transcranial direct current stimulation (HD-tDCS) over the medial prefrontal/dorsal anterior cingulate cortex in our randomized, double-blind, sham- controlled study. Furthermore, we evaluated changes in cue-locked mid-frontal theta power derived from simultaneous electroencephalography (EEG). We hypothesized that active stimulation would reduce Pavlovian bias during manipulation of outcome controllability, and the effect would be accompanied by stronger mid-frontal theta activity, representing arbitration between choice strategies in favor of instrumental relative to Pavlovian valuation. Results We found a progressive decrease in Pavlovian bias during and after loss of control over feedback. Active HD-tDCS counteracted this effect while not affecting the mid-frontal theta signal. Discussion The results were at odds with our hypotheses but also with previous findings reporting LH-like patterns during and after loss of control without brain stimulation. The discrepancy may be related to different protocols for the controllability manipulation. We argue that the subjective evaluation of task controllability is crucial in mediating the balance between Pavlovian and instrumental valuation during reinforcement learning and that the medial prefrontal/dorsal anterior cingulate cortex is a key region in this respect. These findings have implications for understanding the behavioral and neural underpinnings of LH in humans.
Collapse
Affiliation(s)
- Terezie Sedlinská
- Department of Psychology, UiT The Arctic University of Norway, Tromsø, Norway
- Psychiatric University Hospital, Zürich, Switzerland
| | - Lara Bolte
- Department of Psychology, UiT The Arctic University of Norway, Tromsø, Norway
| | - Eirik Melsæter
- Department of Psychology, UiT The Arctic University of Norway, Tromsø, Norway
| | - Matthias Mittner
- Department of Psychology, UiT The Arctic University of Norway, Tromsø, Norway
| | - Gábor Csifcsák
- Department of Psychology, UiT The Arctic University of Norway, Tromsø, Norway
| |
Collapse
|
7
|
Chen H, Belanger MJ, Garbusow M, Kuitunen-Paul S, Huys QJM, Heinz A, Rapp MA, Smolka MN. Susceptibility to interference between Pavlovian and instrumental control predisposes risky alcohol use developmental trajectory from ages 18 to 24. Addict Biol 2023; 28:e13263. [PMID: 36692874 DOI: 10.1111/adb.13263] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2022] [Revised: 10/14/2022] [Accepted: 11/23/2022] [Indexed: 01/06/2023]
Abstract
Pavlovian cues can influence ongoing instrumental behaviour via Pavlovian-to-instrumental transfer (PIT) processes. While appetitive Pavlovian cues tend to promote instrumental approach, they are detrimental when avoidance behaviour is required, and vice versa for aversive cues. We recently reported that susceptibility to interference between Pavlovian and instrumental control assessed via a PIT task was associated with risky alcohol use at age 18. We now investigated whether such susceptibility also predicts drinking trajectories until age 24, based on AUDIT (Alcohol Use Disorders Identification Test) consumption and binge drinking (gramme alcohol/drinking occasion) scores. The interference PIT effect, assessed at ages 18 and 21 during fMRI, was characterized by increased error rates (ER) and enhanced neural responses in the ventral striatum (VS), the lateral and dorsomedial prefrontal cortices (dmPFC) during conflict, that is, when an instrumental approach was required in the presence of an aversive Pavlovian cue or vice versa. We found that a stronger VS response during conflict at age 18 was associated with a higher starting point of both drinking trajectories but predicted a decrease in binge drinking. At age 21, high ER and enhanced neural responses in the dmPFC were associated with increasing AUDIT-C scores over the next 3 years until age 24. Overall, susceptibility to interference between Pavlovian and instrumental control might be viewed as a predisposing mechanism towards hazardous alcohol use during young adulthood, and the identified high-risk group may profit from targeted interventions.
Collapse
Affiliation(s)
- Hao Chen
- Department of Psychiatry and Psychotherapy, Technische Universität Dresden, Dresden, Germany
| | - Matthew J Belanger
- Department of Psychiatry and Psychotherapy, Technische Universität Dresden, Dresden, Germany
| | - Maria Garbusow
- Department of Psychiatry and Psychotherapy, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Sören Kuitunen-Paul
- Institute of Clinical Psychology and Psychotherapy, Technische Universität Dresden, Dresden, Germany
| | - Quentin J M Huys
- Division of Psychiatry and Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
| | - Andreas Heinz
- Department of Psychiatry and Psychotherapy, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Michael A Rapp
- Social and Preventive Medicine, Area of Excellence Cognitive Sciences, University of Potsdam, Potsdam, Germany
| | - Michael N Smolka
- Department of Psychiatry and Psychotherapy, Technische Universität Dresden, Dresden, Germany
| |
Collapse
|
8
|
Tarasi L, di Pellegrino G, Romei V. Are you an empiricist or a believer? Neural signatures of predictive strategies in humans. Prog Neurobiol 2022; 219:102367. [DOI: 10.1016/j.pneurobio.2022.102367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Revised: 10/06/2022] [Accepted: 10/18/2022] [Indexed: 12/04/2022]
|
9
|
Weber I, Zorowitz S, Niv Y, Bennett D. The effects of induced positive and negative affect on Pavlovian-instrumental interactions. Cogn Emot 2022; 36:1343-1360. [PMID: 35929878 PMCID: PMC9852069 DOI: 10.1080/02699931.2022.2109600] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Revised: 07/19/2022] [Accepted: 07/26/2022] [Indexed: 01/22/2023]
Abstract
Across species, animals have an intrinsic drive to approach appetitive stimuli and to withdraw from aversive stimuli. In affective science, influential theories of emotion link positive affect with strengthened behavioural approach and negative affect with avoidance. Based on these theories, we predicted that individuals' positive and negative affect levels should particularly influence their behaviour when innate Pavlovian approach/avoidance tendencies conflict with learned instrumental behaviours. Here, across two experiments - exploratory Experiment 1 (N = 91) and a preregistered confirmatory Experiment 2 (N = 335) - we assessed how induced positive and negative affect influenced Pavlovian-instrumental interactions in a reward/punishment Go/No-Go task. Contrary to our hypotheses, we found no evidence for a main effect of positive/negative affect on either approach/avoidance behaviour or Pavlovian-instrumental interactions. However, we did find evidence that the effects of induced affect on behaviour were moderated by individual differences in self-reported behavioural inhibition and gender. Exploratory computational modelling analyses explained these demographic moderating effects as arising from positive correlations between demographic factors and individual differences in the strength of Pavlovian-instrumental interactions. These findings serve to sharpen our understanding of the effects of positive and negative affect on instrumental behaviour.
Collapse
Affiliation(s)
- Isla Weber
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Sam Zorowitz
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Yael Niv
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Department of Psychology, Princeton University, Princeton, USA
| | - Daniel Bennett
- School of Psychological Sciences, Monash University, Clayton, Australia
| |
Collapse
|
10
|
Static and temporal dynamic changes of intrinsic brain activity in pediatric and adults OCD. J Affect Disord 2022; 311:416-424. [PMID: 35618169 DOI: 10.1016/j.jad.2022.05.101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Revised: 04/29/2022] [Accepted: 05/16/2022] [Indexed: 11/24/2022]
Abstract
Epidemiological and clinical age differences in obsessive-compulsive disorder (OCD) have been reported in clinical symptoms and morphometry changes; however, age differences in amplitude of low-frequency fluctuation and the relationship between ALFF imaging and clinical symptoms has not been thoroughly studied in OCD. Age may be an important feature associated with distinct subtypes of OCD. To examine the effect of age on OCD, the current study enrolled 92 OCD patients (32 pediatrics and 60 adults) and matched HCs (33 pediatrics and 84 adults), undergoing resting-state functional magnetic resonance imaging. The spontaneous brain activity was measured by static and dynamic amplitude of low-frequency fluctuation (ALFF) followed by two-way ANOVA. In pediatric OCD patients versus adult patients, we observed a significantly higher ALFF in the default mode network (DMN), including posterior cingulate, precuneus and superior frontal gyrus, and extending to cuneus, lingual gyrus. Additionally, the increased ALFF and dynamic ALFF in the precentral gyrus were found in pediatric patients. In OCD patients compared with controls, we found a significantly increased ALFF in hippocampal gyrus, cerebellum network (CN), and the dALFF in middle and inferior occipital gyrus, bilateral paracentral lobule and sensorimotor network. The findings emphasized the different patterns of static and dynamic intrinsic brain activity alterations associated with pediatric and adult OCD patients. These results provide unique insights into constructing evidenced-based distinct OCD subtypes based on brain activity and point the need of specified management for pediatric and adult OCD patients in clinical setting.
Collapse
|
11
|
Scholz V, Hook RW, Kandroodi MR, Algermissen J, Ioannidis K, Christmas D, Valle S, Robbins TW, Grant JE, Chamberlain SR, den Ouden HEM. Cortical dopamine reduces the impact of motivational biases governing automated behaviour. Neuropsychopharmacology 2022; 47:1503-1512. [PMID: 35260787 PMCID: PMC9206002 DOI: 10.1038/s41386-022-01291-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 02/02/2022] [Indexed: 11/09/2022]
Abstract
Motivations shape our behaviour: the promise of reward invigorates, while in the face of punishment, we hold back. Abnormalities of motivational processing are implicated in clinical disorders characterised by excessive habits and loss of top-down control, notably substance and behavioural addictions. Striatal and frontal dopamine have been hypothesised to play complementary roles in the respective generation and control of these motivational biases. However, while dopaminergic interventions have indeed been found to modulate motivational biases, these previous pharmacological studies used regionally non-selective pharmacological agents. Here, we tested the hypothesis that frontal dopamine controls the balance between Pavlovian, bias-driven automated responding and instrumentally learned action values. Specifically, we examined whether selective enhancement of cortical dopamine either (i) enables adaptive suppression of Pavlovian control when biases are maladaptive; or (ii) non-specifically modulates the degree of bias-driven automated responding. Healthy individuals (n = 35) received the catechol-o-methyltransferase (COMT) inhibitor tolcapone in a randomised, double-blind, placebo-controlled cross-over design, and completed a motivational Go NoGo task known to elicit motivational biases. In support of hypothesis (ii), tolcapone globally decreased motivational bias. Specifically, tolcapone improved performance on trials where the bias was unhelpful, but impaired performance in bias-congruent conditions. These results indicate a non-selective role for cortical dopamine in the regulation of motivational processes underpinning top-down control over automated behaviour. The findings have direct relevance to understanding neurobiological mechanisms underpinning addiction and obsessive-compulsive disorders, as well as highlighting a potential trans-diagnostic novel mechanism to address such symptoms.
Collapse
Affiliation(s)
- Vanessa Scholz
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands. .,Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, Margarete-Höppel-Platz1, 97080, Würzburg, Germany.
| | - Roxanne W. Hook
- grid.5335.00000000121885934Department of Psychiatry, University of Cambridge, Cambridge, UK
| | - Mojtaba Rostami Kandroodi
- grid.5590.90000000122931605Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands ,grid.46072.370000 0004 0612 7950School of Electrical and Computer Engineering, University of Tehran, Tehran, Iran
| | - Johannes Algermissen
- grid.5590.90000000122931605Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| | - Konstantinos Ioannidis
- grid.5335.00000000121885934Department of Psychiatry, University of Cambridge, Cambridge, UK ,grid.450563.10000 0004 0412 9303Cambridgeshire & Peterborough NHS Foundation Trust, Cambridge, UK ,grid.5012.60000 0001 0481 6099Department of International Health, Care and Public Health Research Institute, Maastricht University, Maastricht, The Netherlands
| | - David Christmas
- grid.5335.00000000121885934Department of Psychiatry, University of Cambridge, Cambridge, UK ,grid.450563.10000 0004 0412 9303Cambridgeshire & Peterborough NHS Foundation Trust, Cambridge, UK
| | - Stephanie Valle
- grid.170205.10000 0004 1936 7822Department of Psychiatry and Behavioral Neuroscience, University of Chicago, Chicago, IL USA
| | - Trevor W. Robbins
- grid.5335.00000000121885934Department of Psychology, and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, UK
| | - Jon E. Grant
- grid.170205.10000 0004 1936 7822Department of Psychiatry and Behavioral Neuroscience, University of Chicago, Chicago, IL USA
| | - Samuel R. Chamberlain
- grid.5335.00000000121885934Department of Psychiatry, University of Cambridge, Cambridge, UK ,grid.5491.90000 0004 1936 9297Department of Psychiatry, Faculty of Medicine, University of Southampton, Southampton, UK. Southern Health NHS Foundation Trust, Southampton, UK
| | - Hanneke E. M. den Ouden
- grid.5590.90000000122931605Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| |
Collapse
|
12
|
Algermissen J, Swart JC, Scheeringa R, Cools R, den Ouden HEM. Striatal BOLD and Midfrontal Theta Power Express Motivation for Action. Cereb Cortex 2021; 32:2924-2942. [PMID: 34849626 PMCID: PMC9290551 DOI: 10.1093/cercor/bhab391] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2020] [Revised: 05/04/2021] [Accepted: 06/06/2021] [Indexed: 11/14/2022] Open
Abstract
Action selection is biased by the valence of anticipated outcomes. To assess mechanisms by which these motivational biases are expressed and controlled, we measured simultaneous EEG-fMRI during a motivational Go/NoGo learning task (N = 36), leveraging the temporal resolution of EEG and subcortical access of fMRI. VmPFC BOLD encoded cue valence, importantly predicting trial-by-trial valence-driven response speed differences and EEG theta power around cue onset. In contrast, striatal BOLD encoded selection of active Go responses and correlated with theta power around response time. Within trials, theta power ramped in the fashion of an evidence accumulation signal for the value of making a "Go" response, capturing the faster responding to reward cues. Our findings reveal a dual nature of midfrontal theta power, with early components reflecting the vmPFC contribution to motivational biases, and late components reflecting their striatal translation into behavior, in line with influential recent "value of work" theories of striatal processing.
Collapse
Affiliation(s)
- Johannes Algermissen
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Thomas van Aquinostraat 4, 6525 GD Nijmegen, The Netherlands
| | - Jennifer C Swart
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Thomas van Aquinostraat 4, 6525 GD Nijmegen, The Netherlands
| | - René Scheeringa
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Thomas van Aquinostraat 4, 6525 GD Nijmegen, The Netherlands.,Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, Kokereiallee 7, 45141 Essen, Germany
| | - Roshan Cools
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Thomas van Aquinostraat 4, 6525 GD Nijmegen, The Netherlands.,Department of Psychiatry, Radboud University Medical Centre, Reinier Postlaan 10, 6525 GC Nijmegen, The Netherlands
| | - Hanneke E M den Ouden
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Thomas van Aquinostraat 4, 6525 GD Nijmegen, The Netherlands
| |
Collapse
|
13
|
Csifcsák G, Bjørkøy J, Kuyateh S, Reithe H, Mittner M. Transcranial Direct Current Stimulation above the Medial Prefrontal Cortex Facilitates Decision-Making following Periods of Low Outcome Controllability. eNeuro 2021; 8:ENEURO.0041-21.2021. [PMID: 34433576 PMCID: PMC8425969 DOI: 10.1523/eneuro.0041-21.2021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2021] [Revised: 08/05/2021] [Accepted: 08/09/2021] [Indexed: 11/21/2022] Open
Abstract
Recent studies suggest that choice behavior in reinforcement learning tasks is shaped by the level of outcome controllability. In particular, Pavlovian bias (PB) seems to be enhanced under low levels of control, manifesting in approach tendencies toward rewards and response inhibition when facing potential losses. The medial prefrontal cortex (mPFC) has been implicated both in evaluating outcome controllability and in the recruitment of cognitive control (CC) to suppress maladaptive PB during reinforcement learning. The current study tested whether high-definition transcranial direct current stimulation (HD-tDCS) above the mPFC of healthy humans can influence PB, and counteract the previously documented, deleterious behavioral effects of low outcome controllability on decision-making. In a preregistered, between-group, double-blind study (N = 103 adults, both sexes), we tested the interaction between controllability and HD-tDCS on parameters of choice behavior in a Go/NoGo task. Relative to sham stimulation, HD-tDCS resulted in more robust performance improvement following reduced control, an effect that was more pronounced in appetitive trials. In addition, we found evidence for weaker PB when HD-tDCS was administered during low controllability over outcomes. Computational modeling revealed that parameter estimates of learning rate and choice randomness were modulated by controllability, HD-tDCS and their interaction. Overall, these results highlight the potential of our HD-tDCS protocol for interfering with choice arbitration under low levels of control, resulting in more adaptive behavior.
Collapse
Affiliation(s)
- Gábor Csifcsák
- Department of Psychology, UiT The Arctic University of Norway, Tromsø 9037, Norway
| | - Jorunn Bjørkøy
- Department of Psychology, UiT The Arctic University of Norway, Tromsø 9037, Norway
| | - Sarjo Kuyateh
- Department of Psychology, UiT The Arctic University of Norway, Tromsø 9037, Norway
| | - Haakon Reithe
- Department of Psychology, UiT The Arctic University of Norway, Tromsø 9037, Norway
| | - Matthias Mittner
- Department of Psychology, UiT The Arctic University of Norway, Tromsø 9037, Norway
| |
Collapse
|
14
|
Richter A, de Boer L, Guitart-Masip M, Behnisch G, Seidenbecher CI, Schott BH. Motivational learning biases are differentially modulated by genetic determinants of striatal and prefrontal dopamine function. J Neural Transm (Vienna) 2021; 128:1705-1720. [PMID: 34302222 PMCID: PMC8536632 DOI: 10.1007/s00702-021-02382-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Accepted: 07/04/2021] [Indexed: 01/20/2023]
Abstract
Dopaminergic neurotransmission plays a pivotal role in appetitively motivated behavior in mammals, including humans. Notably, action and valence are not independent in motivated tasks, and it is particularly difficult for humans to learn the inhibition of an action to obtain a reward. We have previously observed that the carriers of the DRD2/ANKK1 TaqIA A1 allele, that has been associated with reduced striatal dopamine D2 receptor expression, showed a diminished learning performance when required to learn response inhibition to obtain rewards, a finding that was replicated in two independent cohorts. With our present study, we followed two aims: first, we aimed to replicate our finding on the DRD2/ANKK1 TaqIA polymorphism in a third independent cohort (N = 99) and to investigate the nature of the genetic effects more closely using trial-by-trial behavioral analysis and computational modeling in the combined dataset (N = 281). Second, we aimed to assess a potentially modulatory role of prefrontal dopamine availability, using the widely studied COMT Val108/158Met polymorphism as a proxy. We first report a replication of the above mentioned finding. Interestingly, after combining all three cohorts, exploratory analyses regarding the COMT Val108/158Met polymorphism suggest that homozygotes for the Met allele, which has been linked to higher prefrontal dopaminergic tone, show a lower learning bias. Our results corroborate the importance of genetic variability of the dopaminergic system in individual learning differences of action-valence interaction and, furthermore, suggest that motivational learning biases are differentially modulated by genetic determinants of striatal and prefrontal dopamine function.
Collapse
Affiliation(s)
- Anni Richter
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118, Magdeburg, Germany.
| | - Lieke de Boer
- Ageing Research Centre, Karolinska Institute, Stockholm, Sweden
- Max Planck Institute for Human Development, Center for Lifespan Psychology, Berlin, Germany
| | - Marc Guitart-Masip
- Ageing Research Centre, Karolinska Institute, Stockholm, Sweden
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
| | - Gusalija Behnisch
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118, Magdeburg, Germany
| | - Constanze I Seidenbecher
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118, Magdeburg, Germany
- Center for Behavioral Brain Sciences, Magdeburg, Germany
| | - Björn H Schott
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118, Magdeburg, Germany
- Center for Behavioral Brain Sciences, Magdeburg, Germany
- Department of Psychiatry and Psychotherapy, University Medicine Göttingen, Göttingen, Germany
- Department of Neurology, University of Magdeburg, Magdeburg, Germany
- German Center for Neurodegenerative Diseases (DZNE), Göttingen, Germany
| |
Collapse
|
15
|
Moutoussis M, Garzón B, Neufeld S, Bach DR, Rigoli F, Goodyer I, Bullmore E, Guitart-Masip M, Dolan RJ. Decision-making ability, psychopathology, and brain connectivity. Neuron 2021; 109:2025-2040.e7. [PMID: 34019810 PMCID: PMC8221811 DOI: 10.1016/j.neuron.2021.04.019] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2020] [Revised: 02/16/2021] [Accepted: 04/19/2021] [Indexed: 12/11/2022]
Abstract
Decision-making is a cognitive process of central importance for the quality of our lives. Here, we ask whether a common factor underpins our diverse decision-making abilities. We obtained 32 decision-making measures from 830 young people and identified a common factor that we call "decision acuity," which was distinct from IQ and reflected a generic decision-making ability. Decision acuity was decreased in those with aberrant thinking and low general social functioning. Crucially, decision acuity and IQ had dissociable brain signatures, in terms of their associated neural networks of resting-state functional connectivity. Decision acuity was reliably measured, and its relationship with functional connectivity was also stable when measured in the same individuals 18 months later. Thus, our behavioral and brain data identify a new cognitive construct that underpins decision-making ability across multiple domains. This construct may be important for understanding mental health, particularly regarding poor social function and aberrant thought patterns.
Collapse
Affiliation(s)
- Michael Moutoussis
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK.
| | - Benjamín Garzón
- Aging Research Centre, Karolinska Institute, Stockholm, Sweden
| | - Sharon Neufeld
- Department of Psychiatry, University of Cambridge, Cambridge CB2 0SZ, UK
| | - Dominik R Bach
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK; Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, Psychiatric Hospital, University of Zurich, 8032 Zurich, Switzerland
| | | | - Ian Goodyer
- Department of Psychiatry, University of Cambridge, Cambridge CB2 0SZ, UK
| | - Edward Bullmore
- Department of Psychiatry, University of Cambridge, Cambridge CB2 0SZ, UK
| | - Marc Guitart-Masip
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK; Aging Research Centre, Karolinska Institute, Stockholm, Sweden
| | - Raymond J Dolan
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
| |
Collapse
|
16
|
Modulatory effects of positive mood and approach motivation on reward processing: Two sides of the same coin? COGNITIVE AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2021; 20:236-249. [PMID: 32043206 DOI: 10.3758/s13415-019-00764-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/07/2023]
Abstract
In a previous study (Paul & Pourtois, 2017), we found that positive mood substantially influenced the neural processing of reward, mostly by altering expectations and creating an optimistic bias. Under positive mood, the Reward Positivity (RewP) component and fronto-medial theta activity (FMθ) in response to monetary feedback were both changed compared with neutral mood. Nevertheless, whether positive valence per se or motivational intensity drove these neurophysiological effects remained unclear. To address this question, we combined a mindset manipulation with an imagery procedure to create and maintain three different affective states using a between-subjects design: a neutral mood, and positive mood with either high or low motivational intensity. After mood induction, 161 participants performed a simple gambling task while 64-channel EEG was recorded. FMθ activity results showed that irrespective of motivational intensity, positive compared with neutral mood altered reward expectancy. By comparison, RewP was not affected by positive mood nor motivational intensity. These results suggest that positive mood, rather than motivational intensity, is likely driving the change in reward expectation during gambling, which could reflect the presence of an optimistic bias. Moreover, at the methodological level, they confirm that the RewP ERP component and FMθ activity can capture dissociable effects during reward processing.
Collapse
|
17
|
Neural underpinnings of valence-action interactions triggered by cues and targets in a rewarded approach/avoidance task. Cortex 2021; 141:240-261. [PMID: 34098425 DOI: 10.1016/j.cortex.2021.04.013] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2020] [Revised: 08/31/2020] [Accepted: 04/28/2021] [Indexed: 02/07/2023]
Abstract
Incentive-valence signals have a large impact on our actions in everyday life. While it is intuitive (and most often beneficial) to approach positive and avoid negative stimuli, these prepotent response tendencies can also be maladaptive, as exemplified by clinical conditions such as overeating or pathological gambling. We have recently shown that targets associated with monetary incentives can trigger such valence-action biases (target condition), and that these are absent when valence and action information are provided by advance cues (cue condition). Here, we explored the neural correlates underlying the absence of the behavioral bias in this condition using fMRI. Specifically, we tested in how far valence and action information are integrated at all in the cue condition (where no behavioral biases are observed), assessing activity at the moment of the cue (mainly preparation) and the target (mainly implementation). The cue-locked data was dominated by main effects of valence with increased activity for incentive versus no-incentive cues in a network including anterior insula, premotor cortex, (mostly ventral) striatum (voxel-wise analysis), and across five predefined regions of interest (ROI analysis). Only one region, the anterior cingulate cortex, featured a valence-action interaction, with increased activity for win-approach compared to no-incentive-approach cues. The target-locked data revealed a different interaction pattern with increased activity in loss-approach as compared to win-approach targets in the cerebellum (voxel-wise) and across all ROIs. For comparison, the uncued target condition (target-locked data only) featured valence and action main effects (incentive > no-incentive targets; approach > avoid targets), but no interactions. The results resonate with the common observations that performance benefits after incentive-valence cues are promoted by increased preparatory control. Moreover, the data provide support for the idea that valence and action information are integrated according to an evolutionary benefit (cue-locked), requiring additional neural resources to implement non-intuitive valence-action mappings (target-locked).
Collapse
|
18
|
Midfrontal theta as moderator between beta oscillations and precision control. Neuroimage 2021; 235:118022. [PMID: 33836271 DOI: 10.1016/j.neuroimage.2021.118022] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2020] [Revised: 03/17/2021] [Accepted: 03/30/2021] [Indexed: 02/06/2023] Open
Abstract
Control of movements using visual information is crucial for many daily activities, and such visuomotor control has been revealed to be supported by alpha and beta cortical oscillations. However, it has been remained to be unclear how midfrontal theta and occipital gamma oscillations, which are associated with high-level cognitive functions, would be involved in this process to facilitate performance. Here we addressed this fundamental open question in healthy young adults by measuring high-density cortical activity during a precision force-matching task. We manipulated the amount of error by changing visual feedback gain (low, medium, and high visual gains) and analyzed event-related spectral perturbations. Increasing the visual feedback gain resulted in a decrease in force error and variability. There was an increase in theta synchronization in the midfrontal area and also in beta desynchronization in the sensorimotor and posterior parietal areas with higher visual feedback gains. Gamma de/synchronization was not evident during the task. In addition, we found a moderation effect of midfrontal theta on the positive relationship between the beta oscillations and force error. Subsequent simple slope analysis indicated that the effect of beta oscillations on force error was weaker when midfrontal theta was high. Our findings suggest that the midfrontal area signals the increased need of cognitive control to refine behavior by modulating the visuomotor processing at theta frequencies.
Collapse
|
19
|
Gershman SJ, Guitart-Masip M, Cavanagh JF. Neural signatures of arbitration between Pavlovian and instrumental action selection. PLoS Comput Biol 2021; 17:e1008553. [PMID: 33566831 PMCID: PMC7901778 DOI: 10.1371/journal.pcbi.1008553] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2020] [Revised: 02/23/2021] [Accepted: 11/23/2020] [Indexed: 01/10/2023] Open
Abstract
Pavlovian associations drive approach towards reward-predictive cues, and avoidance of punishment-predictive cues. These associations “misbehave” when they conflict with correct instrumental behavior. This raises the question of how Pavlovian and instrumental influences on behavior are arbitrated. We test a computational theory according to which Pavlovian influence will be stronger when inferred controllability of outcomes is low. Using a model-based analysis of a Go/NoGo task with human subjects, we show that theta-band oscillatory power in frontal cortex tracks inferred controllability, and that these inferences predict Pavlovian action biases. Functional MRI data revealed an inferior frontal gyrus correlate of action probability and a ventromedial prefrontal correlate of outcome valence, both of which were modulated by inferred controllability. Using a combination of computational modeling, neuroimaging (both EEG and fMRI), and behavioral analysis, we present evidence for a dual-process architecture in which Pavlovian and instrumental action values are adaptively combined through a Bayesian arbitration mechanism. Building on prior research, we find neural signatures of this arbitration mechanism in frontal cortex. In particular, we show that trial-by-trial changes in Pavlovian influences on action can be predicted by our computational model, and are reflected in midfrontal theta power, as well as inferior frontal and ventromedial prefrontal cortex fMRI responses.
Collapse
Affiliation(s)
- Samuel J Gershman
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, Massachusetts, United States of America.,Center for Brains, Minds and Machines, MIT, Cambridge, Massachusetts, United States of America
| | - Marc Guitart-Masip
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London, United Kingdom.,Aging Research Center, Karolinska Institute, Solna, Sweden
| | - James F Cavanagh
- Department of Psychology, University of New Mexico, Albuquerque, New Mexico, United States of America
| |
Collapse
|
20
|
Garofalo S, Battaglia S, Starita F, di Pellegrino G. Modulation of cue-guided choices by transcranial direct current stimulation. Cortex 2021; 137:124-137. [PMID: 33609898 DOI: 10.1016/j.cortex.2021.01.004] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2020] [Revised: 11/13/2020] [Accepted: 01/13/2021] [Indexed: 10/22/2022]
Abstract
Environmental cues may anticipate the availability of rewards, thus acting as a guide towards a specific choice (i.e., cue-guided choices). Despite the lateral prefrontal cortex having a critical role in using past learning and flexibly selecting relevant information to guide behavior, the literature on the neural basis of human cue-guided choice mainly focused on the subcortical brain structures implicated, while the specific role of cortical areas remained unclear. The present study aimed to provide causal evidence for the involvement of the lateral prefrontal cortex in two forms of human cue-guided choice, namely outcome-specific and general. To do this, 2 mA cathodal, anodal or sham transcranial direct current stimulation was applied over the lateral prefrontal cortex (with the posterior parietal cortex serving as control region) in three separate groups performing a Pavlovian-to-Instrumental Transfer task. Results showed, for the first time, a dissociation in the cortical structures involved in human cue-guided choice. Cathodal stimulation of the lateral prefrontal cortex reduced the outcome-specific transfer. In striking contrast, there was no influence on the general transfer. These results argue in favor of the presence of at least two possible neural pathways underlying cue-guided choices.
Collapse
Affiliation(s)
- Sara Garofalo
- Department of Psychology, University of Bologna, Cesena, Italy.
| | | | | | | |
Collapse
|
21
|
van Nuland AJ, Helmich RC, Dirkx MF, Zach H, Toni I, Cools R, den Ouden HEM. Effects of dopamine on reinforcement learning in Parkinson's disease depend on motor phenotype. Brain 2020; 143:3422-3434. [PMID: 33147621 PMCID: PMC7719026 DOI: 10.1093/brain/awaa335] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2019] [Revised: 07/10/2020] [Accepted: 08/06/2020] [Indexed: 01/16/2023] Open
Abstract
Parkinson's disease is clinically defined by bradykinesia, along with rigidity and tremor. However, the severity of these motor signs is greatly variable between individuals, particularly the presence or absence of tremor. This variability in tremor relates to variation in cognitive/motivational impairment, as well as the spatial distribution of neurodegeneration in the midbrain and dopamine depletion in the striatum. Here we ask whether interindividual heterogeneity in tremor symptoms could account for the puzzlingly large variability in the effects of dopaminergic medication on reinforcement learning, a fundamental cognitive function known to rely on dopamine. Given that tremor-dominant and non-tremor Parkinson's disease patients have different dopaminergic phenotypes, we hypothesized that effects of dopaminergic medication on reinforcement learning differ between tremor-dominant and non-tremor patients. Forty-three tremor-dominant and 20 non-tremor patients with Parkinson's disease were recruited to be tested both OFF and ON dopaminergic medication (200/50 mg levodopa-benserazide), while 22 age-matched control subjects were recruited to be tested twice OFF medication. Participants performed a reinforcement learning task designed to dissociate effects on learning rate from effects on motivational choice (i.e. the tendency to 'Go/NoGo' in the face of reward/threat of punishment). In non-tremor patients, dopaminergic medication improved reward-based choice, replicating previous studies. In contrast, in tremor-dominant patients, dopaminergic medication improved learning from punishment. Formal modelling showed divergent computational effects of dopaminergic medication as a function of Parkinson's disease motor phenotype, with a modulation of motivational choice bias and learning rate in non-tremor and tremor patients, respectively. This finding establishes a novel cognitive/motivational difference between tremor and non-tremor Parkinson's disease patients, and highlights the importance of considering motor phenotype in future work.
Collapse
Affiliation(s)
- Annelies J van Nuland
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, 6500 HB Nijmegen, The Netherlands
| | - Rick C Helmich
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, 6500 HB Nijmegen, The Netherlands
- Radboud University Medical Centre, Donders Institute for Brain, Cognition and Behaviour, Department of Neurology, 6500 HB Nijmegen, The Netherlands
| | - Michiel F Dirkx
- Radboud University Medical Centre, Donders Institute for Brain, Cognition and Behaviour, Department of Neurology, 6500 HB Nijmegen, The Netherlands
| | - Heidemarie Zach
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, 6500 HB Nijmegen, The Netherlands
- Radboud University Medical Centre, Donders Institute for Brain, Cognition and Behaviour, Department of Neurology, 6500 HB Nijmegen, The Netherlands
- Department of Neurology, Medical University Vienna, Vienna, Austria
| | - Ivan Toni
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, 6500 HB Nijmegen, The Netherlands
| | - Roshan Cools
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, 6500 HB Nijmegen, The Netherlands
- Radboud University Medical Center, Donders Institute for Brain, Cognition and Behaviour, Department of Psychiatry, Nijmegen, The Netherlands
| | - Hanneke E M den Ouden
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, 6500 HB Nijmegen, The Netherlands
| |
Collapse
|
22
|
Raab HA, Hartley CA. Adolescents exhibit reduced Pavlovian biases on instrumental learning. Sci Rep 2020; 10:15770. [PMID: 32978451 PMCID: PMC7519144 DOI: 10.1038/s41598-020-72628-w] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2020] [Accepted: 09/03/2020] [Indexed: 02/07/2023] Open
Abstract
Multiple learning systems allow individuals to flexibly respond to opportunities and challenges present in the environment. An evolutionarily conserved "Pavlovian" learning mechanism couples valence and action, promoting a tendency to approach cues associated with reward and to inhibit action in the face of anticipated punishment. Although this default response system may be adaptive, these hard-wired reactions can hinder the ability to learn flexible "instrumental" actions in pursuit of a goal. Such constraints on behavioral flexibility have been studied extensively in adults. However, the extent to which these valence-specific response tendencies bias instrumental learning across development remains poorly characterized. Here, we show that while Pavlovian response biases constrain flexible action learning in children and adults, these biases are attenuated in adolescents. This adolescent-specific reduction in Pavlovian bias may promote unbiased exploration of approach and avoidance responses, facilitating the discovery of rewarding behavior in the many novel contexts that adolescents encounter.
Collapse
Affiliation(s)
- Hillary A Raab
- Department of Psychology, New York University, New York, NY, USA
| | - Catherine A Hartley
- Department of Psychology, New York University, New York, NY, USA.
- Center for Neural Science, New York University, New York, NY, USA.
| |
Collapse
|
23
|
Proactive engagement of cognitive control modulates implicit approach-avoidance bias. COGNITIVE AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2020; 20:998-1010. [PMID: 32761312 DOI: 10.3758/s13415-020-00815-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Abstract
Implicit social-affective biases-reflected in a propensity to approach positive and avoid negative stimuli-have been documented in humans with paradigms, such as the Approach-Avoidance Task (AAT). However, the degree to which preemptively engaging cognitive control can help to down-regulate those behavioral tendencies remains poorly understood. While undergoing functional magnetic resonance imaging (fMRI), 24 healthy participants completed a cued version of the AAT, in which they responded to pictures of happy or angry faces by pulling a joystick toward themselves (approach) or pushing the joystick away (avoidance) based on the color of the stimulus frame. On some trials, they were cued to reverse the frame color/joystick action instructions. Before stimulus onset, a reverse cue was associated with deactivation of a visuo-spatial and motor planning network and subsequent slowing down in response to stimuli. During the stimulus phase, a reverse cue was associated with a) activation of cognitive control areas, including the right inferior frontal gyrus (IFG) and right inferior parietal lobule (IPL); and b) reduced right precentral gyrus activation when having to push (avoid) a happy face. Overall, these results suggest that proactively engaging cognitive control can help fine-tune behavioral and neural adjustment to emotionally incongruent behavioral conditions.
Collapse
|
24
|
Csifcsák G, Melsæter E, Mittner M. Intermittent Absence of Control during Reinforcement Learning Interferes with Pavlovian Bias in Action Selection. J Cogn Neurosci 2019; 32:646-663. [PMID: 31851595 DOI: 10.1162/jocn_a_01515] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
The ability to control the occurrence of rewarding and punishing events is crucial for our well-being. Two ways to optimize performance are to follow heuristics like Pavlovian biases to approach reward and avoid loss or to rely more on slowly accumulated stimulus-action associations. Although reduced control over outcomes has been linked to suboptimal decision-making in clinical conditions associated with learned helplessness, it is unclear how uncontrollability of the environment is related to the arbitration between different response strategies. This study directly tested whether a behavioral manipulation designed to induce learned helplessness in healthy adults (intermittent loss of control over feedback in a reinforcement learning task; "yoking") would modulate the magnitude of Pavlovian bias and the neurophysiological signature of cognitive control (frontal midline theta power) in healthy adults. Using statistical analysis and computational modeling of behavioral data and electroencephalographic signals, we found stronger Pavlovian influences and alterations in frontal theta activity in the yoked group. However, these effects were not accompanied by reduced performance in experimental blocks with regained control, indicating that our behavioral manipulation was not potent enough for inducing helplessness and impaired coping ability with task demands. We conclude that the level of contingency between instrumental choices and rewards/punishments modulates Pavlovian bias during value-based decision-making, probably via interfering with the implementation of cognitive control. These findings might have implications for understanding the mechanisms underlying helplessness in various psychiatric conditions.
Collapse
|
25
|
Sidarus N, Palminteri S, Chambon V. Cost-benefit trade-offs in decision-making and learning. PLoS Comput Biol 2019; 15:e1007326. [PMID: 31490934 PMCID: PMC6750595 DOI: 10.1371/journal.pcbi.1007326] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2018] [Revised: 09/18/2019] [Accepted: 08/08/2019] [Indexed: 11/18/2022] Open
Abstract
Value-based decision-making involves trading off the cost associated with an action against its expected reward. Research has shown that both physical and mental effort constitute such subjective costs, biasing choices away from effortful actions, and discounting the value of obtained rewards. Facing conflicts between competing action alternatives is considered aversive, as recruiting cognitive control to overcome conflict is effortful. Moreover, engaging control to proactively suppress irrelevant information that could conflict with task-relevant information would presumably also be cognitively costly. Yet, it remains unclear whether the cognitive control demands involved in preventing and resolving conflict also constitute costs in value-based decisions. The present study investigated this question by embedding irrelevant distractors (flanker arrows) within a reversal-learning task, with intermixed free and instructed trials. Results showed that participants learned to adapt their free choices to maximize rewards, but were nevertheless biased to follow the suggestions of irrelevant distractors. Thus, the perceived cost of investing cognitive control to suppress an external suggestion could sometimes trump internal value representations. By adapting computational models of reinforcement learning, we assessed the influence of conflict at both the decision and learning stages. Modelling the decision showed that free choices were more biased when participants were less sure about which action was more rewarding. This supports the hypothesis that the costs linked to conflict management were traded off against expected rewards. During the learning phase, we found that learning rates were reduced in instructed, relative to free, choices. Learning rates were further reduced by conflict between an instruction and subjective action values, whereas learning was not robustly influenced by conflict between one’s actions and external distractors. Our results show that the subjective cognitive control costs linked to conflict factor into value-based decision-making, and highlight that different types of conflict may have different effects on learning about action outcomes. Value-based decision-making involves trading off the cost associated with an action–such as physical or mental effort–against its expected reward. Although facing conflicts between competing action alternatives is considered aversive and effortful, it remains unclear whether conflict also constitutes a cost in value-based decisions. We tested this hypothesis by combining a classic conflict (flanker) task with a reinforcement-learning task. Results showed that participants learned to maximise their earnings, but were nevertheless biased to follow irrelevant suggestions. Computational model-based analyses showed a greater choice bias with more uncertainty about the best action to make, supporting the hypothesis that the costs linked to conflict management were traded off against expected rewards. We additionally found that learning rates were reduced when following instructions, relative to when choosing freely what to do. Learning was further reduced by conflict between instructions and subjective action values. In short, we found that the subjective cognitive control costs linked to conflict factor into value-based decision-making, and that different types of conflict may have different effects on learning about action outcomes.
Collapse
Affiliation(s)
- Nura Sidarus
- Institut Jean Nicod, Département d’Études Cognitives, École Normale Supérieure, EHESS, CNRS, PSL University, Paris, France
- Laboratoire de Neurosciences Cognitives Computationnelles, Département d’Études Cognitives, École Normale Supérieure, INSERM, PSL University, Paris, France
- Department of Psychology, Royal Holloway University of London, Surrey, United Kingdom
- * E-mail:
| | - Stefano Palminteri
- Laboratoire de Neurosciences Cognitives Computationnelles, Département d’Études Cognitives, École Normale Supérieure, INSERM, PSL University, Paris, France
| | - Valérian Chambon
- Institut Jean Nicod, Département d’Études Cognitives, École Normale Supérieure, EHESS, CNRS, PSL University, Paris, France
| |
Collapse
|