1
|
Algermissen J, den Ouden HEM. Pupil dilation reflects effortful action invigoration in overcoming aversive Pavlovian biases. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2024; 24:720-739. [PMID: 38773022 PMCID: PMC11233311 DOI: 10.3758/s13415-024-01191-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 04/22/2024] [Indexed: 05/23/2024]
Abstract
"Pavlovian" or "motivational" biases describe the phenomenon that the valence of prospective outcomes modulates action invigoration: Reward prospect invigorates action, whereas punishment prospect suppresses it. The adaptive role of these biases in decision-making is still unclear. One idea is that they constitute a fast-and-frugal decision strategy in situations characterized by high arousal, e.g., in presence of a predator, which demand a quick response. In this pre-registered study (N = 35), we tested whether such a situation-induced via subliminally presented angry versus neutral faces-leads to increased reliance on Pavlovian biases. We measured trial-by-trial arousal by tracking pupil diameter while participants performed an orthogonalized Motivational Go/NoGo Task. Pavlovian biases were present in responses, reaction times, and even gaze, with lower gaze dispersion under aversive cues reflecting "freezing of gaze." The subliminally presented faces did not affect responses, reaction times, or pupil diameter, suggesting that the arousal manipulation was ineffective. However, pupil dilations reflected facets of bias suppression, specifically the physical (but not cognitive) effort needed to overcome aversive inhibition: Particularly strong and sustained dilations occurred when participants managed to perform Go responses to aversive cues. Conversely, no such dilations occurred when they managed to inhibit responses to Win cues. These results suggest that pupil diameter does not reflect response conflict per se nor the inhibition of prepotent responses, but specifically effortful action invigoration as needed to overcome aversive inhibition. We discuss our results in the context of the "value of work" theory of striatal dopamine.
Collapse
Affiliation(s)
- Johannes Algermissen
- Donders Institute for Brain, Radboud University, Cognition, and Behaviour, Thomas van Aquinostraat 4, 6526 GD, Nijmegen, The Netherlands.
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Hanneke E M den Ouden
- Donders Institute for Brain, Radboud University, Cognition, and Behaviour, Thomas van Aquinostraat 4, 6526 GD, Nijmegen, The Netherlands.
| |
Collapse
|
2
|
Algermissen J, Swart JC, Scheeringa R, Cools R, den Ouden HEM. Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases. Nat Commun 2024; 15:19. [PMID: 38168089 PMCID: PMC10762147 DOI: 10.1038/s41467-023-44632-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 12/22/2023] [Indexed: 01/05/2024] Open
Abstract
Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.
Collapse
Affiliation(s)
- Johannes Algermissen
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| | - Jennifer C Swart
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - René Scheeringa
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, Essen, Germany
| | - Roshan Cools
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Department of Psychiatry, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Hanneke E M den Ouden
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| |
Collapse
|
3
|
Queirazza F, Steele JD, Krishnadas R, Cavanagh J, Philiastides MG. Functional Magnetic Resonance Imaging Signatures of Pavlovian and Instrumental Valuation Systems during a Modified Orthogonalized Go/No-go Task. J Cogn Neurosci 2023; 35:2089-2109. [PMID: 37788326 DOI: 10.1162/jocn_a_02062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/05/2023]
Abstract
Motivational (i.e., Pavlovian) values interfere with instrumental responding and can lead to suboptimal decision-making. In humans, task-based neuroimaging studies have only recently started illuminating the functional neuroanatomy of Pavlovian biasing of instrumental control. To provide a mechanistic understanding of the neural dynamics underlying the Pavlovian and instrumental valuation systems, analysis of neuroimaging data has been informed by computational modeling of conditioned behavior. Nonetheless, because of collinearities in Pavlovian and instrumental predictions, previous research failed to tease out hemodynamic activity that is parametrically and dynamically modulated by coexistent Pavlovian and instrumental value expectations. Moreover, neural correlates of Pavlovian to instrumental transfer effects have so far only been identified in extinction (i.e., in the absence of learning). In this study, we devised a modified version of the orthogonalized go/no-go paradigm, which introduced Pavlovian-only catch trials to better disambiguate trial-by-trial Pavlovian and instrumental predictions in both sexes. We found that hemodynamic activity in the ventromedial pFC covaried uniquely with the model-derived Pavlovian value expectations. Notably, modulation of neural activity encoding for instrumental predictions in the supplementary motor cortex was linked to successful action selection in conflict conditions. Furthermore, hemodynamic activity in regions pertaining to the limbic system and medial pFC was correlated with synergistic Pavlovian and instrumental predictions and improved conditioned behavior during congruent trials. Altogether, our results provide new insights into the functional neuroanatomy of decision-making and corroborate the validity of our variant of the orthogonalized go/no-go task as a behavioral assay of the Pavlovian and instrumental valuation systems.
Collapse
|
4
|
Möhring L, Gläscher J. Prediction errors drive dynamic changes in neural patterns that guide behavior. Cell Rep 2023; 42:112931. [PMID: 37540597 DOI: 10.1016/j.celrep.2023.112931] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 06/13/2023] [Accepted: 07/18/2023] [Indexed: 08/06/2023] Open
Abstract
Learning describes the process by which our internal expectation models of the world are updated by surprising outcomes (prediction errors [PEs]) to improve predictions of future events. However, the mechanisms through which error signals dynamically influence existing neural representations are unknown. Here, we use functional magnetic resonance imaging (fMRI) in humans solving a two-step Markov decision task to investigate changes in neural activation patterns following PEs. Using a dynamic multivariate pattern analysis, we can show that PE-related fMRI responses in error-coding regions predict trial-by-trial changes in multivariate neural patterns in the orbitofrontal cortex, the precuneus, and the ventromedial prefrontal cortex (vmPFC). Importantly, the dynamics of these pattern changes in the vmPFC also predicted upcoming changes in choice strategies and thus highlight the importance of these pattern changes for behavior.
Collapse
Affiliation(s)
- Leon Möhring
- Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246 Hamburg, Germany.
| | - Jan Gläscher
- Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246 Hamburg, Germany.
| |
Collapse
|
5
|
Sedlinská T, Bolte L, Melsæter E, Mittner M, Csifcsák G. Transcranial direct-current stimulation enhances Pavlovian tendencies during intermittent loss of control. Front Psychiatry 2023; 14:1164208. [PMID: 37229391 PMCID: PMC10203223 DOI: 10.3389/fpsyt.2023.1164208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/12/2023] [Accepted: 04/13/2023] [Indexed: 05/27/2023] Open
Abstract
Introduction Pavlovian bias is an innate motivational tendency to approach rewards and remain passive in the face of punishment. The relative reliance on Pavlovian valuation has been found to increase when the perceived control over environmental reinforcers is compromised, leading to behavior resembling learned helplessness (LH). Methods Sixty healthy young adults underwent a Go-NoGo reinforcement learning task and received anodal high-definition transcranial direct current stimulation (HD-tDCS) over the medial prefrontal/dorsal anterior cingulate cortex in our randomized, double-blind, sham- controlled study. Furthermore, we evaluated changes in cue-locked mid-frontal theta power derived from simultaneous electroencephalography (EEG). We hypothesized that active stimulation would reduce Pavlovian bias during manipulation of outcome controllability, and the effect would be accompanied by stronger mid-frontal theta activity, representing arbitration between choice strategies in favor of instrumental relative to Pavlovian valuation. Results We found a progressive decrease in Pavlovian bias during and after loss of control over feedback. Active HD-tDCS counteracted this effect while not affecting the mid-frontal theta signal. Discussion The results were at odds with our hypotheses but also with previous findings reporting LH-like patterns during and after loss of control without brain stimulation. The discrepancy may be related to different protocols for the controllability manipulation. We argue that the subjective evaluation of task controllability is crucial in mediating the balance between Pavlovian and instrumental valuation during reinforcement learning and that the medial prefrontal/dorsal anterior cingulate cortex is a key region in this respect. These findings have implications for understanding the behavioral and neural underpinnings of LH in humans.
Collapse
Affiliation(s)
- Terezie Sedlinská
- Department of Psychology, UiT The Arctic University of Norway, Tromsø, Norway
- Psychiatric University Hospital, Zürich, Switzerland
| | - Lara Bolte
- Department of Psychology, UiT The Arctic University of Norway, Tromsø, Norway
| | - Eirik Melsæter
- Department of Psychology, UiT The Arctic University of Norway, Tromsø, Norway
| | - Matthias Mittner
- Department of Psychology, UiT The Arctic University of Norway, Tromsø, Norway
| | - Gábor Csifcsák
- Department of Psychology, UiT The Arctic University of Norway, Tromsø, Norway
| |
Collapse
|
6
|
Pinger M, Thome J, Halli P, Sommer WH, Koppe G, Kirsch P. Comparing Discounting of Potentially Real Rewards and Losses by Means of Functional Magnetic Resonance Imaging. Front Syst Neurosci 2022; 16:867202. [PMID: 35965996 PMCID: PMC9365957 DOI: 10.3389/fnsys.2022.867202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Accepted: 06/13/2022] [Indexed: 11/13/2022] Open
Abstract
AimDelay discounting (DD) has often been investigated in the context of decision making whereby individuals attribute decreasing value to rewards in the distant future. Less is known about DD in the context of negative consequences. The aim of this pilot study was to identify commonalities and differences between reward and loss discounting on the behavioral as well as the neural level by means of computational modeling and functional Magnetic Resonance Imaging (fMRI). We furthermore compared the neural activation between anticipation of rewards and losses.MethodWe conducted a study combining an intertemporal choice task for potentially real rewards and losses (decision-making) with a monetary incentive/loss delay task (reward/loss anticipation). Thirty healthy participants (age 18-35, 14 female) completed the study. In each trial, participants had to choose between a smaller immediate loss/win and a larger loss/win at a fixed delay of two weeks. Task-related brain activation was measured with fMRI.ResultsHyperbolic discounting parameters of loss and reward conditions were correlated (r = 0.56). During decision-making, BOLD activation was observed in the parietal and prefrontal cortex, with no differences between reward and loss conditions. During reward and loss anticipation, dissociable activation was observed in the striatum, the anterior insula and the anterior cingulate cortex.ConclusionWe observed behavior concurrent with DD in both the reward and loss condition, with evidence for similar behavioral and neural patterns in the two conditions. Intertemporal decision-making recruited the fronto-parietal network, whilst reward and loss anticipation were related to activation in the salience network. The interpretation of these findings may be limited to short delays and small monetary outcomes.
Collapse
Affiliation(s)
- Mathieu Pinger
- Department of Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Heidelberg, Germany
- *Correspondence: Mathieu Pinger,
| | - Janine Thome
- Department of Theoretical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Heidelberg, Germany
- Clinic for Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Heidelberg, Germany
| | - Patrick Halli
- Department of Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Heidelberg, Germany
| | - Wolfgang H. Sommer
- Institute of Psychopharmacology, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Heidelberg, Germany
- Bethanien Hospital for Psychiatry, Psychosomatics and Psychotherapy, Greifswald, Germany
| | - Georgia Koppe
- Department of Theoretical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Heidelberg, Germany
- Clinic for Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Heidelberg, Germany
| | - Peter Kirsch
- Department of Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Heidelberg, Germany
- Department of Psychology, University of Heidelberg, Heidelberg, Germany
| |
Collapse
|
7
|
Scholz V, Hook RW, Kandroodi MR, Algermissen J, Ioannidis K, Christmas D, Valle S, Robbins TW, Grant JE, Chamberlain SR, den Ouden HEM. Cortical dopamine reduces the impact of motivational biases governing automated behaviour. Neuropsychopharmacology 2022; 47:1503-1512. [PMID: 35260787 PMCID: PMC9206002 DOI: 10.1038/s41386-022-01291-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 02/02/2022] [Indexed: 11/09/2022]
Abstract
Motivations shape our behaviour: the promise of reward invigorates, while in the face of punishment, we hold back. Abnormalities of motivational processing are implicated in clinical disorders characterised by excessive habits and loss of top-down control, notably substance and behavioural addictions. Striatal and frontal dopamine have been hypothesised to play complementary roles in the respective generation and control of these motivational biases. However, while dopaminergic interventions have indeed been found to modulate motivational biases, these previous pharmacological studies used regionally non-selective pharmacological agents. Here, we tested the hypothesis that frontal dopamine controls the balance between Pavlovian, bias-driven automated responding and instrumentally learned action values. Specifically, we examined whether selective enhancement of cortical dopamine either (i) enables adaptive suppression of Pavlovian control when biases are maladaptive; or (ii) non-specifically modulates the degree of bias-driven automated responding. Healthy individuals (n = 35) received the catechol-o-methyltransferase (COMT) inhibitor tolcapone in a randomised, double-blind, placebo-controlled cross-over design, and completed a motivational Go NoGo task known to elicit motivational biases. In support of hypothesis (ii), tolcapone globally decreased motivational bias. Specifically, tolcapone improved performance on trials where the bias was unhelpful, but impaired performance in bias-congruent conditions. These results indicate a non-selective role for cortical dopamine in the regulation of motivational processes underpinning top-down control over automated behaviour. The findings have direct relevance to understanding neurobiological mechanisms underpinning addiction and obsessive-compulsive disorders, as well as highlighting a potential trans-diagnostic novel mechanism to address such symptoms.
Collapse
Affiliation(s)
- Vanessa Scholz
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands. .,Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, Margarete-Höppel-Platz1, 97080, Würzburg, Germany.
| | - Roxanne W. Hook
- grid.5335.00000000121885934Department of Psychiatry, University of Cambridge, Cambridge, UK
| | - Mojtaba Rostami Kandroodi
- grid.5590.90000000122931605Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands ,grid.46072.370000 0004 0612 7950School of Electrical and Computer Engineering, University of Tehran, Tehran, Iran
| | - Johannes Algermissen
- grid.5590.90000000122931605Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| | - Konstantinos Ioannidis
- grid.5335.00000000121885934Department of Psychiatry, University of Cambridge, Cambridge, UK ,grid.450563.10000 0004 0412 9303Cambridgeshire & Peterborough NHS Foundation Trust, Cambridge, UK ,grid.5012.60000 0001 0481 6099Department of International Health, Care and Public Health Research Institute, Maastricht University, Maastricht, The Netherlands
| | - David Christmas
- grid.5335.00000000121885934Department of Psychiatry, University of Cambridge, Cambridge, UK ,grid.450563.10000 0004 0412 9303Cambridgeshire & Peterborough NHS Foundation Trust, Cambridge, UK
| | - Stephanie Valle
- grid.170205.10000 0004 1936 7822Department of Psychiatry and Behavioral Neuroscience, University of Chicago, Chicago, IL USA
| | - Trevor W. Robbins
- grid.5335.00000000121885934Department of Psychology, and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, UK
| | - Jon E. Grant
- grid.170205.10000 0004 1936 7822Department of Psychiatry and Behavioral Neuroscience, University of Chicago, Chicago, IL USA
| | - Samuel R. Chamberlain
- grid.5335.00000000121885934Department of Psychiatry, University of Cambridge, Cambridge, UK ,grid.5491.90000 0004 1936 9297Department of Psychiatry, Faculty of Medicine, University of Southampton, Southampton, UK. Southern Health NHS Foundation Trust, Southampton, UK
| | - Hanneke E. M. den Ouden
- grid.5590.90000000122931605Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| |
Collapse
|