1
|
Koch C, Zika O, Bruckner R, Schuck NW. Influence of surprise on reinforcement learning in younger and older adults. PLoS Comput Biol 2024; 20:e1012331. [PMID: 39141681 PMCID: PMC11346965 DOI: 10.1371/journal.pcbi.1012331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 08/26/2024] [Accepted: 07/16/2024] [Indexed: 08/16/2024] Open
Abstract
Surprise is a key component of many learning experiences, and yet its precise computational role, and how it changes with age, remain debated. One major challenge is that surprise often occurs jointly with other variables, such as uncertainty and outcome probability. To assess how humans learn from surprising events, and whether aging affects this process, we studied choices while participants learned from bandits with either Gaussian or bi-modal outcome distributions, which decoupled outcome probability, uncertainty, and surprise. A total of 102 participants (51 older, aged 50-73; 51 younger, 19-30 years) chose between three bandits, one of which had a bimodal outcome distribution. Behavioral analyses showed that both age-groups learned the average of the bimodal bandit less well. A trial-by-trial analysis indicated that participants performed choice reversals immediately following large absolute prediction errors, consistent with heightened sensitivity to surprise. This effect was stronger in older adults. Computational models indicated that learning rates in younger as well as older adults were influenced by surprise, rather than uncertainty, but also suggested large interindividual variability in the process underlying learning in our task. Our work bridges between behavioral economics research that has focused on how outcomes with low probability affect choice in older adults, and reinforcement learning work that has investigated age differences in the effects of uncertainty and suggests that older adults overly adapt to surprising events, even when accounting for probability and uncertainty effects.
Collapse
Affiliation(s)
- Christoph Koch
- Max Planck Institute for Human Development, Berlin, Germany
- Institute of Psychology, Universität Hamburg, Hamburg, Germany
| | - Ondrej Zika
- Max Planck Institute for Human Development, Berlin, Germany
- Max Planck UCL Centre for Computational Psychiatry and Aging Research, Berlin, Germany, and London, United Kingdom
| | - Rasmus Bruckner
- Max Planck Institute for Human Development, Berlin, Germany
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
| | - Nicolas W. Schuck
- Max Planck Institute for Human Development, Berlin, Germany
- Institute of Psychology, Universität Hamburg, Hamburg, Germany
- Max Planck UCL Centre for Computational Psychiatry and Aging Research, Berlin, Germany, and London, United Kingdom
| |
Collapse
|
2
|
Kabanova A, Fedorov L, Eschenko O. The Projection-Specific Noradrenergic Modulation of Perseverative Spatial Behavior in Adult Male Rats. eNeuro 2024; 11:ENEURO.0063-24.2024. [PMID: 39160074 PMCID: PMC11334950 DOI: 10.1523/eneuro.0063-24.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2024] [Revised: 06/10/2024] [Accepted: 06/17/2024] [Indexed: 08/21/2024] Open
Abstract
Adaptive behavior relies on efficient cognitive control. The anterior cingulate cortex (ACC) is a key node within the executive prefrontal network. The reciprocal connectivity between the locus ceruleus (LC) and ACC is thought to support behavioral reorganization triggered by the detection of an unexpected change. We transduced LC neurons with either excitatory or inhibitory chemogenetic receptors in adult male rats and trained rats on a spatial task. Subsequently, we altered LC activity and confronted rats with an unexpected change of reward locations. In a new spatial context, rats with decreased noradrenaline (NA) in the ACC entered unbaited maze arms more persistently which was indicative of perseveration. In contrast, the suppression of the global NA transmission reduced perseveration. Neither chemogenetic manipulation nor inactivation of the ACC by muscimol affected the rate of learning, possibly due to partial virus transduction of the LC neurons and/or the compensatory engagement of other prefrontal regions. Importantly, we observed behavioral deficits in rats with LC damage caused by virus injection. The latter finding highlights the importance of careful histological assessment of virus-transduced brain tissue as inadvertent damage of the targeted cell population due to virus neurotoxicity or other factors might cause unwanted side effects. Although the specific role of ACC in the flexibility of spatial behavior has not been convincingly demonstrated, our results support the beneficial role of noradrenergic transmission for an optimal function of the ACC. Overall, our findings suggest the LC exerts the projection-specific modulation of neural circuits mediating the flexibility of spatial behavior.
Collapse
Affiliation(s)
- Anna Kabanova
- Department of Physiology of Cognitive Processes, Max Planck Institute for Biological Cybernetics, 72076 Tübingen, Germany
| | - Leonid Fedorov
- Department of Physiology of Cognitive Processes, Max Planck Institute for Biological Cybernetics, 72076 Tübingen, Germany
| | - Oxana Eschenko
- Department of Physiology of Cognitive Processes, Max Planck Institute for Biological Cybernetics, 72076 Tübingen, Germany
| |
Collapse
|
3
|
Vázquez D, Peña-Flores N, Maulhardt SR, Solway A, Charpentier CJ, Roesch MR. Anterior cingulate cortex lesions impair multiple facets of task engagement not mediated by dorsomedial striatum neuron firing. Cereb Cortex 2024; 34:bhae332. [PMID: 39128939 DOI: 10.1093/cercor/bhae332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2024] [Revised: 07/22/2024] [Accepted: 07/25/2024] [Indexed: 08/13/2024] Open
Abstract
The anterior cingulate cortex (ACC) has been implicated across multiple highly specialized cognitive functions-including task engagement, motivation, error detection, attention allocation, value processing, and action selection. Here, we ask if ACC lesions disrupt task performance and firing in dorsomedial striatum (DMS) during the performance of a reward-guided decision-making task that engages many of these cognitive functions. We found that ACC lesions impacted several facets of task performance-including decreasing the initiation and completion of trials, slowing reaction times, and resulting in suboptimal and inaccurate action selection. Reductions in movement times towards the end of behavioral sessions further suggested attenuations in motivation, which paralleled reductions in directional action selection signals in the DMS that were observed later in recording sessions. Surprisingly, however, beyond altered action signals late in sessions-neural correlates in the DMS were largely unaffected, even though behavior was disrupted at multiple levels. We conclude that ACC lesions result in overall deficits in task engagement that impact multiple facets of task performance during our reward-guided decision-making task, which-beyond impacting motivated action signals-arise from dysregulated attentional signals in the ACC and are mediated via downstream targets other than DMS.
Collapse
Affiliation(s)
- Daniela Vázquez
- Department of Psychology, University of Maryland, College Park, Maryland 20742, United States
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742, United States
| | - Norma Peña-Flores
- Department of Psychology, University of Maryland, College Park, Maryland 20742, United States
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742, United States
| | - Sean R Maulhardt
- Department of Psychology, University of Maryland, College Park, Maryland 20742, United States
| | - Alec Solway
- Department of Psychology, University of Maryland, College Park, Maryland 20742, United States
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742, United States
| | - Caroline J Charpentier
- Department of Psychology, University of Maryland, College Park, Maryland 20742, United States
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742, United States
| | - Matthew R Roesch
- Department of Psychology, University of Maryland, College Park, Maryland 20742, United States
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742, United States
| |
Collapse
|
4
|
Hajnal MA, Tran D, Szabó Z, Albert A, Safaryan K, Einstein M, Vallejo Martelo M, Polack PO, Golshani P, Orbán G. Shifts in attention drive context-dependent subspace encoding in anterior cingulate cortex in mice during decision making. Nat Commun 2024; 15:5559. [PMID: 38956080 PMCID: PMC11220070 DOI: 10.1038/s41467-024-49845-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Accepted: 06/19/2024] [Indexed: 07/04/2024] Open
Abstract
Attention supports decision making by selecting the features that are relevant for decisions. Selective enhancement of the relevant features and inhibition of distractors has been proposed as potential neural mechanisms driving this selection process. Yet, how attention operates when relevance cannot be directly determined, and the attention signal needs to be internally constructed is less understood. Here we recorded from populations of neurons in the anterior cingulate cortex (ACC) of mice in an attention-shifting task where relevance of stimulus modalities changed across blocks of trials. In contrast with V1 recordings, decoding of the irrelevant modality gradually declined in ACC after an initial transient. Our analytical proof and a recurrent neural network model of the task revealed mutually inhibiting connections that produced context-gated suppression as observed in mice. Using this RNN model we predicted a correlation between contextual modulation of individual neurons and their stimulus drive, which we confirmed in ACC but not in V1.
Collapse
Affiliation(s)
- Márton Albert Hajnal
- Department of Computational Sciences, HUN-REN Wigner Research Centre for Physics, Budapest, Hungary.
| | - Duy Tran
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA
- Albert Einstein College of Medicine, New York, NY, USA
| | - Zsombor Szabó
- Department of Computational Sciences, HUN-REN Wigner Research Centre for Physics, Budapest, Hungary
| | - Andrea Albert
- Department of Computational Sciences, HUN-REN Wigner Research Centre for Physics, Budapest, Hungary
| | - Karen Safaryan
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA
| | - Michael Einstein
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA
| | - Mauricio Vallejo Martelo
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA
| | - Pierre-Olivier Polack
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, NJ, USA
| | - Peyman Golshani
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA.
- Integrative Center for Learning and Memory, Brain Research Institute, University of California, Los Angeles, Los Angeles, CA, USA.
- West Los Angeles VA Medical Center, Los Angeles, CA, USA.
| | - Gergő Orbán
- Department of Computational Sciences, HUN-REN Wigner Research Centre for Physics, Budapest, Hungary.
| |
Collapse
|
5
|
Ullsperger M. Beyond peaks and troughs: Multiplexed performance monitoring signals in the EEG. Psychophysiology 2024; 61:e14553. [PMID: 38415791 DOI: 10.1111/psyp.14553] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Revised: 02/08/2024] [Accepted: 02/10/2024] [Indexed: 02/29/2024]
Abstract
With the discovery of event-related potentials elicited by errors more than 30 years ago, a new avenue of research on performance monitoring, cognitive control, and decision making emerged. Since then, the field has developed and expanded fulminantly. After a brief overview on the EEG correlates of performance monitoring, this article reviews recent advancements based on single-trial analyses using independent component analysis, multiple regression, and multivariate pattern classification. Given the close interconnection between performance monitoring and reinforcement learning, computational modeling and model-based EEG analyses have made a particularly strong impact. The reviewed findings demonstrate that error- and feedback-related EEG dynamics represent variables reflecting how performance-monitoring signals are weighted and transformed into an adaptation signal that guides future decisions and actions. The model-based single-trial analysis approach goes far beyond conventional peak-and-trough analyses of event-related potentials and enables testing mechanistic theories of performance monitoring, cognitive control, and decision making.
Collapse
Affiliation(s)
- Markus Ullsperger
- Department of Neuropsychology, Otto-von-Guericke University Magdeburg, Magdeburg, Germany
- Center for Behavioral Brain Sciences, Magdeburg, Germany
- German Center for Mental Health (DZPG), partner site Halle-Jena-Magdeburg, Magdeburg, Germany
- Center for Intervention and Research on adaptive and maladaptive brain Circuits underlying mental health (C-I-R-C), Halle-Jena-Magdeburg, Germany
| |
Collapse
|
6
|
Chung WY, Darriba Á, Yeung N, Waszak F. Give it a second try? The influence of feedback and performance in the decision of reattempting. Cognition 2024; 248:105803. [PMID: 38703619 DOI: 10.1016/j.cognition.2024.105803] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 03/15/2024] [Accepted: 04/23/2024] [Indexed: 05/06/2024]
Abstract
Feedback evaluation can affect behavioural continuation or discontinuation, and is essential for cognitive and motor skill learning. One critical factor that influences feedback evaluation is participants' internal estimation of self-performance. Previous research has shown that two event-related potential components, the Feedback-Related Negativity (FRN) and the P3, are related to feedback evaluation. In the present study, we used a time estimation task and EEG recordings to test the influence of feedback and performance on participants' decisions, and the sensitivity of the FRN and P3 components to those factors. In the experiment, participants were asked to reproduce the total duration of an intermittently presented visual stimulus. Feedback was given after every response, and participants had then to decide whether to retry the same trial and try to earn reward points, or to move on to the next trial. Results showed that both performance and feedback influenced participants' decision on whether to retry the ongoing trial. In line with previous studies, the FRN showed larger amplitude in response to negative than to positive feedback. Moreover, our results were also in agreement with previous works showing the relationship between the amplitude of the FRN and the size of feedback-related prediction error (PE), and provide further insight in how PE size influences participants' decisions on whether or not to retry a task. Specifically, we found that the larger the FRN, the more likely participants were to base their decision on their performance - choosing to retry the current trial after good performance or to move on to the next trial after poor performance, regardless of the feedback received. Conversely, the smaller the FRN, the more likely participants were to base their decision on the feedback received.
Collapse
Affiliation(s)
- Wai Ying Chung
- Université Paris Cité, INCC UMR 8002, CNRS, F-75006 Paris, France.
| | - Álvaro Darriba
- Université Paris Cité, INCC UMR 8002, CNRS, F-75006 Paris, France
| | - Nick Yeung
- Department of Experimental Psychology, University of Oxford, Anna Watts Building, Radcliffe Observatory Quarter, Woodstock Road, Oxford OX2 6GG, United Kingdom
| | - Florian Waszak
- Université Paris Cité, INCC UMR 8002, CNRS, F-75006 Paris, France
| |
Collapse
|
7
|
Treuting RL, Boroujeni KB, Gerrity CG, Tiesinga P, Womelsdorf T. Anterior Cingulate Cortex Causally Supports Meta-Learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.06.12.598723. [PMID: 38915609 PMCID: PMC11195175 DOI: 10.1101/2024.06.12.598723] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/26/2024]
Abstract
In dynamic environments with volatile rewards the anterior cingulate cortex (ACC) is believed to determine whether a visual object is relevant and should be chosen. The ACC may achieve this by integrating reward information over time to estimate which objects are worth to explore and which objects should be avoided. Such a higher-order meta-awareness about which objects should be explored predicts that the ACC causally contributes to choices when the reward values of objects are unknown and must be inferred from ongoing exploration. We tested this suggestion in nonhuman primates using a learning task that varied the number of object features that could be relevant, and by controlling the motivational value of choosing objects. During learning the ACC was transiently micro-stimulated when subjects foveated the to-be-chosen stimulus. We found that stimulation selectively impaired learning when feature uncertainty and motivational value of choices were high, which was linked to a deficit in using reward outcomes for feature-specific credit assignment. Application of an adaptive reinforcement learning model confirmed a primary deficit in weighting prediction errors that led to a meta-learning impairment to adaptively increase exploration during learning and to an impaired use of working memory to support learning. These findings provide causal evidence that the reward history traces in ACC are essential for meta-adjusting the exploration-exploitation balance and the strength of working memory of object values during adaptive behavior.
Collapse
Affiliation(s)
- Robert Louis Treuting
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN 37240
- Vanderbilt Brain Institute, Nashville, TN 372404
| | - Kianoush Banaie Boroujeni
- Department of Psychology, Vanderbilt University, Nashville, TN 37240
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08540
| | - Charles Grimes Gerrity
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37240
| | - Paul Tiesinga
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen 6525 EN, Netherlands
| | - Thilo Womelsdorf
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN 37240
- Vanderbilt Brain Institute, Nashville, TN 372404
- Department of Psychology, Vanderbilt University, Nashville, TN 37240
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37240
| |
Collapse
|
8
|
Vázquez D, Maulhardt SR, Stalnaker TA, Solway A, Charpentier CJ, Roesch MR. Optogenetic Inhibition of Rat Anterior Cingulate Cortex Impairs the Ability to Initiate and Stay on Task. J Neurosci 2024; 44:e1850232024. [PMID: 38569923 PMCID: PMC11097287 DOI: 10.1523/jneurosci.1850-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Revised: 01/16/2024] [Accepted: 01/20/2024] [Indexed: 04/05/2024] Open
Abstract
Our prior research has identified neural correlates of cognitive control in the anterior cingulate cortex (ACC), leading us to hypothesize that the ACC is necessary for increasing attention as rats flexibly learn new contingencies during a complex reward-guided decision-making task. Here, we tested this hypothesis by using optogenetics to transiently inhibit the ACC, while rats of either sex performed the same two-choice task. ACC inhibition had a profound impact on behavior that extended beyond deficits in attention during learning when expected outcomes were uncertain. We found that ACC inactivation slowed and reduced the number of trials rats initiated and impaired both their accuracy and their ability to complete sessions. Furthermore, drift-diffusion model analysis suggested that free-choice performance and evidence accumulation (i.e., reduced drift rates) were degraded during initial learning-leading to weaker associations that were more easily overridden in later trial blocks (i.e., stronger bias). Together, these results suggest that in addition to attention-related functions, the ACC contributes to the ability to initiate trials and generally stay on task.
Collapse
Affiliation(s)
- Daniela Vázquez
- Department of Psychology, University of Maryland, College Park, Maryland 20742
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742
| | - Sean R Maulhardt
- Department of Psychology, University of Maryland, College Park, Maryland 20742
| | - Thomas A Stalnaker
- Intramural Research Program, National Institute on Drug Abuse, Baltimore, Maryland 21224
| | - Alec Solway
- Department of Psychology, University of Maryland, College Park, Maryland 20742
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742
| | - Caroline J Charpentier
- Department of Psychology, University of Maryland, College Park, Maryland 20742
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742
| | - Matthew R Roesch
- Department of Psychology, University of Maryland, College Park, Maryland 20742
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742
| |
Collapse
|
9
|
Alejandro RJ, Holroyd CB. Hierarchical control over foraging behavior by anterior cingulate cortex. Neurosci Biobehav Rev 2024; 160:105623. [PMID: 38490499 DOI: 10.1016/j.neubiorev.2024.105623] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Revised: 02/14/2024] [Accepted: 03/13/2024] [Indexed: 03/17/2024]
Abstract
Foraging is a natural behavior that involves making sequential decisions to maximize rewards while minimizing the costs incurred when doing so. The prevalence of foraging across species suggests that a common brain computation underlies its implementation. Although anterior cingulate cortex is believed to contribute to foraging behavior, its specific role has been contentious, with predominant theories arguing either that it encodes environmental value or choice difficulty. Additionally, recent attempts to characterize foraging have taken place within the reinforcement learning framework, with increasingly complex models scaling with task complexity. Here we review reinforcement learning foraging models, highlighting the hierarchical structure of many foraging problems. We extend this literature by proposing that ACC guides foraging according to principles of model-based hierarchical reinforcement learning. This idea holds that ACC function is organized hierarchically along a rostral-caudal gradient, with rostral structures monitoring the status and completion of high-level task goals (like finding food), and midcingulate structures overseeing the execution of task options (subgoals, like harvesting fruit) and lower-level actions (such as grabbing an apple).
Collapse
Affiliation(s)
| | - Clay B Holroyd
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| |
Collapse
|
10
|
Mah A, Golden C, Constantinople C. Mesolimbic dopamine encodes reward prediction errors independent of learning rates. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.18.590090. [PMID: 38659861 PMCID: PMC11042285 DOI: 10.1101/2024.04.18.590090] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/26/2024]
Abstract
Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. These values are thought to be represented in synaptic weights in the striatum, and updated by dopamine-dependent plasticity, suggesting that dopamine release might reflect the product of the learning rate and RPE. Here, we leveraged the fact that animals learn faster in volatile environments to characterize dopamine encoding of learning rates. We trained rats on a task with semi-observable states offering different rewards, and rats adjusted how quickly they initiated trials across states using RPEs. Computational modeling and behavioral analyses showed that learning rates were higher following state transitions, and scaled with trial-by-trial changes in beliefs about hidden states, approximating normative Bayesian strategies. Notably, dopamine release in the nucleus accumbens encoded RPEs independent of learning rates, suggesting that dopamine-independent mechanisms instantiate dynamic learning rates.
Collapse
Affiliation(s)
- Andrew Mah
- Center for Neural Science, New York University
| | | | | |
Collapse
|
11
|
De Filippo R, Schmitz D. Synthetic surprise as the foundation of the psychedelic experience. Neurosci Biobehav Rev 2024; 157:105538. [PMID: 38220035 PMCID: PMC10839673 DOI: 10.1016/j.neubiorev.2024.105538] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 01/03/2024] [Accepted: 01/05/2024] [Indexed: 01/16/2024]
Abstract
Psychedelic agents, such as LSD and psilocybin, induce marked alterations in consciousness via activation of the 5-HT2A receptor (5-HT2ARs). We hypothesize that psychedelics enforce a state of synthetic surprise through the biased activation of the 5-HTRs system. This idea is informed by recent insights into the role of 5-HT in signaling surprise. The effects on consciousness, explained by the cognitive penetrability of perception, can be described within the predictive coding framework where surprise corresponds to prediction error, the mismatch between predictions and actual sensory input. Crucially, the precision afforded to the prediction error determines its effect on priors, enabling a dynamic interaction between top-down expectations and incoming sensory data. By integrating recent findings on predictive coding circuitry and 5-HT2ARs transcriptomic data, we propose a biological implementation with emphasis on the role of inhibitory interneurons. Implications arise for the clinical use of psychedelics, which may rely primarily on their inherent capacity to induce surprise in order to disrupt maladaptive patterns.
Collapse
Affiliation(s)
- Roberto De Filippo
- Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität Berlin, and Berlin Institute of Health, Neuroscience Research Center, 10117 Berlin, Germany.
| | - Dietmar Schmitz
- Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität Berlin, and Berlin Institute of Health, Neuroscience Research Center, 10117 Berlin, Germany; German Center for Neurodegenerative Diseases (DZNE) Berlin, 10117 Berlin, Germany; Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität Berlin, and Berlin Institute of Health, Einstein Center for Neuroscience, 10117 Berlin, Germany; Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität Berlin, and Berlin Institute of Health, NeuroCure Cluster of Excellence, 10117 Berlin, Germany; Humboldt-Universität zu Berlin, Bernstein Center for Computational Neuroscience, Philippstr. 13, 10115 Berlin, Germany
| |
Collapse
|
12
|
Jordan R. The locus coeruleus as a global model failure system. Trends Neurosci 2024; 47:92-105. [PMID: 38102059 DOI: 10.1016/j.tins.2023.11.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 09/27/2023] [Accepted: 11/20/2023] [Indexed: 12/17/2023]
Abstract
Predictive processing models posit that brains constantly attempt to predict their sensory inputs. Prediction errors signal when these predictions are incorrect and are thought to be instructive signals that drive corrective plasticity. Recent findings support the idea that the locus coeruleus (LC) - a brain-wide neuromodulatory system - signals several types of prediction error. I discuss how these findings support models proposing that the LC signals global model failures: instances where predictions about the world are strongly violated. Focusing on the cortex, I explore the utility of this signal in learning rate control, how the LC circuit may compute the signal, and how this view may aid our understanding of neurodivergence.
Collapse
Affiliation(s)
- Rebecca Jordan
- Simons Initiative for the Developing Brain, University of Edinburgh, 1 George Square, EH8 9JZ, Edinburgh, UK.
| |
Collapse
|
13
|
Drew A, Soto-Faraco S. Perceptual oddities: assessing the relationship between film editing and prediction processes. Philos Trans R Soc Lond B Biol Sci 2024; 379:20220426. [PMID: 38104604 PMCID: PMC10725757 DOI: 10.1098/rstb.2022.0426] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Accepted: 10/16/2023] [Indexed: 12/19/2023] Open
Abstract
During film viewing, humans parse sequences of individual shots into larger narrative structures, often weaving transitions at edit points into an apparently seamless and continuous flow. Editing helps filmmakers manipulate visual transitions to induce feelings of fluency/disfluency, tension/relief, curiosity, expectation and several emotional responses. We propose that the perceptual dynamics induced by film editing can be captured by a predictive processing (PP) framework. We hypothesise that visual discontinuities at edit points produce discrepancies between anticipated and actual sensory input, leading to prediction error. Further, we propose that the magnitude of prediction error depends on the predictability of each shot within the narrative flow, and lay out an account based on conflict monitoring. We test this hypothesis in two empirical studies measuring electroencephalography (EEG) during passive viewing of film excerpts, as well as behavioural responses during an active edit detection task. We report the neural and behavioural modulations at editing boundaries across three levels of narrative depth, showing greater modulations for edits spanning less predictable, deeper narrative transitions. Overall, our contribution lays the groundwork for understanding film editing from a PP perspective. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectivess'.
Collapse
Affiliation(s)
- Alice Drew
- Multisensory Research Group, Centre for Brain and Cognition, Universitat Pompeu Fabra, Carrer de Ramon Trias Fargas, 25-27, 08005 Barcelona, Spain
| | - Salvador Soto-Faraco
- Multisensory Research Group, Centre for Brain and Cognition, Universitat Pompeu Fabra, Carrer de Ramon Trias Fargas, 25-27, 08005 Barcelona, Spain
- Institució Catalana de Recerca i Estudis Avançats (ICREA), 08010 Barcelona, Spain
| |
Collapse
|
14
|
Hoy CW, Quiroga-Martinez DR, Sandoval E, King-Stephens D, Laxer KD, Weber P, Lin JJ, Knight RT. Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex. Nat Commun 2023; 14:8520. [PMID: 38129440 PMCID: PMC10739882 DOI: 10.1038/s41467-023-44248-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2022] [Accepted: 12/05/2023] [Indexed: 12/23/2023] Open
Abstract
The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.
Collapse
Affiliation(s)
- Colin W Hoy
- Department of Neurology, University of California, San Francisco, San Francisco, CA, USA.
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA.
| | - David R Quiroga-Martinez
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
- Center for Music in the Brain, Aarhus University & The Royal Academy of Music, Aarhus, Denmark
| | - Eduardo Sandoval
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
| | - David King-Stephens
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
- Department of Neurology, Yale School of Medicine, New Haven, CT, USA
| | - Kenneth D Laxer
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
| | - Peter Weber
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
| | - Jack J Lin
- Department of Neurology, University of California, Davis, Davis, CA, USA
- Center for Mind and Brain, University of California, Davis, Davis, CA, USA
| | - Robert T Knight
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
- Department of Psychology, University of California, Berkeley, Berkeley, CA, USA
| |
Collapse
|
15
|
Clairis N, Lopez-Persem A. Debates on the dorsomedial prefrontal/dorsal anterior cingulate cortex: insights for future research. Brain 2023; 146:4826-4844. [PMID: 37530487 PMCID: PMC10690029 DOI: 10.1093/brain/awad263] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Revised: 07/19/2023] [Accepted: 07/22/2023] [Indexed: 08/03/2023] Open
Abstract
The dorsomedial prefrontal cortex/dorsal anterior cingulate cortex (dmPFC/dACC) is a brain area subject to many theories and debates over its function(s). Even its precise anatomical borders are subject to much controversy. In the past decades, the dmPFC/dACC has been associated with more than 15 different cognitive processes, which sometimes appear quite unrelated (e.g. body perception, cognitive conflict). As a result, understanding what the dmPFC/dACC does has become a real challenge for many neuroscientists. Several theories of this brain area's function(s) have been developed, leading to successive and competitive publications bearing different models, which sometimes contradict each other. During the last two decades, the lively scientific exchanges around the dmPFC/dACC have promoted fruitful research in cognitive neuroscience. In this review, we provide an overview of the anatomy of the dmPFC/dACC, summarize the state of the art of functions that have been associated with this brain area and present the main theories aiming at explaining the dmPFC/dACC function(s). We explore the commonalities and the arguments between the different theories. Finally, we explain what can be learned from these debates for future investigations of the dmPFC/dACC and other brain regions' functions.
Collapse
Affiliation(s)
- Nicolas Clairis
- Laboratory of Behavioral Genetics (LGC)- Brain Mind Institute (BMI)- Sciences de la Vie (SV), École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland
| | - Alizée Lopez-Persem
- FrontLab, Institut du Cerveau - Paris Brain Institute - ICM, Inserm, CNRS, Sorbonne University, AP HP, Hôpital de la Pitié Salpêtrière, 75013 Paris, France
| |
Collapse
|
16
|
Zhang T, Zhou S, Bai X, Zhou F, Zhai Y, Long Y, Lu C. Neurocomputations on dual-brain signals underlie interpersonal prediction during a natural conversation. Neuroimage 2023; 282:120400. [PMID: 37783363 DOI: 10.1016/j.neuroimage.2023.120400] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 09/21/2023] [Accepted: 09/30/2023] [Indexed: 10/04/2023] Open
Abstract
Prediction on the partner's speech plays a key role in a smooth conversation. However, previous studies on this issue have been majorly conducted at the single-brain rather than dual-brain level, leaving the interpersonal prediction hypothesis untested. To fill this gap, this study combined a neurocomputational modeling approach with a natural conversation paradigm in which two salespersons persuaded a customer to buy their product with their haemodynamic signals being collected using functional near-infrared spectroscopy hyperscanning. First, the results showed a cognitive hierarchy in a natural conversation, with the lower-level process (i.e., pragmatic representation of the persuasion) in the salesperson interacting with the higher-level process (i.e., value representation of the product) in the customer. Next, we found that the right dorsal lateral prefrontal cortex (rdlPFC) and temporoparietal junction (rTPJ) were associated with the representation of the product's value in the customer, while the right inferior frontal cortex (rIFC) was associated with the representation of the pragmatic processes in the salesperson. Finally, neurocomputational modeling results supported the prediction of the salesperson's lower-level brain activity based on the customer's higher-level brain activity. Moreover, the updating weight of the prediction model based on the neural computation between the rIFC of the salesperson and the rTPJ of the customer was closely associated with the interaction context, whereas that based on the rIFC-rdlPFC was not. In summary, these findings provide initial support for the interpersonal prediction hypothesis at the dual-brain level and reveal a hierarchy for the interpersonal prediction process.
Collapse
Affiliation(s)
- Tengfei Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Siyuan Zhou
- Institute of Brain and Psychological Sciences, Sichuan Normal University, Chengdu 610066, PR China
| | - Xialu Bai
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Faxin Zhou
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Yu Zhai
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Yuhang Long
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Chunming Lu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China.
| |
Collapse
|
17
|
Kim D, Wang Z, Sakagami M, Sasaki Y, Watanabe T. Only cortical prediction error signals are involved in visual learning, despite availability of subcortical prediction error signals. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.11.13.566726. [PMID: 38014275 PMCID: PMC10680585 DOI: 10.1101/2023.11.13.566726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/29/2023]
Abstract
Both the midbrain systems, encompassing the ventral striatum (VS), and the cortical systems, including the dorsal anterior cingulate cortex (dACC), play roles in reinforcing and enhancing learning. However, the specific contributions of signals from these regions in learning remains unclear. To investigate this, we examined how VS and dACC are involved in visual perceptual learning (VPL) through an orientation discrimination task. In the primary experiment, subjects fasted for 5 hours before each of 14 days of training sessions and 3 days of test sessions. Subjects were rewarded with water for accurate trial responses. During the test sessions, BOLD signals were recorded from regions including VS and dACC. Although BOLD signals in both areas were associated with positive and negative RPEs, only those in dACC associated with negative RPE showed a significant correlation with performance improvement. Additionally, no significant correlation was observed between BOLD signals associated with RPEs in VS and dACC. These results suggest that although signals associated with positive and negative RPEs from both midbrain and cortical systems are readily accessible, only RPE signals in the prefrontal system, generated without linking to RPE signals in VS, are utilized for the enhancement of VPL.
Collapse
|
18
|
Ryskin R, Nieuwland MS. Prediction during language comprehension: what is next? Trends Cogn Sci 2023; 27:1032-1052. [PMID: 37704456 DOI: 10.1016/j.tics.2023.08.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 08/03/2023] [Accepted: 08/04/2023] [Indexed: 09/15/2023]
Abstract
Prediction is often regarded as an integral aspect of incremental language comprehension, but little is known about the cognitive architectures and mechanisms that support it. We review studies showing that listeners and readers use all manner of contextual information to generate multifaceted predictions about upcoming input. The nature of these predictions may vary between individuals owing to differences in language experience, among other factors. We then turn to unresolved questions which may guide the search for the underlying mechanisms. (i) Is prediction essential to language processing or an optional strategy? (ii) Are predictions generated from within the language system or by domain-general processes? (iii) What is the relationship between prediction and memory? (iv) Does prediction in comprehension require simulation via the production system? We discuss promising directions for making progress in answering these questions and for developing a mechanistic understanding of prediction in language.
Collapse
Affiliation(s)
- Rachel Ryskin
- Department of Cognitive and Information Sciences, University of California Merced, 5200 Lake Road, Merced, CA 95343, USA.
| | - Mante S Nieuwland
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands; Donders Institute for Brain, Cognition, and Behaviour, Nijmegen, The Netherlands
| |
Collapse
|
19
|
Jee HJ, Zhu E, Sun M, Liu W, Zhang Q, Wang J. Anterior cingulate cortex regulates pain catastrophizing-like behaviors in rats. Mol Brain 2023; 16:71. [PMID: 37833814 PMCID: PMC10576271 DOI: 10.1186/s13041-023-01060-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Accepted: 09/19/2023] [Indexed: 10/15/2023] Open
Abstract
Negative pain expectation including pain catastrophizing is a well-known clinical phenomenon whereby patients amplify the aversive value of a painful or oftentimes even a similar, non-painful stimulus. Mechanisms of pain catastrophizing, however, remain elusive. Here, we modeled pain catastrophizing behavior in rats, and found that rats subjected to repeated noxious pin pricks on one paw demonstrated an aversive response to similar but non-noxious mechanical stimuli delivered to the contralateral paw. Optogenetic inhibition of pyramidal neuron activity in the anterior cingulate cortex (ACC) during the application of repetitive noxious pin pricks eliminated this catastrophizing behavior. Time-lapse calcium (Ca2+) imaging in the ACC further revealed an increase in spontaneous neural activity after the delivery of noxious stimuli. Together these results suggest that the experience of repeated noxious stimuli may drive hyperactivity in the ACC, causing increased avoidance of subthreshold stimuli, and that reducing this hyperactivity may play a role in treating pain catastrophizing.
Collapse
Affiliation(s)
- Hyun Jung Jee
- Department of Anesthesiology, Perioperative Care and Pain Medicine, New York University Grossman School of Medicine, New York, NY, 10016, USA
- Interdisciplinary Pain Research Program, New York University Langone Health, New York, NY, 10016, USA
| | - Elaine Zhu
- Department of Anesthesiology, Perioperative Care and Pain Medicine, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Mengqi Sun
- Department of Anesthesiology, Perioperative Care and Pain Medicine, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Weizhuo Liu
- Department of Anesthesiology, Perioperative Care and Pain Medicine, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Qiaosheng Zhang
- Department of Anesthesiology, Perioperative Care and Pain Medicine, New York University Grossman School of Medicine, New York, NY, 10016, USA.
- Interdisciplinary Pain Research Program, New York University Langone Health, New York, NY, 10016, USA.
| | - Jing Wang
- Department of Anesthesiology, Perioperative Care and Pain Medicine, New York University Grossman School of Medicine, New York, NY, 10016, USA.
- Interdisciplinary Pain Research Program, New York University Langone Health, New York, NY, 10016, USA.
- Department of Neuroscience and Physiology, New York University Grossman School of Medicine, New York, NY, 10016, USA.
- Neuroscience Institute, New York University Grossman School of Medicine, New York, NY, 10016, USA.
| |
Collapse
|
20
|
Hajnal MA, Tran D, Szabó Z, Albert A, Safaryan K, Einstein M, Martelo MV, Polack PO, Golshani P, Orbán G. Shifts in attention drive context-dependent subspace encoding in anterior cingulate cortex during decision making. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.10.10.561737. [PMID: 37873364 PMCID: PMC10592791 DOI: 10.1101/2023.10.10.561737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/25/2023]
Abstract
Attention is a cognitive faculty that selects part of a larger set of percepts, driven by cues such as stimulus saliency, internal goals or priors. The enhancement of the attended representation and inhibition of distractors have been proposed as potential neural mechanisms driving this selection process. Yet, how attention operates when the cue has to be internally constructed from conflicting stimuli, decision rules, and reward contingencies, is less understood. Here we recorded from populations of neurons in the anterior cingulate cortex (ACC), an area implicated in ongoing error monitoring and correction during decision conflicts, in a challenging attention-shifting task. In this task, mice had to attend to the rewarded modality when presented identical auditory and visual stimuli in two contexts without direct external cues. In the ACC, the irrelevant stimulus continuously became less decodable than the relevant stimulus as the trial progressed to the decision point. This contrasted strongly with our previous findings in V1 where both relevant and irrelevant stimuli were equally decodable throughout the trial. Using analytical tools and a recurrent neural network (RNN) model, we found that the linearly independent representation of stimulus modalities in ACC was well suited to context-gated suppression of a stimulus modality. We demonstrated that the feedback structure of lateral connections in the RNN consisted of excitatory interactions between cell ensembles representing the same modality and mutual inhibition between cell ensembles representing distinct stimulus modalities. Using this RNN model showing signatures of context-gated suppression, we predicted that the level of contextual modulation of individual neurons should be correlated with their relative responsiveness to the two stimulus modalities used in the task. We verified this prediction in recordings from ACC neurons but not from recordings from V1 neurons. Therefore, ACC effectively operates on low-dimensional neuronal subspaces to combine stimulus related information with internal cues to drive actions under conflict.
Collapse
Affiliation(s)
- Márton Albert Hajnal
- Department of Computational Sciences, Wigner Research Center for Physics, Budapest, 1121, Hungary
| | - Duy Tran
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA 90095, United States
- Albert Einstein College of Medicine, New York, NY 10461, United States
| | - Zsombor Szabó
- Department of Computational Sciences, Wigner Research Center for Physics, Budapest, 1121, Hungary
| | - Andrea Albert
- Department of Computational Sciences, Wigner Research Center for Physics, Budapest, 1121, Hungary
| | - Karen Safaryan
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA 90095, United States
| | - Michael Einstein
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA 90095, United States
| | - Mauricio Vallejo Martelo
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA 90095, United States
| | - Pierre-Olivier Polack
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, NJ 07102, United States
| | - Peyman Golshani
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA 90095, United States
- Integrative Center for Learning and Memory, Brain Research Institute, University of California, Los Angeles, Los Angeles, CA 90095, United States
- West Los Angeles VA Medical Center, CA 90073 Los Angeles, United States
| | - Gergő Orbán
- Department of Computational Sciences, Wigner Research Center for Physics, Budapest, 1121, Hungary
| |
Collapse
|
21
|
Modirshanechi A, Becker S, Brea J, Gerstner W. Surprise and novelty in the brain. Curr Opin Neurobiol 2023; 82:102758. [PMID: 37619425 DOI: 10.1016/j.conb.2023.102758] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Revised: 06/30/2023] [Accepted: 07/20/2023] [Indexed: 08/26/2023]
Abstract
Notions of surprise and novelty have been used in various experimental and theoretical studies across multiple brain areas and species. However, 'surprise' and 'novelty' refer to different quantities in different studies, which raises concerns about whether these studies indeed relate to the same functionalities and mechanisms in the brain. Here, we address these concerns through a systematic investigation of how different aspects of surprise and novelty relate to different brain functions and physiological signals. We review recent classifications of definitions proposed for surprise and novelty along with links to experimental observations. We show that computational modeling and quantifiable definitions enable novel interpretations of previous findings and form a foundation for future theoretical and experimental studies.
Collapse
Affiliation(s)
- Alireza Modirshanechi
- Brain-Mind Institute, School of Life Sciences, EPFL, Lausanne, Switzerland; School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland.
| | - Sophia Becker
- Brain-Mind Institute, School of Life Sciences, EPFL, Lausanne, Switzerland; School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland. https://twitter.com/sophiabecker95
| | - Johanni Brea
- Brain-Mind Institute, School of Life Sciences, EPFL, Lausanne, Switzerland; School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland
| | - Wulfram Gerstner
- Brain-Mind Institute, School of Life Sciences, EPFL, Lausanne, Switzerland; School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland.
| |
Collapse
|
22
|
Feng YY, Bromberg-Martin ES, Monosov IE. Dorsal raphe neurons signal integrated value during multi-attribute decision-making. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.08.17.553745. [PMID: 37662243 PMCID: PMC10473596 DOI: 10.1101/2023.08.17.553745] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/05/2023]
Abstract
The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.
Collapse
Affiliation(s)
- Yang-Yang Feng
- Department of Neuroscience, Washington University School of Medicine, St. Louis, Missouri, USA
- Department of Biomedical Engineering, Washington University, St. Louis, Missouri, USA
| | | | - Ilya E. Monosov
- Department of Neuroscience, Washington University School of Medicine, St. Louis, Missouri, USA
- Department of Biomedical Engineering, Washington University, St. Louis, Missouri, USA
- Washington University Pain Center, Washington University, St. Louis, Missouri, USA
- Department of Neurosurgery, Washington University, St. Louis, Missouri, USA
- Department of Electrical Engineering, Washington University, St. Louis, Missouri, USA
| |
Collapse
|
23
|
Chien JM, Wallis JD, Rich EL. Abstraction of Reward Context Facilitates Relative Reward Coding in Neural Populations of the Macaque Anterior Cingulate Cortex. J Neurosci 2023; 43:5944-5962. [PMID: 37495383 PMCID: PMC10436688 DOI: 10.1523/jneurosci.0292-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2023] [Revised: 06/26/2023] [Accepted: 07/22/2023] [Indexed: 07/28/2023] Open
Abstract
The anterior cingulate cortex (ACC) is believed to be involved in many cognitive processes, including linking goals to actions and tracking decision-relevant contextual information. ACC neurons robustly encode expected outcomes, but how this relates to putative functions of ACC remains unknown. Here, we approach this question from the perspective of population codes by analyzing neural spiking data in the ventral and dorsal banks of the ACC in two male monkeys trained to perform a stimulus-motor mapping task to earn rewards or avoid losses. We found that neural populations favor a low dimensional representational geometry that emphasizes the valence of potential outcomes while also facilitating the independent, abstract representation of multiple task-relevant variables. Valence encoding persisted throughout the trial, and realized outcomes were primarily encoded in a relative sense, such that cue valence acted as a context for outcome encoding. This suggests that the population coding we observe could be a mechanism that allows feedback to be interpreted in a context-dependent manner. Together, our results point to a prominent role for ACC in context setting and relative interpretation of outcomes, facilitated by abstract, or untangled, representations of task variables.SIGNIFICANCE STATEMENT The ability to interpret events in light of the current context is a critical facet of higher-order cognition. The ACC is suggested to be important for tracking contextual information, whereas alternate views hold that its function is more related to the motor system and linking goals to appropriate actions. We evaluated these possibilities by analyzing geometric properties of neural population activity in monkey ACC when contexts were determined by the valence of potential outcomes and found that this information was represented as a dominant, abstract concept. Ensuing outcomes were then coded relative to these contexts, suggesting an important role for these representations in context-dependent evaluation. Such mechanisms may be critical for the abstract reasoning and generalization characteristic of biological intelligence.
Collapse
Affiliation(s)
- Jonathan M Chien
- Nash Family Department of Neuroscience and Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, New York 10029
- Department of Neuroscience and Physiology, New York University Grossman School of Medicine, New York, New York 10016
| | - Joni D Wallis
- Department of Psychology and Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California 94720
| | - Erin L Rich
- Nash Family Department of Neuroscience and Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, New York 10029
| |
Collapse
|
24
|
Brown VM, Price R, Dombrovski AY. Anxiety as a disorder of uncertainty: implications for understanding maladaptive anxiety, anxious avoidance, and exposure therapy. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2023; 23:844-868. [PMID: 36869259 PMCID: PMC10475148 DOI: 10.3758/s13415-023-01080-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 02/14/2023] [Indexed: 03/05/2023]
Abstract
In cognitive-behavioral conceptualizations of anxiety, exaggerated threat expectancies underlie maladaptive anxiety. This view has led to successful treatments, notably exposure therapy, but is not consistent with the empirical literature on learning and choice alterations in anxiety. Empirically, anxiety is better described as a disorder of uncertainty learning. How disruptions in uncertainty lead to impairing avoidance and are treated with exposure-based methods, however, is unclear. Here, we integrate concepts from neurocomputational learning models with clinical literature on exposure therapy to propose a new framework for understanding maladaptive uncertainty functioning in anxiety. Specifically, we propose that anxiety disorders are fundamentally disorders of uncertainty learning and that successful treatments, particularly exposure therapy, work by remediating maladaptive avoidance from dysfunctional explore/exploit decisions in uncertain, potentially aversive situations. This framework reconciles several inconsistencies in the literature and provides a path forward to better understand and treat anxiety.
Collapse
Affiliation(s)
- Vanessa M Brown
- Department of Psychiatry, University of Pittsburgh, Pittsburgh, PA, USA.
| | - Rebecca Price
- Department of Psychiatry, University of Pittsburgh, Pittsburgh, PA, USA
| | | |
Collapse
|
25
|
Jamous R, Takacs A, Frings C, Münchau A, Mückschel M, Beste C. Unsigned surprise but not reward magnitude modulates the integration of motor elements during actions. Sci Rep 2023; 13:5379. [PMID: 37009782 PMCID: PMC10068803 DOI: 10.1038/s41598-023-32508-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2022] [Accepted: 03/28/2023] [Indexed: 04/04/2023] Open
Abstract
It seems natural that motor responses unfold smoothly and that we are able to easily concatenate different components of movements to achieve goal-directed actions. Theoretical frameworks suggest that different motor features have to be bound to each other to achieve a coherent action. Yet, the nature of the "glue" (i.e., bindings) between elements constituting a motor sequence and enabling a smooth unfolding of motor acts is not well understood. We examined in how far motor feature bindings are affected by reward magnitude or the effects of an unsigned surprise signal. We show that the consistency of action file binding strength is modulated by unsigned surprise, but not by reward magnitude. On a conceptual and theoretical level, the results provide links between frameworks, which have until now not been brought into connection. In particular, theoretical accounts stating that only the unexpectedness (surprisingness) is essential for action control are connected to meta-control accounts of human action control.
Collapse
Affiliation(s)
- Roula Jamous
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Faculty of Medicine, TU Dresden, Fetscherstrasse 74, 01307, Dresden, Germany
- University Neuropsychology Centre, Faculty of Medicine, TU Dresden, Dresden, Germany
| | - Adam Takacs
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Faculty of Medicine, TU Dresden, Fetscherstrasse 74, 01307, Dresden, Germany
- University Neuropsychology Centre, Faculty of Medicine, TU Dresden, Dresden, Germany
| | - Christian Frings
- Cognitive Psychology Unit, Chair of General Psychology and Methodology, Faculty I - Psychology, University of Trier, Trier, Germany
| | - Alexander Münchau
- Institute of Systems Motor Science, University of Lübeck, Lübeck, Germany
| | - Moritz Mückschel
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Faculty of Medicine, TU Dresden, Fetscherstrasse 74, 01307, Dresden, Germany
- University Neuropsychology Centre, Faculty of Medicine, TU Dresden, Dresden, Germany
| | - Christian Beste
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Faculty of Medicine, TU Dresden, Fetscherstrasse 74, 01307, Dresden, Germany.
- University Neuropsychology Centre, Faculty of Medicine, TU Dresden, Dresden, Germany.
| |
Collapse
|
26
|
Peciña M, Chen J, Karp JF, Dombrovski AY. Dynamic Feedback Between Antidepressant Placebo Expectancies and Mood. JAMA Psychiatry 2023; 80:389-398. [PMID: 36857039 PMCID: PMC9979016 DOI: 10.1001/jamapsychiatry.2023.0010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Accepted: 01/02/2023] [Indexed: 03/02/2023]
Abstract
Importance Despite high antidepressant placebo response rates, the mechanisms underlying the persistence of antidepressant placebo effects are still poorly understood. Objective To investigate the neurobehavioral mechanisms underlying the evolution of antidepressant placebo effects using a reinforcement learning (RL) framework. Design, Setting, and Participants In this acute within-patient cross-sectional study of antidepressant placebos, patients aged 18 to 55 years not receiving medication for major depressive disorder (MDD) were recruited at the University of Pittsburgh between February 21, 2017, to March 1, 2021. Interventions The antidepressant placebo functional magnetic resonance imaging task manipulates placebo-associated expectancies using visually cued fast-acting antidepressant infusions and controls their reinforcement with sham visual neurofeedback while assessing expected and experienced mood improvement. Main Outcomes and Measures The trial-by-trial evolution of expectancies and mood was examined using multilevel modeling and RL, relating model-predicted signals to spatiotemporal dynamics of blood oxygenation level-dependent (BOLD) response. Results A bayesian RL model comparison in 60 individuals (mean [SE] age, 24.5 [0.8] years; 51 females [85%]) with MDD revealed that antidepressant placebo trial-wise expectancies were updated by composite learning signals multiplexing sensory evidence (neurofeedback) and trial-wise mood (bayesian omnibus risk <0.001; exceedance probability = 97%). Placebo expectancy, neurofeedback manipulations, and composite learning signals modulated the visual cortex and dorsal attention network (threshold-free cluster enhancement [TFCE] = 1 - P >.95). As participants anticipated antidepressant infusions, learned placebo expectancies modulated the salience network (SN, TFCE = 1 - P >.95), positively scaling with depression severity. Conclusions and Relevance Results of this cross-sectional study suggest that on a timescale of minutes, antidepressant placebo effects were maintained by positive feedback loops between expectancies and mood improvement. During learning, representations of placebos and their perceived effects were enhanced in primary and secondary sensory cortices. Latent learned placebo expectancies were encoded in the SN.
Collapse
Affiliation(s)
- Marta Peciña
- Department of Psychiatry, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Jiazhou Chen
- Section on Development and Affective Neuroscience, National Institute of Health, Bethesda, Maryland
- Division of Psychiatry, University College London, London, United Kingdom
| | | | | |
Collapse
|
27
|
Viglione A, Mazziotti R, Pizzorusso T. From pupil to the brain: New insights for studying cortical plasticity through pupillometry. Front Neural Circuits 2023; 17:1151847. [PMID: 37063384 PMCID: PMC10102476 DOI: 10.3389/fncir.2023.1151847] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2023] [Accepted: 03/08/2023] [Indexed: 04/03/2023] Open
Abstract
Pupil size variations have been associated with changes in brain activity patterns related with specific cognitive factors, such as arousal, attention, and mental effort. The locus coeruleus (LC), a key hub in the noradrenergic system of the brain, is considered to be a key regulator of cognitive control on pupil size, with changes in pupil diameter corresponding to the release of norepinephrine (NE). Advances in eye-tracking technology and open-source software have facilitated accurate pupil size measurement in various experimental settings, leading to increased interest in using pupillometry to track the nervous system activation state and as a potential biomarker for brain disorders. This review explores pupillometry as a non-invasive and fully translational tool for studying cortical plasticity starting from recent literature suggesting that pupillometry could be a promising technique for estimating the degree of residual plasticity in human subjects. Given that NE is known to be a critical mediator of cortical plasticity and arousal, the review includes data revealing the importance of the LC-NE system in modulating brain plasticity and pupil size. Finally, we will review data suggesting that pupillometry could provide a quantitative and complementary measure of cortical plasticity also in pre-clinical studies.
Collapse
Affiliation(s)
- Aurelia Viglione
- BIO@SNS Lab, Scuola Normale Superiore, Pisa, Italy
- *Correspondence: Aurelia Viglione,
| | | | - Tommaso Pizzorusso
- BIO@SNS Lab, Scuola Normale Superiore, Pisa, Italy
- Institute of Neuroscience, National Research Council, Pisa, Italy
| |
Collapse
|
28
|
Talukdar T, Zwilling CE, Barbey AK. Integrating Nutrient Biomarkers, Cognitive Function, and Structural MRI Data to Build Multivariate Phenotypes of Healthy Aging. J Nutr 2023; 153:1338-1346. [PMID: 36965693 DOI: 10.1016/j.tjnut.2023.03.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 02/20/2023] [Accepted: 03/13/2023] [Indexed: 03/27/2023] Open
Abstract
BACKGROUND Research in the emerging field of Nutritional Cognitive Neuroscience demonstrates that many aspects of nutrition - from entire diets to specific nutrients - affect cognitive performance and brain health. OBJECTIVE While prior research has primarily examined the bivariate relationship between nutrition and cognition, or nutrition and brain health, the present study sought to investigate the joint relationship between these essential and interactive elements of human health. METHODS We applied a state-of-the-art data fusion method, Coupled Matrix Tensor Factorization, to characterize the joint association between measures of nutrition (52 nutrient biomarkers), cognition (Wechsler Abbreviated Test of Intelligence and Wechsler Memory Scale), and brain health (high-resolution Magnetic Resonance Imaging measures of structural brain volume) within a cross-sectional sample of 111 healthy older adults that had an average age of 69.1 years, were 62% female and had an average Body Mass Index of 26.0. RESULTS Data fusion uncovered 3 latent factors that capture the joint association between specific nutrient profiles, cognitive measures, and cortical volumes, demonstrating the respects in which these health domains are coupled. Hierarchical cluster analysis further revealed systematic differences between the observed latent factors, providing evidence for multivariate phenotypes that represent high versus low levels of performance across multiple health domains. The primary features that distinguish between each phenotype were: (i) nutrient biomarkers for monounsaturated and polyunsaturated fatty acids; (ii) cognitive measures of immediate, auditory, and delayed memory; and (iii) brain volumes within frontal, temporal, and parietal cortex. CONCLUSIONS By incorporating innovations in nutritional epidemiology (nutrient biomarker analysis), cognitive neuroscience (high-resolution structural brain imaging), and statistics (data fusion), the present study provides an interdisciplinary synthesis of methods that elucidate how nutrition, cognition, and brain health are integrated through lifestyle choices that affect healthy aging.
Collapse
Affiliation(s)
- Tanveer Talukdar
- Decision Neuroscience Laboratory, Beckman Institute, University of Illinois, Urbana, IL. USA
| | - Christopher E Zwilling
- Decision Neuroscience Laboratory, Beckman Institute, University of Illinois, Urbana, IL. USA
| | - Aron K Barbey
- Decision Neuroscience Laboratory, Beckman Institute, University of Illinois, Urbana, IL. USA; Department of Psychology, University of Illinois, Urbana, IL. USA; Carl R. Woese Institute for Genomic Biology, University of Illinois, Champaign, IL. USA; Department of Bioengineering, University of Illinois, Champaign, IL. USA; Division of Nutritional Sciences, University of Illinois, Champaign, IL. USA; Neuroscience Program, University of Illinois, Champaign, IL. USA.
| |
Collapse
|
29
|
Hein TP, Gong Z, Ivanova M, Fedele T, Nikulin V, Herrojo Ruiz M. Anterior cingulate and medial prefrontal cortex oscillations underlie learning alterations in trait anxiety in humans. Commun Biol 2023; 6:271. [PMID: 36922553 PMCID: PMC10017780 DOI: 10.1038/s42003-023-04628-1] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Accepted: 02/27/2023] [Indexed: 03/18/2023] Open
Abstract
Anxiety has been linked to altered belief formation and uncertainty estimation, impacting learning. Identifying the neural processes underlying these changes is important for understanding brain pathology. Here, we show that oscillatory activity in the medial prefrontal, anterior cingulate and orbitofrontal cortex (mPFC, ACC, OFC) explains anxiety-related learning alterations. In a magnetoencephalography experiment, two groups of human participants pre-screened with high and low trait anxiety (HTA, LTA: 39) performed a probabilistic reward-based learning task. HTA undermined learning through an overestimation of volatility, leading to faster belief updating, more stochastic decisions and pronounced lose-shift tendencies. On a neural level, we observed increased gamma activity in the ACC, dmPFC, and OFC during encoding of precision-weighted prediction errors in HTA, accompanied by suppressed ACC alpha/beta activity. Our findings support the association between altered learning and belief updating in anxiety and changes in gamma and alpha/beta activity in the ACC, dmPFC, and OFC.
Collapse
Affiliation(s)
- Thomas P Hein
- Goldsmiths, University of London, Psychology Department, Whitehead Building New Cross, London, SE14 6NW, UK
| | - Zheng Gong
- Centre for Cognition and Decision making, Institute for Cognitive Neuroscience, HSE University, Moscow, Russian Federation
| | - Marina Ivanova
- Centre for Cognition and Decision making, Institute for Cognitive Neuroscience, HSE University, Moscow, Russian Federation
| | - Tommaso Fedele
- Centre for Cognition and Decision making, Institute for Cognitive Neuroscience, HSE University, Moscow, Russian Federation
| | - Vadim Nikulin
- Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Maria Herrojo Ruiz
- Goldsmiths, University of London, Psychology Department, Whitehead Building New Cross, London, SE14 6NW, UK.
| |
Collapse
|
30
|
Tecilla M, Großbach M, Gentile G, Holland P, Sporn S, Antonini A, Herrojo Ruiz M. Modulation of Motor Vigor by Expectation of Reward Probability Trial-by-Trial Is Preserved in Healthy Ageing and Parkinson's Disease Patients. J Neurosci 2023; 43:1757-1777. [PMID: 36732072 PMCID: PMC10010462 DOI: 10.1523/jneurosci.1583-22.2022] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2022] [Revised: 12/13/2022] [Accepted: 12/31/2022] [Indexed: 02/04/2023] Open
Abstract
Motor improvements, such as faster movement times or increased velocity, have been associated with reward magnitude in deterministic contexts. Yet whether individual inferences on reward probability influence motor vigor dynamically remains undetermined. We investigated how dynamically inferring volatile action-reward contingencies modulated motor performance trial-by-trial. We conducted three studies that coupled a reversal learning paradigm with a motor sequence task and used a validated hierarchical Bayesian model to fit trial-by-trial data. In Study 1, we tested healthy younger [HYA; 37 (24 females)] and older adults [HOA; 37 (17 females)], and medicated Parkinson's disease (PD) patients [20 (7 females)]. We showed that stronger predictions about the tendency of the action-reward contingency led to faster performance tempo, commensurate with movement time, on a trial-by-trial basis without robustly modulating reaction time (RT). Using Bayesian linear mixed models, we demonstrated a similar invigoration effect on performance tempo in HYA, HOA, and PD, despite HOA and PD being slower than HYA. In Study 2 [HYA, 39 (29 females)], we additionally showed that retrospective subjective inference about credit assignment did not contribute to differences in motor vigor effects. Last, Study 3 [HYA, 33 (27 females)] revealed that explicit beliefs about the reward tendency (confidence ratings) modulated performance tempo trial-by-trial. Our study is the first to reveal that the dynamic updating of beliefs about volatile action-reward contingencies positively biases motor performance through faster tempo. We also provide robust evidence for a preserved sensitivity of motor vigor to inferences about the action-reward mapping in aging and medicated PD.SIGNIFICANCE STATEMENT Navigating a world rich in uncertainty relies on updating beliefs about the probability that our actions lead to reward. Here, we investigated how inferring the action-reward contingencies in a volatile environment modulated motor vigor trial-by-trial in healthy younger and older adults, and in Parkinson's disease (PD) patients on medication. We found an association between trial-by-trial predictions about the tendency of the action-reward contingency and performance tempo, with stronger expectations speeding the movement. We additionally provided evidence for a similar sensitivity of performance tempo to the strength of these predictions in all groups. Thus, dynamic beliefs about the changing relationship between actions and their outcome enhanced motor vigor. This positive bias was not compromised by age or Parkinson's disease.
Collapse
Affiliation(s)
- Margherita Tecilla
- Department of Psychology, Goldsmiths, University of London, London SE146NW, United Kingdom
| | - Michael Großbach
- Institute of Music Physiology and Musicians' Medicine, Hannover University of Music Drama and Media, Hannover 30175, Germany
| | - Giovanni Gentile
- Parkinson and Movement Disorders Unit, Study Center for Neurodegeneration (CESNE), Department of Neuroscience, University of Padua, Padua 35131, Italy
| | - Peter Holland
- Department of Psychology, Goldsmiths, University of London, London SE146NW, United Kingdom
| | - Sebastian Sporn
- Department of Clinical and Movement Neuroscience, Queen Square Institute of Neurology, University College London, London WC1N3BG, United Kingdom
| | - Angelo Antonini
- Parkinson and Movement Disorders Unit, Study Center for Neurodegeneration (CESNE), Department of Neuroscience, University of Padua, Padua 35131, Italy
| | - Maria Herrojo Ruiz
- Department of Psychology, Goldsmiths, University of London, London SE146NW, United Kingdom
| |
Collapse
|
31
|
Guath M, Kleberg JL, Weis J, Widegren E, Frick M, Möller S, Klevebrant L, Karlsson B, Fällmar D, Mårtensson J, Pine DS, Brocki K, Gingnell M, Frick A. Pupil dilation during negative prediction errors is related to brain choline concentration and depressive symptoms in adolescents. Behav Brain Res 2023; 436:114060. [DOI: 10.1016/j.bbr.2022.114060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2022] [Revised: 08/10/2022] [Accepted: 08/14/2022] [Indexed: 10/15/2022]
|
32
|
Task-evoked pupillary responses track precision-weighted prediction errors and learning rate during interceptive visuomotor actions. Sci Rep 2022; 12:22098. [PMID: 36543845 PMCID: PMC9772236 DOI: 10.1038/s41598-022-26544-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2022] [Accepted: 12/15/2022] [Indexed: 12/24/2022] Open
Abstract
In this study, we examined the relationship between physiological encoding of surprise and the learning of anticipatory eye movements. Active inference portrays perception and action as interconnected inference processes, driven by the imperative to minimise the surprise of sensory observations. To examine this characterisation of oculomotor learning during a hand-eye coordination task, we tested whether anticipatory eye movements were updated in accordance with Bayesian principles and whether trial-by-trial learning rates tracked pupil dilation as a marker of 'surprise'. Forty-four participants completed an interception task in immersive virtual reality that required them to hit bouncing balls that had either expected or unexpected bounce profiles. We recorded anticipatory eye movements known to index participants' beliefs about likely ball bounce trajectories. By fitting a hierarchical Bayesian inference model to the trial-wise trajectories of these predictive eye movements, we were able to estimate each individual's expectations about bounce trajectories, rates of belief updating, and precision-weighted prediction errors. We found that the task-evoked pupil response tracked prediction errors and learning rates but not beliefs about ball bounciness or environmental volatility. These findings are partially consistent with active inference accounts and shed light on how encoding of surprise may shape the control of action.
Collapse
|
33
|
Pisauro MA, Fouragnan EF, Arabadzhiyska DH, Apps MAJ, Philiastides MG. Neural implementation of computational mechanisms underlying the continuous trade-off between cooperation and competition. Nat Commun 2022; 13:6873. [PMID: 36369180 PMCID: PMC9652314 DOI: 10.1038/s41467-022-34509-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2021] [Accepted: 10/27/2022] [Indexed: 11/13/2022] Open
Abstract
Social interactions evolve continuously. Sometimes we cooperate, sometimes we compete, while at other times we strategically position ourselves somewhere in between to account for the ever-changing social contexts around us. Research on social interactions often focuses on a binary dichotomy between competition and cooperation, ignoring people's evolving shifts along a continuum. Here, we develop an economic game - the Space Dilemma - where two players change their degree of cooperativeness over time in cooperative and competitive contexts. Using computational modelling we show how social contexts bias choices and characterise how inferences about others' intentions modulate cooperativeness. Consistent with the modelling predictions, brain regions previously linked to social cognition, including the temporo-parietal junction, dorso-medial prefrontal cortex and the anterior cingulate gyrus, encode social prediction errors and context-dependent signals, correlating with shifts along a cooperation-competition continuum. These results provide a comprehensive account of the computational and neural mechanisms underlying the continuous trade-off between cooperation and competition.
Collapse
Affiliation(s)
- M A Pisauro
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK.
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK.
| | - E F Fouragnan
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
- Brain Research Imaging Center and School of Psychology, Faculty of Health, University of Plymouth, Plymouth, UK
| | - D H Arabadzhiyska
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
| | - M A J Apps
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
| | - M G Philiastides
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
| |
Collapse
|
34
|
Woo JH, Azab H, Jahn A, Hayden B, Brown JW. The PRO model accounts for the anterior cingulate cortex role in risky decision-making and monitoring. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2022; 22:952-968. [PMID: 35332510 PMCID: PMC11059203 DOI: 10.3758/s13415-022-00992-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 02/17/2022] [Indexed: 11/08/2022]
Abstract
The anterior cingulate cortex (ACC) has been implicated in a number of functions, including performance monitoring and decision-making involving effort. The prediction of responses and outcomes (PRO) model has provided a unified account of much human and monkey ACC data involving anatomy, neurophysiology, EEG, fMRI, and behavior. We explored the computational nature of ACC with the PRO model, extending it to account specifically for both human and macaque monkey decision-making under risk, including both behavioral and neural data. We show that the PRO model can account for a number of additional effects related to outcome prediction, decision-making under risk, gambling behavior. In particular, we show that the ACC represents the variance of uncertain outcomes, suggesting a link between ACC function and mean-variance theories of decision making. The PRO model provides a unified account of a large set of data regarding the ACC.
Collapse
Affiliation(s)
- Jae Hyung Woo
- Department of Psychological & Brain Sciences, Indiana University, Bloomington, IN, USA
- Department of Psychological & Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Habiba Azab
- Baylor College of Medicine, Houston, TX, USA
| | - Andrew Jahn
- Department of Psychological & Brain Sciences, Indiana University, Bloomington, IN, USA
- fMRI Laboratory, University of Michigan, Ann Arbor, MI, USA
| | - Benjamin Hayden
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| | - Joshua W Brown
- Department of Psychological & Brain Sciences, Indiana University, Bloomington, IN, USA.
| |
Collapse
|
35
|
Vázquez D, Schneider KN, Roesch MR. Neural signals implicated in the processing of appetitive and aversive events in social and non-social contexts. Front Syst Neurosci 2022; 16:926388. [PMID: 35993086 PMCID: PMC9381696 DOI: 10.3389/fnsys.2022.926388] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Accepted: 06/30/2022] [Indexed: 11/13/2022] Open
Abstract
In 2014, we participated in a special issue of Frontiers examining the neural processing of appetitive and aversive events. Specifically, we reviewed brain areas that contribute to the encoding of prediction errors and value versus salience, attention and motivation. Further, we described how we disambiguated these cognitive processes and their neural substrates by using paradigms that incorporate both appetitive and aversive stimuli. We described a circuit in which the orbitofrontal cortex (OFC) signals expected value and the basolateral amygdala (BLA) encodes the salience and valence of both appetitive and aversive events. This information is integrated by the nucleus accumbens (NAc) and dopaminergic (DA) signaling in order to generate prediction and prediction error signals, which guide decision-making and learning via the dorsal striatum (DS). Lastly, the anterior cingulate cortex (ACC) is monitoring actions and outcomes, and signals the need to engage attentional control in order to optimize behavioral output. Here, we expand upon this framework, and review our recent work in which within-task manipulations of both appetitive and aversive stimuli allow us to uncover the neural processes that contribute to the detection of outcomes delivered to a conspecific and behaviors in social contexts. Specifically, we discuss the involvement of single-unit firing in the ACC and DA signals in the NAc during the processing of appetitive and aversive events in both social and non-social contexts.
Collapse
Affiliation(s)
- Daniela Vázquez
- Department of Psychology, University of Maryland, College Park, College Park, MD, United States
- Neuroscience and Cognitive Science Program, University of Maryland, College Park, College Park, MD, United States
| | - Kevin N. Schneider
- Department of Psychology, University of Maryland, College Park, College Park, MD, United States
- Neuroscience and Cognitive Science Program, University of Maryland, College Park, College Park, MD, United States
| | - Matthew R. Roesch
- Department of Psychology, University of Maryland, College Park, College Park, MD, United States
- Neuroscience and Cognitive Science Program, University of Maryland, College Park, College Park, MD, United States
- *Correspondence: Matthew R. Roesch,
| |
Collapse
|
36
|
Lee JY, You T, Lee CH, Im GH, Seo H, Woo CW, Kim SG. Role of anterior cingulate cortex inputs to periaqueductal gray for pain avoidance. Curr Biol 2022; 32:2834-2847.e5. [PMID: 35609604 DOI: 10.1016/j.cub.2022.04.090] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Revised: 03/07/2022] [Accepted: 04/28/2022] [Indexed: 12/13/2022]
Abstract
Although pain-related excessive fear is known to be a key factor in chronic pain disability, which involves the anterior cingulate cortex (ACC), little is known about the downstream circuits of the ACC for fear avoidance in pain processing. Using behavioral experiments and functional magnetic resonance imaging with optogenetics at 15.2 T, we demonstrate that the ACC is a part of the abnormal circuit changes in chronic pain and its downstream circuits are closely related to modulating sensorimotor integration and generating active movement rather than carrying sensory information. The projection from the ACC to the dorsolateral and lateral parts of the periaqueductal gray (dl/lPAG) especially enhances both reflexive and active avoidance behavior toward pain. Collectively, our results indicate that increased signals from the ACC to the dl/lPAG might be critical for excessive fear avoidance in chronic pain disability.
Collapse
Affiliation(s)
- Jeong-Yun Lee
- Center for Neuroscience Imaging Research (CNIR), Institute for Basic Science (IBS), Suwon 16419, Republic of Korea.
| | - Taeyi You
- Center for Neuroscience Imaging Research (CNIR), Institute for Basic Science (IBS), Suwon 16419, Republic of Korea; Department of Biomedical Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea; Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University, Suwon 16419, Republic of Korea
| | - Choong-Hee Lee
- Center for Neuroscience Imaging Research (CNIR), Institute for Basic Science (IBS), Suwon 16419, Republic of Korea
| | - Geun Ho Im
- Center for Neuroscience Imaging Research (CNIR), Institute for Basic Science (IBS), Suwon 16419, Republic of Korea
| | - Heewon Seo
- Center for Neuroscience Imaging Research (CNIR), Institute for Basic Science (IBS), Suwon 16419, Republic of Korea; Department of Chemistry and Biochemistry, Oberlin College, Oberlin, OH 44704, USA
| | - Choong-Wan Woo
- Center for Neuroscience Imaging Research (CNIR), Institute for Basic Science (IBS), Suwon 16419, Republic of Korea; Department of Biomedical Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea; Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University, Suwon 16419, Republic of Korea
| | - Seong-Gi Kim
- Center for Neuroscience Imaging Research (CNIR), Institute for Basic Science (IBS), Suwon 16419, Republic of Korea; Department of Biomedical Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea; Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University, Suwon 16419, Republic of Korea.
| |
Collapse
|
37
|
Johnson C, Kretsge LN, Yen WW, Sriram B, O'Connor A, Liu RS, Jimenez JC, Phadke RA, Wingfield KK, Yeung C, Jinadasa TJ, Nguyen TPH, Cho ES, Fuchs E, Spevack ED, Velasco BE, Hausmann FS, Fournier LA, Brack A, Melzer S, Cruz-Martín A. Highly unstable heterogeneous representations in VIP interneurons of the anterior cingulate cortex. Mol Psychiatry 2022; 27:2602-2618. [PMID: 35246635 PMCID: PMC11128891 DOI: 10.1038/s41380-022-01485-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/05/2021] [Revised: 02/07/2022] [Accepted: 02/09/2022] [Indexed: 11/09/2022]
Abstract
A hallmark of the anterior cingulate cortex (ACC) is its functional heterogeneity. Functional and imaging studies revealed its importance in the encoding of anxiety-related and social stimuli, but it is unknown how microcircuits within the ACC encode these distinct stimuli. One type of inhibitory interneuron, which is positive for vasoactive intestinal peptide (VIP), is known to modulate the activity of pyramidal cells in local microcircuits, but it is unknown whether VIP cells in the ACC (VIPACC) are engaged by particular contexts or stimuli. Additionally, recent studies demonstrated that neuronal representations in other cortical areas can change over time at the level of the individual neuron. However, it is not known whether stimulus representations in the ACC remain stable over time. Using in vivo Ca2+ imaging and miniscopes in freely behaving mice to monitor neuronal activity with cellular resolution, we identified individual VIPACC that preferentially activated to distinct stimuli across diverse tasks. Importantly, although the population-level activity of the VIPACC remained stable across trials, the stimulus-selectivity of individual interneurons changed rapidly. These findings demonstrate marked functional heterogeneity and instability within interneuron populations in the ACC. This work contributes to our understanding of how the cortex encodes information across diverse contexts and provides insight into the complexity of neural processes involved in anxiety and social behavior.
Collapse
Affiliation(s)
- Connor Johnson
- Neurobiology Section in the Department of Biology, Boston University, Boston, MA, USA
| | - Lisa N Kretsge
- The Graduate Program for Neuroscience, Boston University, Boston, MA, USA
- Neurophotonics Center, Boston University, Boston, MA, USA
| | - William W Yen
- Neurobiology Section in the Department of Biology, Boston University, Boston, MA, USA
| | | | - Alexandra O'Connor
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Ruichen Sky Liu
- MS in Statistical Practice Program, Boston University, Boston, MA, USA
| | - Jessica C Jimenez
- Department of Neurology, University of California, San Francisco, San Francisco, CA, USA
| | - Rhushikesh A Phadke
- Molecular Biology, Cell Biology and Biochemistry Program, Boston University, Boston, MA, USA
| | - Kelly K Wingfield
- Neurophotonics Center, Boston University, Boston, MA, USA
- Department of Pharmacology and Experimental Therapeutics, Boston University, Boston, MA, USA
| | - Charlotte Yeung
- Neurobiology Section in the Department of Biology, Boston University, Boston, MA, USA
| | - Tushare J Jinadasa
- Neurobiology Section in the Department of Biology, Boston University, Boston, MA, USA
| | - Thanh P H Nguyen
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Eun Seon Cho
- Neurobiology Section in the Department of Biology, Boston University, Boston, MA, USA
| | - Erelle Fuchs
- Neurobiology Section in the Department of Biology, Boston University, Boston, MA, USA
| | - Eli D Spevack
- Neurobiology Section in the Department of Biology, Boston University, Boston, MA, USA
| | - Berta Escude Velasco
- Neurobiology Section in the Department of Biology, Boston University, Boston, MA, USA
| | - Frances S Hausmann
- Neurobiology Section in the Department of Biology, Boston University, Boston, MA, USA
| | - Luke A Fournier
- Neurobiology Section in the Department of Biology, Boston University, Boston, MA, USA
| | - Alison Brack
- Molecular Biology, Cell Biology and Biochemistry Program, Boston University, Boston, MA, USA
| | - Sarah Melzer
- Department of Neurobiology, Howard Hughes Medical Institute, Harvard Medical School, Boston, MA, USA
| | - Alberto Cruz-Martín
- Neurobiology Section in the Department of Biology, Boston University, Boston, MA, USA.
- Neurophotonics Center, Boston University, Boston, MA, USA.
- Molecular Biology, Cell Biology and Biochemistry Program, Boston University, Boston, MA, USA.
- Center for Systems Neuroscience, Boston University, Boston, MA, USA.
- The Center for Network Systems Biology, Boston University, Boston, MA, USA.
| |
Collapse
|
38
|
Kostadinov D, Häusser M. Reward signals in the cerebellum: origins, targets, and functional implications. Neuron 2022; 110:1290-1303. [PMID: 35325616 DOI: 10.1016/j.neuron.2022.02.015] [Citation(s) in RCA: 34] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2021] [Revised: 12/22/2021] [Accepted: 02/16/2022] [Indexed: 12/14/2022]
Abstract
The cerebellum has long been proposed to play a role in cognitive function, although this has remained controversial. This idea has received renewed support with the recent discovery that signals associated with reward can be observed in the cerebellar circuitry, particularly in goal-directed learning tasks involving an interplay between the cerebellar cortex, basal ganglia, and cerebral cortex. Remarkably, a wide range of reward contingencies-including reward expectation, delivery, size, and omission-can be encoded by specific circuit elements in a manner that reflects the microzonal organization of the cerebellar cortex. The facts that reward signals have been observed in both the mossy fiber and climbing fiber input pathways to the cerebellar cortex and that their convergence may trigger plasticity in Purkinje cells suggest that these interactions may be crucial for the role of the cerebellar cortex in learned behavior. These findings strengthen the emerging consensus that the cerebellum plays a pivotal role in shaping cognitive processing and suggest that the cerebellum may combine both supervised learning and reinforcement learning to optimize goal-directed action. We make specific predictions about how cerebellar circuits can work in concert with the basal ganglia to guide different stages of learning.
Collapse
Affiliation(s)
- Dimitar Kostadinov
- Wolfson Institute for Biomedical Research and Department of Neuroscience, Physiology and Pharmacology, University College London, London, UK.
| | - Michael Häusser
- Wolfson Institute for Biomedical Research and Department of Neuroscience, Physiology and Pharmacology, University College London, London, UK.
| |
Collapse
|
39
|
Joshi S, Gold JI. Context-dependent relationships between locus coeruleus firing patterns and coordinated neural activity in the anterior cingulate cortex. eLife 2022; 11:63490. [PMID: 34994344 PMCID: PMC8765756 DOI: 10.7554/elife.63490] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2020] [Accepted: 12/16/2021] [Indexed: 01/30/2023] Open
Abstract
Ascending neuromodulatory projections from the locus coeruleus (LC) affect cortical neural networks via the release of norepinephrine (NE). However, the exact nature of these neuromodulatory effects on neural activity patterns in vivo is not well understood. Here, we show that in awake monkeys, LC activation is associated with changes in coordinated activity patterns in the anterior cingulate cortex (ACC). These relationships, which are largely independent of changes in firing rates of individual ACC neurons, depend on the type of LC activation: ACC pairwise correlations tend to be reduced when ongoing (baseline) LC activity increases but enhanced when external events evoke transient LC responses. Both relationships covary with pupil changes that reflect LC activation and arousal. These results suggest that modulations of information processing that reflect changes in coordinated activity patterns in cortical networks can result partly from ongoing, context-dependent, arousal-related changes in activation of the LC-NE system.
Collapse
Affiliation(s)
- Siddhartha Joshi
- Department of Neuroscience, University of Pennsylvania, Philadelphia, United States
| | - Joshua I Gold
- Department of Neuroscience, University of Pennsylvania, Philadelphia, United States
| |
Collapse
|
40
|
Monosov IE, Rushworth MFS. Interactions between ventrolateral prefrontal and anterior cingulate cortex during learning and behavioural change. Neuropsychopharmacology 2022; 47:196-210. [PMID: 34234288 PMCID: PMC8617208 DOI: 10.1038/s41386-021-01079-2] [Citation(s) in RCA: 23] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Revised: 05/27/2021] [Accepted: 06/15/2021] [Indexed: 02/06/2023]
Abstract
Hypotheses and beliefs guide credit assignment - the process of determining which previous events or actions caused an outcome. Adaptive hypothesis formation and testing are crucial in uncertain and changing environments in which associations and meanings are volatile. Despite primates' abilities to form and test hypotheses, establishing what is causally responsible for the occurrence of particular outcomes remains a fundamental challenge for credit assignment and learning. Hypotheses about what surprises are due to stochasticity inherent in an environment as opposed to real, systematic changes are necessary for identifying the environment's predictive features, but are often hard to test. We review evidence that two highly interconnected frontal cortical regions, anterior cingulate cortex and ventrolateral prefrontal area 47/12o, provide a biological substrate for linking two crucial components of hypothesis-formation and testing: the control of information seeking and credit assignment. Neuroimaging, targeted disruptions, and neurophysiological studies link an anterior cingulate - 47/12o circuit to generation of exploratory behaviour, non-instrumental information seeking, and interpretation of subsequent feedback in the service of credit assignment. Our observations support the idea that information seeking and credit assignment are linked at the level of neural circuits and explain why this circuit is important for ensuring behaviour is flexible and adaptive.
Collapse
Affiliation(s)
- Ilya E Monosov
- Department of Neuroscience, Washington University School of Medicine, St. Louis, MO, USA.
- Department of Biomedical Engineering, Washington University, St. Louis, MO, USA.
- Department of Electrical Engineering, Washington University, St. Louis, MO, USA.
- Department of Neurosurgery, Washington University, St. Louis, MO, USA.
- Pain Center, Washington University, St. Louis, MO, USA.
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, UK.
| |
Collapse
|
41
|
Rudebeck PH, Izquierdo A. Foraging with the frontal cortex: A cross-species evaluation of reward-guided behavior. Neuropsychopharmacology 2022; 47:134-146. [PMID: 34408279 PMCID: PMC8617092 DOI: 10.1038/s41386-021-01140-0] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/05/2021] [Revised: 07/30/2021] [Accepted: 07/30/2021] [Indexed: 02/07/2023]
Abstract
Efficient foraging is essential to survival and depends on frontal cortex in mammals. Because of its role in psychiatric disorders, frontal cortex and its contributions to reward procurement have been studied extensively in both rodents and non-human primates. How frontal cortex of these animal models compares is a source of intense debate. Here we argue that translating findings from rodents to non-human primates requires an appreciation of both the niche in which each animal forages as well as the similarities in frontal cortex anatomy and function. Consequently, we highlight similarities and differences in behavior and anatomy, before focusing on points of convergence in how parts of frontal cortex contribute to distinct aspects of foraging in rats and macaques, more specifically. In doing so, our aim is to emphasize where translation of frontal cortex function between species is clearer, where there is divergence, and where future work should focus. We finish by highlighting aspects of foraging for which have received less attention but we believe are critical to uncovering how frontal cortex promotes survival in each species.
Collapse
Affiliation(s)
| | - Alicia Izquierdo
- Department of Psychology, UCLA, Los Angeles, CA, USA.
- The Brain Research Institute, UCLA, Los Angeles, CA, USA.
- Integrative Center for Learning and Memory, UCLA, Los Angeles, CA, USA.
- Integrative Center for Addictions, UCLA, Los Angeles, CA, USA.
| |
Collapse
|
42
|
Soltani A, Koechlin E. Computational models of adaptive behavior and prefrontal cortex. Neuropsychopharmacology 2022; 47:58-71. [PMID: 34389808 PMCID: PMC8617006 DOI: 10.1038/s41386-021-01123-1] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Revised: 07/19/2021] [Accepted: 07/20/2021] [Indexed: 02/07/2023]
Abstract
The real world is uncertain, and while ever changing, it constantly presents itself in terms of new sets of behavioral options. To attain the flexibility required to tackle these challenges successfully, most mammalian brains are equipped with certain computational abilities that rely on the prefrontal cortex (PFC). By examining learning in terms of internal models associating stimuli, actions, and outcomes, we argue here that adaptive behavior relies on specific interactions between multiple systems including: (1) selective models learning stimulus-action associations through rewards; (2) predictive models learning stimulus- and/or action-outcome associations through statistical inferences anticipating behavioral outcomes; and (3) contextual models learning external cues associated with latent states of the environment. Critically, the PFC combines these internal models by forming task sets to drive behavior and, moreover, constantly evaluates the reliability of actor task sets in predicting external contingencies to switch between task sets or create new ones. We review different models of adaptive behavior to demonstrate how their components map onto this unifying framework and specific PFC regions. Finally, we discuss how our framework may help to better understand the neural computations and the cognitive architecture of PFC regions guiding adaptive behavior.
Collapse
Affiliation(s)
- Alireza Soltani
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA.
| | - Etienne Koechlin
- Institut National de la Sante et de la Recherche Medicale, Universite Pierre et Marie Curie, Ecole Normale Superieure, Paris, France.
| |
Collapse
|
43
|
Seamans JK, Floresco SB. Event-based control of autonomic and emotional states by the anterior cingulate cortex. Neurosci Biobehav Rev 2021; 133:104503. [PMID: 34922986 DOI: 10.1016/j.neubiorev.2021.12.026] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Revised: 11/25/2021] [Accepted: 12/14/2021] [Indexed: 12/25/2022]
Abstract
Despite being an intensive area of research, the function of the anterior cingulate cortex (ACC) remains somewhat of a mystery. Human imaging studies implicate the ACC in various cognitive functions, yet surgical ACC lesions used to treat emotional disorders have minimal lasting effects on cognition. An alternative view is that ACC regulates autonomic states, consistent with its interconnectivity with autonomic control regions and that stimulation evokes changes in autonomic/emotional states. At the cellular level, ACC neurons are highly multi-modal and promiscuous, and can represent a staggering array of task events. These neurons nevertheless combine to produce highly event-specific ensemble patterns that likely alter activity in downstream regions controlling emotional and autonomic tone. Since neuromodulators regulate the strength of the ensemble activity patterns, they would regulate the impact these patterns have on downstream targets. Through these mechanisms, the ACC may determine how strongly to react to the very events its ensembles represent. Pathologies arise when specific event-related representations gain excessive control over autonomic/emotional states.
Collapse
Affiliation(s)
- Jeremy K Seamans
- Depts. of Psychiatry, Djavad Mowafaghian Centre for Brain Health, University of British Columbia, 2211 Wesbrook Mall, Vancouver, BC, V6B2T5, Canada.
| | - Stan B Floresco
- Depts. of Psychology, Djavad Mowafaghian Centre for Brain Health, University of British Columbia, 2211 Wesbrook Mall, Vancouver, BC, V6B2T5, Canada
| |
Collapse
|
44
|
K Namboodiri VM, Hobbs T, Trujillo-Pisanty I, Simon RC, Gray MM, Stuber GD. Relative salience signaling within a thalamo-orbitofrontal circuit governs learning rate. Curr Biol 2021; 31:5176-5191.e5. [PMID: 34637750 PMCID: PMC8849135 DOI: 10.1016/j.cub.2021.09.037] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2021] [Revised: 07/19/2021] [Accepted: 09/15/2021] [Indexed: 11/20/2022]
Abstract
Learning to predict rewards is essential for the sustained fitness of animals. Contemporary views suggest that such learning is driven by a reward prediction error (RPE)-the difference between received and predicted rewards. The magnitude of learning induced by an RPE is proportional to the product of the RPE and a learning rate. Here we demonstrate using two-photon calcium imaging and optogenetics in mice that certain functionally distinct subpopulations of ventral/medial orbitofrontal cortex (vmOFC) neurons signal learning rate control. Consistent with learning rate control, trial-by-trial fluctuations in vmOFC activity positively correlate with behavioral updating when the RPE is positive, and negatively correlates with behavioral updating when the RPE is negative. Learning rate is affected by many variables including the salience of a reward. We found that the average reward response of these neurons signals the relative salience of a reward, because it decreases after reward prediction learning or the introduction of another highly salient aversive stimulus. The relative salience signaling in vmOFC is sculpted by medial thalamic inputs. These results support emerging theoretical views that prefrontal cortex encodes and controls learning parameters.
Collapse
Affiliation(s)
- Vijay Mohan K Namboodiri
- The Center for the Neurobiology of Addiction, Pain, and Emotion, Department of Anesthesiology and Pain Medicine, Department of Pharmacology, University of Washington, Seattle, WA 98195, USA
| | - Taylor Hobbs
- The Center for the Neurobiology of Addiction, Pain, and Emotion, Department of Anesthesiology and Pain Medicine, Department of Pharmacology, University of Washington, Seattle, WA 98195, USA
| | - Ivan Trujillo-Pisanty
- The Center for the Neurobiology of Addiction, Pain, and Emotion, Department of Anesthesiology and Pain Medicine, Department of Pharmacology, University of Washington, Seattle, WA 98195, USA
| | - Rhiana C Simon
- Graduate Program in Neuroscience, University of Washington, Seattle, WA 98195, USA
| | - Madelyn M Gray
- Graduate Program in Neuroscience, University of Washington, Seattle, WA 98195, USA
| | - Garret D Stuber
- The Center for the Neurobiology of Addiction, Pain, and Emotion, Department of Anesthesiology and Pain Medicine, Department of Pharmacology, University of Washington, Seattle, WA 98195, USA; Graduate Program in Neuroscience, University of Washington, Seattle, WA 98195, USA.
| |
Collapse
|
45
|
Anterior cingulate cortex is necessary for spontaneous opioid withdrawal and withdrawal-induced hyperalgesia in male mice. Neuropsychopharmacology 2021; 46:1990-1999. [PMID: 34341495 PMCID: PMC8429582 DOI: 10.1038/s41386-021-01118-y] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Revised: 07/13/2021] [Accepted: 07/20/2021] [Indexed: 02/05/2023]
Abstract
The anterior cingulate cortex (ACC) is implicated in many pathologies, including depression, anxiety, substance-use disorders, and pain. There is also evidence from brain imaging that the ACC is hyperactive during periods of opioid withdrawal. However, there are limited data contributing to our understanding of ACC function at the cellular level during opioid withdrawal. Here, we address this issue by performing ex vivo electrophysiological analysis of thick-tufted, putative dopamine D2 receptor expressing, layer V pyramidal neurons in the ACC (ACC L5 PyNs) in a mouse model of spontaneous opioid withdrawal. We found that escalating doses of morphine (20, 40, 60, 80, and 100 mg/kg, i.p. on days 1-5, respectively) injected twice daily into male C57BL/6 mice evoked withdrawal behaviors and an associated withdrawal-induced mechanical hypersensitivity. Brain slices prepared 24 h following the last morphine injection showed increases in ACC L5 thick-tufted PyN-intrinsic membrane excitability, increases in membrane resistance, reductions in the rheobase, and reductions in HCN channel-mediated currents (IH). We did not observe changes in intrinsic or synaptic properties on thin-tufted, dopamine D1-receptor-expressing ACC L5 PyNs recorded from male Drd1a-tdTomato transgenic mice. In addition, we found that chemogenetic inhibition of the ACC blocked opioid-induced withdrawal and withdrawal-induced mechanical hypersensitivity. These results demonstrate that spontaneous opioid withdrawal alters neuronal properties within the ACC and that ACC activity is necessary to control behaviors associated with opioid withdrawal and withdrawal-induced mechanical hypersensitivity. The ability of the ACC to regulate both withdrawal behaviors and withdrawal-induced mechanical hypersensitivity suggests overlapping mechanisms between two seemingly distinguishable behaviors. This commonality potentially suggests that the ACC is a locus for multiple withdrawal symptoms.
Collapse
|
46
|
Characterizing anhedonia: A systematic review of neuroimaging across the subtypes of reward processing deficits in depression. COGNITIVE AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2021; 20:816-841. [PMID: 32472419 PMCID: PMC7395022 DOI: 10.3758/s13415-020-00804-6] [Citation(s) in RCA: 79] [Impact Index Per Article: 26.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Abstract
Anhedonia is a key symptom of major depressive disorder (MDD) and comprises behavioural deficits in three reward processing subtypes: reward liking, reward wanting, and reward learning. However, neuroimaging findings regarding the neural abnormalities underpinning these deficits are complex. We have conducted a systematic review to update, reframe and summarize neuroimaging findings across the three subtypes of anhedonia in MDD. Using PubMed, The Cochrane Library, PsycINFO, and Web of Science databases, we identified 59 fMRI studies comparing participants with current or remitted MDD with controls, using reward processing tasks. For reward liking and wanting, striatal hypoactivation was observed, alongside hypoactivation and hyperactivation across frontal regions. For reward learning, blunted frontostriatal sensitivity to positive feedback was observed. These findings highlight the importance of studying anhedonia not only as a clinical manifestation but also as a neurobiological mechanism underlying depressive disorder and other broader psychiatric conditions.
Collapse
|
47
|
Jauhar S, Fortea L, Solanes A, Albajes-Eizagirre A, McKenna PJ, Radua J. Brain activations associated with anticipation and delivery of monetary reward: A systematic review and meta-analysis of fMRI studies. PLoS One 2021; 16:e0255292. [PMID: 34351957 PMCID: PMC8341642 DOI: 10.1371/journal.pone.0255292] [Citation(s) in RCA: 35] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2020] [Accepted: 07/13/2021] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND While multiple studies have examined the brain functional correlates of reward, meta-analyses have either focused on studies using the monetary incentive delay (MID) task, or have adopted a broad strategy, combining data from studies using both monetary and non-monetary reward, as probed using a wide range of tasks. OBJECTIVE To meta-analyze fMRI studies that used monetary reward and in which there was a definable cue-reward contingency. Studies were limited to those using monetary reward in order to avoid potential heterogeneity from use of other rewards, especially social rewards. Studies using gambling or delay discounting tasks were excluded on the grounds that reward anticipation is not easily quantifiable. STUDY ELIGIBILITY English-language fMRI studies (i) that reported fMRI findings on healthy adults; (ii) that used monetary reward; and (iii) in which a cue that was predictive of reward was compared to a no win (or lesser win) condition. Only voxel-based studies were included; those where brain coverage was incomplete were excluded. DATA SOURCES Ovid, Medline and PsycInfo, from 2000 to 2020, plus checking of review articles and meta-analyses. DATA SYNTHESIS Data were pooled using Seed-based d Mapping with Permutation of Subject Images (SDM-PSI). Heterogeneity among studies was examined using the I2 statistic. Publication bias was examined using funnel plots and statistical examination of asymmetries. Moderator variables including whether the task was pre-learnt, sex distribution, amount of money won and width of smoothing kernel were examined. RESULTS Pooled data from 45 studies of reward anticipation revealed activations in the ventral striatum, the middle cingulate cortex/supplementary motor area and the insula. Pooled data from 28 studies of reward delivery again revealed ventral striatal activation, plus cortical activations in the anterior and posterior cingulate cortex. There was relatively little evidence of publication bias. Among moderating variables, only whether the task was pre-learnt exerted an influence. CONCLUSIONS According to this meta-analysis monetary reward anticipation and delivery both activate the ventral but not the dorsal striatum, and are associated with different patterns of cortical activation.
Collapse
Affiliation(s)
- S. Jauhar
- Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, United Kingdom
| | - L. Fortea
- Imaging of Mood- and Anxiety-Related Disorders (IMARD) group, Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
| | - A. Solanes
- Imaging of Mood- and Anxiety-Related Disorders (IMARD) group, Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
- FIDMAG, Germanes Hospitalàries Research foundation, Barcelona, Spain
- Antonomous University of Barcelona, Barcelona, Spain
| | - A. Albajes-Eizagirre
- Imaging of Mood- and Anxiety-Related Disorders (IMARD) group, Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
- FIDMAG, Germanes Hospitalàries Research foundation, Barcelona, Spain
- Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Madrid, Spain
| | - P. J. McKenna
- Imaging of Mood- and Anxiety-Related Disorders (IMARD) group, Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
- Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Madrid, Spain
| | - J. Radua
- Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, United Kingdom
- Imaging of Mood- and Anxiety-Related Disorders (IMARD) group, Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
- FIDMAG, Germanes Hospitalàries Research foundation, Barcelona, Spain
- Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Madrid, Spain
- Center for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institute, Stockholm, Sweden
| |
Collapse
|
48
|
Zou L, Wu X, Tao S, Yang Y, Zhang Q, Hong X, Xie Y, Li T, Zheng S, Tao F. Anterior cingulate gyrus acts as a moderator of the relationship between problematic mobile phone use and depressive symptoms in college students. Soc Cogn Affect Neurosci 2021; 16:484-491. [PMID: 33522589 PMCID: PMC8094992 DOI: 10.1093/scan/nsab016] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2020] [Revised: 12/08/2020] [Accepted: 01/29/2021] [Indexed: 11/20/2022] Open
Abstract
This study aimed to investigate the brain grey matter volume (GMV) related to problematic mobile phone use (PMPU), and whether these regions of GMV play a potential moderating role in the relationship between PMPU and depressive symptoms. We recruited 266 students who underwent magnetic resonance imaging (MRI) scanning. PMPU and depressive symptoms were assessed by a self-rating questionnaire for adolescent PMPU and patient health questionnaire-9, respectively. A multiple regression model was performed to detect GMV and white matter (WM) integrity associated with PMPU by voxel-based morphometry (VBM) and tract-based spatial statistics (TBSS) methods, and the moderating analysis was conducted by PROCESS using SPSS software. VBM analysis found an inverse correlation between the GMV of the anterior cingulate gyrus (ACC) and right fusiform gyrus (FFG) with PMPU (PFDR < 0.05), and TBSS analysis revealed that fractional anisotropy (FA) in the body of the corpus callosum was negatively correlated with PMPU. The correlation between PMPU and depressive symptoms was moderated by the GMV of the ACC. These results suggest that the GMV of the ACC and right FFG, as well as FA in the body of the corpus callosum, was related to PMPU, and we further found that increased GMV of the ACC could reduce the relationship between PMPU and depressive symptoms in college students.
Collapse
Affiliation(s)
- Liwei Zou
- Department of Maternal, Child and Adolescent Health, School of Public Health, Anhui Medical University, Hefei, Anhui 230032, China.,MOE Key Laboratory of Population Health Across Life Cycle, Hefei, Anhui 230032, China.,Anhui Provincial Key Laboratory of Population Health and Aristogenics, Hefei, Anhui 230032, China
| | - Xiaoyan Wu
- Department of Maternal, Child and Adolescent Health, School of Public Health, Anhui Medical University, Hefei, Anhui 230032, China.,MOE Key Laboratory of Population Health Across Life Cycle, Hefei, Anhui 230032, China.,Anhui Provincial Key Laboratory of Population Health and Aristogenics, Hefei, Anhui 230032, China.,NHC Key Laboratory of Study on Abnormal Gametes and Reproductive Tract, Hefei, Anhui 230032, China
| | - Shuman Tao
- Department of Nephrology, The Second Hospital of Anhui Medical University, Hefei, Anhui 230601, China
| | - Yajuan Yang
- School of Nursing, Anhui Medical University, Hefei, Anhui 230601, China
| | - Qingjun Zhang
- Ping An Healthcare Diagnostics Center, Hefei, Anhui 230000, China
| | - Xuedong Hong
- Ping An Healthcare Diagnostics Center, Hefei, Anhui 230000, China
| | - Yang Xie
- Department of Maternal, Child and Adolescent Health, School of Public Health, Anhui Medical University, Hefei, Anhui 230032, China.,MOE Key Laboratory of Population Health Across Life Cycle, Hefei, Anhui 230032, China.,Anhui Provincial Key Laboratory of Population Health and Aristogenics, Hefei, Anhui 230032, China
| | - Tingting Li
- Department of Maternal, Child and Adolescent Health, School of Public Health, Anhui Medical University, Hefei, Anhui 230032, China.,MOE Key Laboratory of Population Health Across Life Cycle, Hefei, Anhui 230032, China.,Anhui Provincial Key Laboratory of Population Health and Aristogenics, Hefei, Anhui 230032, China
| | - Suisheng Zheng
- Ping An Healthcare Diagnostics Center, Hefei, Anhui 230000, China
| | - Fangbiao Tao
- Department of Maternal, Child and Adolescent Health, School of Public Health, Anhui Medical University, Hefei, Anhui 230032, China.,MOE Key Laboratory of Population Health Across Life Cycle, Hefei, Anhui 230032, China.,Anhui Provincial Key Laboratory of Population Health and Aristogenics, Hefei, Anhui 230032, China.,NHC Key Laboratory of Study on Abnormal Gametes and Reproductive Tract, Hefei, Anhui 230032, China
| |
Collapse
|
49
|
Duan LY, Horst NK, Cranmore SAW, Horiguchi N, Cardinal RN, Roberts AC, Robbins TW. Controlling one's world: Identification of sub-regions of primate PFC underlying goal-directed behavior. Neuron 2021; 109:2485-2498.e5. [PMID: 34171290 PMCID: PMC8346232 DOI: 10.1016/j.neuron.2021.06.003] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2020] [Revised: 04/13/2021] [Accepted: 06/02/2021] [Indexed: 12/30/2022]
Abstract
Impaired detection of causal relationships between actions and their outcomes can lead to maladaptive behavior. However, causal roles of specific prefrontal cortex (PFC) sub-regions and the caudate nucleus in mediating such relationships in primates are unclear. We inactivated and overactivated five PFC sub-regions, reversibly and pharmacologically: areas 24 (perigenual anterior cingulate cortex), 32 (medial PFC), 11 (anterior orbitofrontal cortex, OFC), 14 (rostral ventromedial PFC/medial OFC), and 14-25 (caudal ventromedial PFC) and the anteromedial caudate to examine their role in expressing learned action-outcome contingencies using a contingency degradation paradigm in marmoset monkeys. Area 24 or caudate inactivation impaired the response to contingency change, while area 11 inactivation enhanced it, and inactivation of areas 14, 32, or 14-25 had no effect. Overactivation of areas 11 and 24 impaired this response. These findings demonstrate the distinct roles of PFC sub-regions in goal-directed behavior and illuminate the candidate neurobehavioral substrates of psychiatric disorders, including obsessive-compulsive disorder. Monkey pgACC-24 is necessary for detecting causal control of actions over outcomes Its projection target in the caudate nucleus is also implicated Three other subregions of the ventromedial prefrontal cortex are not necessary Anterior OFC-11 may mediate Pavlovian influences on goal-directed behavior
Collapse
Affiliation(s)
- Lisa Y Duan
- Department of Psychology, University of Cambridge, Downing Street, Cambridge CB2 3EB, UK; Behavioural and Clinical Neuroscience Institute, Downing Street, University of Cambridge, Cambridge CB2 3EB, UK.
| | - Nicole K Horst
- Department of Psychology, University of Cambridge, Downing Street, Cambridge CB2 3EB, UK; Behavioural and Clinical Neuroscience Institute, Downing Street, University of Cambridge, Cambridge CB2 3EB, UK
| | - Stacey A W Cranmore
- Department of Psychology, University of Cambridge, Downing Street, Cambridge CB2 3EB, UK; Behavioural and Clinical Neuroscience Institute, Downing Street, University of Cambridge, Cambridge CB2 3EB, UK
| | - Naotaka Horiguchi
- Department of Psychology, University of Cambridge, Downing Street, Cambridge CB2 3EB, UK; Behavioural and Clinical Neuroscience Institute, Downing Street, University of Cambridge, Cambridge CB2 3EB, UK
| | - Rudolf N Cardinal
- Department of Psychiatry, University of Cambridge, Herchel Smith Building for Brain & Mind Sciences, Forvie Site, Robinson Way, Cambridge CB2 0SZ, UK; Behavioural and Clinical Neuroscience Institute, Downing Street, University of Cambridge, Cambridge CB2 3EB, UK; Cambridgeshire and Peterborough NHS Foundation Trust, Liaison Psychiatry Service, Box 190, Cambridge Biomedical Campus, Cambridge CB2 0QQ, UK
| | - Angela C Roberts
- Department of Physiology, Development and Neuroscience, University of Cambridge, Downing Street, Cambridge CB2 3DY, UK; Behavioural and Clinical Neuroscience Institute, Downing Street, University of Cambridge, Cambridge CB2 3EB, UK
| | - Trevor W Robbins
- Department of Psychology, University of Cambridge, Downing Street, Cambridge CB2 3EB, UK; Behavioural and Clinical Neuroscience Institute, Downing Street, University of Cambridge, Cambridge CB2 3EB, UK
| |
Collapse
|
50
|
Hu Y, van Wingerden M, Sellitto M, Schäble S, Kalenscher T. Anterior Cingulate Cortex Lesions Abolish Budget Effects on Effort-Based Decision-Making in Rat Consumers. J Neurosci 2021; 41:4448-4460. [PMID: 33753545 PMCID: PMC8152606 DOI: 10.1523/jneurosci.2541-20.2021] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2020] [Revised: 02/26/2021] [Accepted: 02/27/2021] [Indexed: 11/21/2022] Open
Abstract
Demand theory can be applied to analyze how animal consumers change their selection of commodities in response to changes in commodity prices, given budget constraints. Previous work has shown that demand elasticities in rats differed between uncompensated budget conditions in which the budget available to be spent on the commodities (e.g., the finite number of discrete operants to "purchase" rewards in two-alternative fixed-ratio schedules) was kept constant, and compensated budget conditions in which the budget was adjusted so that consumers could potentially continue to obtain the original reward bundles. Here, we hypothesized that rat anterior cingulate cortex (ACC) was necessary to produce this budget effect on demand elasticities. We applied excitotoxic or sham lesions to ACC in rats performing an effort task in which the prices of liquid vanilla or chocolate rewards (the effort required to obtain rewards) and the budget (the total number of operants) was manipulated. When reward prices changed, and the budget was compensated, all rats adjusted their demand for chocolate and vanilla accordingly. In sham-lesioned rats, changes in demand were even more pronounced when the budget was not compensated for the price changes. By contrast, ACC-lesioned animals did not show this additional budget effect. An in-depth comparison of the rats' choice patterns showed that, unlike sham rats, ACC-lesioned animals failed to maximize session-bundle utility after price/budget changes, revealing deficits in higher-order choice-strategy adaptations. Our results suggest a novel role of ACC in considering purchasing power during complex cost-benefit value computations.SIGNIFICANCE STATEMENT Anterior cingulate cortex (ACC) is important for allocating effort in cost-benefit calculations in animals and humans. Economic theory prescribes that the value of the costs in cost-benefit analyses not only depends on the net nominal costs required to purchase a reward, but also on the available budget resources, i.e., on the budget's "purchasing value." We asked whether ACC, a region implicated in effort-based decision-making and reward comparisons, is required for computing the value of effort relative to a budget constraint. Applying demand theory to describe rat choices in a rodent effort allocation task with varying effort prices and budgets, we show that ACC integrity was necessary for computing purchasing power, a core variable in economic choice theory.
Collapse
Affiliation(s)
- Yue Hu
- Comparative Psychology, Institute of Experimental Psychology, Heinrich Heine University Düsseldorf, Düsseldorf 40225, Germany
| | - Marijn van Wingerden
- Department of Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg 5037 AB, The Netherlands
| | - Manuela Sellitto
- Comparative Psychology, Institute of Experimental Psychology, Heinrich Heine University Düsseldorf, Düsseldorf 40225, Germany
| | - Sandra Schäble
- Comparative Psychology, Institute of Experimental Psychology, Heinrich Heine University Düsseldorf, Düsseldorf 40225, Germany
| | - Tobias Kalenscher
- Comparative Psychology, Institute of Experimental Psychology, Heinrich Heine University Düsseldorf, Düsseldorf 40225, Germany
| |
Collapse
|