1
|
Ortiz-Tudela J, Turan G, Vilas M, Melloni L, Shing YL. Schema-driven prediction effects on episodic memory across the lifespan. Philos Trans R Soc Lond B Biol Sci 2024; 379:20230401. [PMID: 39278241 PMCID: PMC11449153 DOI: 10.1098/rstb.2023.0401] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2024] [Revised: 04/12/2024] [Accepted: 04/30/2024] [Indexed: 09/18/2024] Open
Abstract
The predictive processing framework posits that one of the main functions of the brain is to anticipate the incoming information. Internal models facilitate interactions with the world by predicting future states against which actual evidence is compared. The difference between predicted and actual states, the prediction error (PE), signals novel information. However, how PE affects cognitive processing downstream is not fully understood: one such aspect pertains to how PE influences episodic memories, and whether those effect on memory differ across the lifespan. We examine the relationship between PE and episodic memory in children, young and older adults. We use a novel paradigm whereby rich visual narratives are used to build action schemas that enable probing different mnemonic aspects. To create different levels of PE, we manipulate the story endings to be either expected, neutral or unexpected with respect to the unfolded action. We show that (i) expected endings are better encoded than neutral endings and (ii) unexpected endings improve the encoding of mismatching events and other aspects of the narrative. These effects are differentially modulated across the lifespan with PE-driven encoding being more prominent in children and young adults and with schema integration playing a larger role on memory encoding in older adults. These results highlight the role of predictions by enriching past experiences and informing future anticipations.This article is part of the theme issue 'Elements of episodic memory: lessons from 40 years of research'.
Collapse
Affiliation(s)
- Javier Ortiz-Tudela
- Mind, Brain, and Behavior Research Center, Department of Experimental Psychology, University of Granada, Granada, Spain
| | - Gözem Turan
- Department of Psychology, Goethe University Frankfurt, Frankfurt, Germany
- Center for Individual Development and Adaptive Education of Children at Risk (IDeA), Frankfurt, Germany
| | - Martina Vilas
- Research Group Neural Circuits, Consciousness and Cognition, Max-Planck Institute for Empirical Aesthetics, Frankfurt, Germany
| | - Lucia Melloni
- Research Group Neural Circuits, Consciousness and Cognition, Max-Planck Institute for Empirical Aesthetics, Frankfurt, Germany
| | - Yee Lee Shing
- Department of Psychology, Goethe University Frankfurt, Frankfurt, Germany
- Center for Individual Development and Adaptive Education of Children at Risk (IDeA), Frankfurt, Germany
| |
Collapse
|
2
|
Shteyn MR, Olson CR. Neurons of Macaque Frontal Eye Field Signal Reward-Related Surprise. J Neurosci 2024; 44:e0441242024. [PMID: 39107059 PMCID: PMC11411596 DOI: 10.1523/jneurosci.0441-24.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2024] [Revised: 07/26/2024] [Accepted: 07/30/2024] [Indexed: 08/09/2024] Open
Abstract
The frontal eye field (FEF) plays a well-established role in the control of visual attention. The strength of an FEF neuron's response to a visual stimulus presented in its receptive field is enhanced if the stimulus captures spatial attention by virtue of its salience. A stimulus can be rendered salient by cognitive factors as well as by physical attributes. These include surprise. The aim of the present experiment was to determine whether surprise-induced salience would result in enhanced visual-response strength in the FEF. Toward this end, we monitored neuronal activity in two male monkeys while presenting first a visual cue predicting with high probability that the reward delivered at the end of the trial would be good or bad (large or small) and then a visual cue announcing the size of the impending reward with certainty. The second cue usually confirmed but occasionally violated the expectation set up by the first cue. Neurons responded more strongly to the second cue when it violated than when it confirmed expectation. The increase in the firing rate was accompanied by a decrease in spike-count correlation as expected from capture of attention. Although both good surprise and bad surprise induced enhanced firing, the effects appeared to arise from distinct mechanisms as indicated by the fact that the bad-surprise signal appeared at a longer latency than the good-surprise signal and by the fact that the strength of the two signals varied independently across neurons.
Collapse
Affiliation(s)
- Michael R Shteyn
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
- Department of Neuroscience, University of Pittsburgh, Pittsburgh, Pennsylvania 15260
| | - Carl R Olson
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
- Department of Neuroscience, University of Pittsburgh, Pittsburgh, Pennsylvania 15260
| |
Collapse
|
3
|
Franco JP, Bossaerts P, Murawski C. The neural dynamics associated with computational complexity. PLoS Comput Biol 2024; 20:e1012447. [PMID: 39312586 PMCID: PMC11449275 DOI: 10.1371/journal.pcbi.1012447] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Revised: 10/03/2024] [Accepted: 08/30/2024] [Indexed: 09/25/2024] Open
Abstract
Many everyday tasks require people to solve computationally complex problems. However, little is known about the effects of computational hardness on the neural processes associated with solving such problems. Here, we draw on computational complexity theory to address this issue. We performed an experiment in which participants solved several instances of the 0-1 knapsack problem, a combinatorial optimization problem, while undergoing ultra-high field (7T) functional magnetic resonance imaging (fMRI). Instances varied in computational hardness. We characterize a network of brain regions whose activation was correlated with computational complexity, including the anterior insula, dorsal anterior cingulate cortex and the intra-parietal sulcus/angular gyrus. Activation and connectivity changed dynamically as a function of complexity, in line with theoretical computational requirements. Overall, our results suggest that computational complexity theory provides a suitable framework to study the effects of computational hardness on the neural processes associated with solving complex cognitive tasks.
Collapse
Affiliation(s)
- Juan Pablo Franco
- Centre for Brain, Mind and Markets The University of Melbourne, Melbourne, Victoria, Australia
| | - Peter Bossaerts
- Centre for Brain, Mind and Markets The University of Melbourne, Melbourne, Victoria, Australia
- Faculty of Economics, Cambridge University, Cambridge, United Kingdom
| | - Carsten Murawski
- Centre for Brain, Mind and Markets The University of Melbourne, Melbourne, Victoria, Australia
| |
Collapse
|
4
|
Alí Diez I, Fàbrega-Camps G, Parra-Tíjaro J, Marco-Pallarés J. Anticipatory and consummatory neural correlates of monetary and music rewarding stimuli. Brain Cogn 2024; 179:106186. [PMID: 38843763 DOI: 10.1016/j.bandc.2024.106186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Revised: 05/29/2024] [Accepted: 05/29/2024] [Indexed: 06/17/2024]
Abstract
Most of the literature on the neural bases of human reward and punishment processing has used monetary gains and losses, but less is known about the neurophysiological mechanisms underlying the anticipation and consumption of other types of rewarding stimuli. In the present study, EEG was recorded from 19 participants who completed a modified version of the Monetary Incentive Delay (MID) task. During the task, cues providing information about potential future outcomes were presented to the participants. Then, they had to respond rapidly to a target stimulus to win money or listening to pleasant music, or to avoid losing money or listening to unpleasant music. Results revealed similar responses for monetary and music cues, with increased activity for cues indicating potential gains compared to losses. However, differences emerged in the outcome phase between money and music. Monetary outcomes showed an interaction between the type of the cue and the outcome in the Feedback Related Negativity and Fb-P3 ERPs and increased theta activity increased for negative feedbacks. In contrast, music outcomes showed significant interactions in the Fb-P3 and theta activities. These findings suggest similar neurophysiological mechanisms in processing cues for potential positive or negative outcomes in these two types of stimuli.
Collapse
Affiliation(s)
- Italo Alí Diez
- Department of Cognition, Development and Educational Psychology, Institute of Neurosciences, University of Barcelona, Spain; Bellvitge Biomedical Research Institute (IDIBELL), Spain; Department of Psychology, University of La Frontera, Chile
| | - Gemma Fàbrega-Camps
- Department of Cognition, Development and Educational Psychology, Institute of Neurosciences, University of Barcelona, Spain; Bellvitge Biomedical Research Institute (IDIBELL), Spain
| | - Jeison Parra-Tíjaro
- Department of Cognition, Development and Educational Psychology, Institute of Neurosciences, University of Barcelona, Spain; Bellvitge Biomedical Research Institute (IDIBELL), Spain
| | - Josep Marco-Pallarés
- Department of Cognition, Development and Educational Psychology, Institute of Neurosciences, University of Barcelona, Spain; Bellvitge Biomedical Research Institute (IDIBELL), Spain.
| |
Collapse
|
5
|
Kondo HM, Oba T, Ezaki T, Kochiyama T, Shimada Y, Ohira H. Striatal GABA levels correlate with risk sensitivity in monetary loss. Front Neurosci 2024; 18:1439656. [PMID: 39145302 PMCID: PMC11321969 DOI: 10.3389/fnins.2024.1439656] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2024] [Accepted: 07/17/2024] [Indexed: 08/16/2024] Open
Abstract
Background Decision-making under risk is a common challenge. It is known that risk-taking behavior varies between contexts of reward and punishment, yet the mechanisms underlying this asymmetry in risk sensitivity remain unclear. Methods This study used a monetary task to investigate neurochemical mechanisms and brain dynamics underpinning risk sensitivity. Twenty-eight participants engaged in a task requiring selection of visual stimuli to maximize monetary gains and minimize monetary losses. We modeled participant trial-and-error processes using reinforcement learning. Results Participants with higher subjective utility parameters showed risk preference in the gain domain (r = -0.59) and risk avoidance in the loss domain (r = -0.77). Magnetic resonance spectroscopy (MRS) revealed that risk avoidance in the loss domain was associated with γ-aminobutyric acid (GABA) levels in the ventral striatum (r = -0.42), but not in the insula (r = -0.15). Using functional magnetic resonance imaging (fMRI), we tested whether risk-sensitive brain dynamics contribute to participant risky choices. Energy landscape analyses demonstrated that higher switching rates between brain states, including the striatum and insula, were correlated with risk avoidance in the loss domain (r = -0.59), a relationship not observed in the gain domain (r = -0.02). Conclusions These findings from MRS and fMRI suggest that distinct mechanisms are involved in gain/loss decision making, mediated by subcortical neurometabolite levels and brain dynamic transitions.
Collapse
Affiliation(s)
| | - Takeyuki Oba
- Graduate School of Informatics, Nagoya University, Nagoya, Aichi, Japan
| | - Takahiro Ezaki
- Precursory Research for Embryonic Science and Technology, Japan Science and Technology Agency, Kawaguchi, Saitama, Japan
- Research Center for Advanced Science and Technology, University of Tokyo, Tokyo, Japan
| | | | - Yasuhiro Shimada
- Advanced ICT Research Institute, National Institute of Information and Communications Technology, Osaka, Japan
| | - Hideki Ohira
- Graduate School of Informatics, Nagoya University, Nagoya, Aichi, Japan
| |
Collapse
|
6
|
Combrisson E, Basanisi R, Gueguen MCM, Rheims S, Kahane P, Bastin J, Brovelli A. Neural interactions in the human frontal cortex dissociate reward and punishment learning. eLife 2024; 12:RP92938. [PMID: 38941238 PMCID: PMC11213568 DOI: 10.7554/elife.92938] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/30/2024] Open
Abstract
How human prefrontal and insular regions interact while maximizing rewards and minimizing punishments is unknown. Capitalizing on human intracranial recordings, we demonstrate that the functional specificity toward reward or punishment learning is better disentangled by interactions compared to local representations. Prefrontal and insular cortices display non-selective neural populations to rewards and punishments. Non-selective responses, however, give rise to context-specific interareal interactions. We identify a reward subsystem with redundant interactions between the orbitofrontal and ventromedial prefrontal cortices, with a driving role of the latter. In addition, we find a punishment subsystem with redundant interactions between the insular and dorsolateral cortices, with a driving role of the insula. Finally, switching between reward and punishment learning is mediated by synergistic interactions between the two subsystems. These results provide a unifying explanation of distributed cortical representations and interactions supporting reward and punishment learning.
Collapse
Affiliation(s)
- Etienne Combrisson
- Institut de Neurosciences de la Timone, Aix Marseille UniversitéMarseilleFrance
| | - Ruggero Basanisi
- Institut de Neurosciences de la Timone, Aix Marseille UniversitéMarseilleFrance
| | - Maelle CM Gueguen
- Univ. Grenoble Alpes, Inserm, U1216, Grenoble Institut NeurosciencesGrenobleFrance
| | - Sylvain Rheims
- Department of Functional Neurology and Epileptology, Hospices Civils de Lyon and University of LyonLyonFrance
| | - Philippe Kahane
- Univ. Grenoble Alpes, Inserm, U1216, CHU Grenoble Alpes, Grenoble Institut NeurosciencesGrenobleFrance
| | - Julien Bastin
- Univ. Grenoble Alpes, Inserm, U1216, Grenoble Institut NeurosciencesGrenobleFrance
| | - Andrea Brovelli
- Institut de Neurosciences de la Timone, Aix Marseille UniversitéMarseilleFrance
| |
Collapse
|
7
|
Zheltyakova M, Korotkov A, Cherednichenko D, Didur M, Kireev M. To lie or to tell the truth? The influence of processing the opponent's feedback on the forthcoming choice. Front Psychol 2024; 15:1275884. [PMID: 38784609 PMCID: PMC11112074 DOI: 10.3389/fpsyg.2024.1275884] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2023] [Accepted: 04/22/2024] [Indexed: 05/25/2024] Open
Abstract
Introduction The brain mechanisms of deceptive behavior are relatively well studied, and the key brain regions involved in its processing were established. At the same time, the brain mechanisms underlying the processes of preparation for deception are less known. Methods We studied BOLD-signal changes during the presentation of the opponent's feedback to a previous deceptive or honest action during the computer game. The goal of the game was to mislead the opponent either by means of deception or by means of telling the truth. Results As a result, it was shown that several brain regions that were previously demonstrated as involved in deception execution, such as the left anterior cingulate cortex and anterior insula, also underlie processes related to deception preparation. Discussion The results obtained also allowed us to suggest that brain regions responsible for performance monitoring, intention assessment, suppression of non-selected solutions, and reward processing could be involved in shaping future action selection and preparation for deception. By shedding light on the brain mechanisms underlying deception, our study contributes to a deeper understanding of this complex cognitive process. Furthermore, it emphasizes the significance of exploring brain mechanisms governing the choice between deception and truth at various stages of decision-making.
Collapse
Affiliation(s)
| | | | | | | | - Maxim Kireev
- N.P. Bechtereva Institute of the Human Brain, Russian Academy of Science, Saint Petersburg, Russia
| |
Collapse
|
8
|
Clairis N, Pessiglione M. Value Estimation versus Effort Mobilization: A General Dissociation between Ventromedial and Dorsomedial Prefrontal Cortex. J Neurosci 2024; 44:e1176232024. [PMID: 38514180 PMCID: PMC11044108 DOI: 10.1523/jneurosci.1176-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Revised: 03/01/2024] [Accepted: 03/03/2024] [Indexed: 03/23/2024] Open
Abstract
Deciding on a course of action requires both an accurate estimation of option values and the right amount of effort invested in deliberation to reach sufficient confidence in the final choice. In a previous study, we have provided evidence, across a series of judgment and choice tasks, for a dissociation between the ventromedial prefrontal cortex (vmPFC), which would represent option values, and the dorsomedial prefrontal cortex (dmPFC), which would represent the duration of deliberation. Here, we first replicate this dissociation and extend it to the case of an instrumental learning task, in which 24 human volunteers (13 women) choose between options associated with probabilistic gains and losses. According to fMRI data recorded during decision-making, vmPFC activity reflects the sum of option values generated by a reinforcement learning model and dmPFC activity the deliberation time. To further generalize the role of the dmPFC in mobilizing effort, we then analyze fMRI data recorded in the same participants while they prepare to perform motor and cognitive tasks (squeezing a handgrip or making numerical comparisons) to maximize gains or minimize losses. In both cases, dmPFC activity is associated with the output of an effort regulation model, and not with response time. Taken together, these results strengthen a general theory of behavioral control that implicates the vmPFC in the estimation of option values and the dmPFC in the energization of relevant motor and cognitive processes.
Collapse
Affiliation(s)
- Nicolas Clairis
- Motivation, Brain and Behavior team, Paris Brain Institute (ICM), Paris 75013, France
- CNRS U7225, Inserm U1127, Sorbonne Université, Paris 75005, France
- Laboratory of Behavioral Genetics (LGC), Brain Mind Institute (BMI), École Polytechnique Fédérale de Lausanne (EPFL), Lausanne 1004, Switzerland
| | - Mathias Pessiglione
- Motivation, Brain and Behavior team, Paris Brain Institute (ICM), Paris 75013, France
- CNRS U7225, Inserm U1127, Sorbonne Université, Paris 75005, France
| |
Collapse
|
9
|
Garlichs A, Blank H. Prediction error processing and sharpening of expected information across the face-processing hierarchy. Nat Commun 2024; 15:3407. [PMID: 38649694 PMCID: PMC11035707 DOI: 10.1038/s41467-024-47749-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Accepted: 04/10/2024] [Indexed: 04/25/2024] Open
Abstract
The perception and neural processing of sensory information are strongly influenced by prior expectations. The integration of prior and sensory information can manifest through distinct underlying mechanisms: focusing on unexpected input, denoted as prediction error (PE) processing, or amplifying anticipated information via sharpened representation. In this study, we employed computational modeling using deep neural networks combined with representational similarity analyses of fMRI data to investigate these two processes during face perception. Participants were cued to see face images, some generated by morphing two faces, leading to ambiguity in face identity. We show that expected faces were identified faster and perception of ambiguous faces was shifted towards priors. Multivariate analyses uncovered evidence for PE processing across and beyond the face-processing hierarchy from the occipital face area (OFA), via the fusiform face area, to the anterior temporal lobe, and suggest sharpened representations in the OFA. Our findings support the proposition that the brain represents faces grounded in prior expectations.
Collapse
Affiliation(s)
- Annika Garlichs
- Department of Systems Neuroscience, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany.
| | - Helen Blank
- Department of Systems Neuroscience, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany.
| |
Collapse
|
10
|
Zárate-Rochín AM. Contemporary neurocognitive models of memory: A descriptive comparative analysis. Neuropsychologia 2024; 196:108846. [PMID: 38430963 DOI: 10.1016/j.neuropsychologia.2024.108846] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Revised: 02/27/2024] [Accepted: 02/27/2024] [Indexed: 03/05/2024]
Abstract
The great complexity involved in the study of memory has given rise to numerous hypotheses and models associated with various phenomena at different levels of analysis. This has allowed us to delve deeper in our knowledge about memory but has also made it difficult to synthesize and integrate data from different lines of research. In this context, this work presents a descriptive comparative analysis of contemporary models that address the structure and function of multiple memory systems. The main goal is to outline a panoramic view of the key elements that constitute these models in order to visualize both the current state of research and possible future directions. The elements that stand out from different levels of analysis are distributed neural networks, hierarchical organization, predictive coding, homeostasis, and evolutionary perspective.
Collapse
Affiliation(s)
- Alba Marcela Zárate-Rochín
- Instituto de Investigaciones Cerebrales, Universidad Veracruzana, Dr. Castelazo Ayala s/n, Industrial Animas, 91190, Xalapa-Enríquez, Veracruz, Mexico.
| |
Collapse
|
11
|
Fouragnan EF, Hosking B, Cheung Y, Prakash B, Rushworth M, Sel A. Timing along the cardiac cycle modulates neural signals of reward-based learning. Nat Commun 2024; 15:2976. [PMID: 38582905 PMCID: PMC10998831 DOI: 10.1038/s41467-024-46921-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Accepted: 03/14/2024] [Indexed: 04/08/2024] Open
Abstract
Natural fluctuations in cardiac activity modulate brain activity associated with sensory stimuli, as well as perceptual decisions about low magnitude, near-threshold stimuli. However, little is known about the relationship between fluctuations in heart activity and other internal representations. Here we investigate whether the cardiac cycle relates to learning-related internal representations - absolute and signed prediction errors. We combined machine learning techniques with electroencephalography with both simple, direct indices of task performance and computational model-derived indices of learning. Our results demonstrate that just as people are more sensitive to low magnitude, near-threshold sensory stimuli in certain cardiac phases, so are they more sensitive to low magnitude absolute prediction errors in the same cycles. However, this occurs even when the low magnitude prediction errors are associated with clearly suprathreshold sensory events. In addition, participants exhibiting stronger differences in their prediction error representations between cardiac cycles exhibited higher learning rates and greater task accuracy.
Collapse
Affiliation(s)
- Elsa F Fouragnan
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK.
- Brain Research Imaging Centre (BRIC), Faculty of Health, University of Plymouth, Plymouth, PL6 8BU, UK.
- School of Psychology, Faculty of Health, University of Plymouth, Plymouth, PL4 8AA, UK.
| | - Billy Hosking
- Brain Research Imaging Centre (BRIC), Faculty of Health, University of Plymouth, Plymouth, PL6 8BU, UK
- School of Psychology, Faculty of Health, University of Plymouth, Plymouth, PL4 8AA, UK
| | - Yin Cheung
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK
| | - Brooke Prakash
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK
| | - Matthew Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK
| | - Alejandra Sel
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK
- Centre for Brain Science, Department of Psychology, University of Essex, Wivenhoe Park, Colchester, CO4 3SQ, UK
- Essex ESNEFT Psychological Research Unit for Behaviour, Health and Wellbeing, University of Essex, Wivenhoe Park, Colchester, CO4 3SQ, UK
| |
Collapse
|
12
|
Brevers D, Baeken C, Bechara A, He Q, Maurage P, Sescousse G, Vögele C, Billieux J. Increased ventral anterior insular connectivity to sports betting availability indexes problem gambling. Addict Biol 2024; 29:e13389. [PMID: 38516877 PMCID: PMC11061852 DOI: 10.1111/adb.13389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Revised: 02/06/2024] [Accepted: 02/20/2024] [Indexed: 03/23/2024]
Abstract
With the advent of digital technologies, online sports betting is spurring a fast-growing expansion. In this study, we examined how sports betting availability modulates the brain connectivity of frequent sports bettors with [problem bettors (PB)] or without [non-problem bettors (NPB)] problematic sports betting. We conducted functional connectivity analyses centred on the ventral anterior insular cortex (vAI), a brain region playing a key role in the dynamic interplay between reward-based processes. We re-analysed a dataset on sports betting availability undertaken in PB (n = 30) and NPB (n = 35). Across all participants, we observed that sports betting availability elicited positive vAI coupling with extended clusters of brain activation (encompassing the putamen, cerebellum, occipital, temporal, precentral and central operculum regions) and negative vAI coupling with the orbitofrontal cortex. Between-group analyses showed increased positive vAI coupling in the PB group, as compared with the NPB group, in the left lateral occipital cortex, extending to the left inferior frontal gyrus, the anterior cingulate gyrus and the right frontal pole. Taken together, these results are in line with the central assumptions of triadic models of addictions, which posit that the insular cortex plays a pivotal role in promoting the drive and motivation to get a reward by 'hijacking' goal-oriented processes toward addiction-related cues. Taken together, these findings showed that vAI functional connectivity is sensitive not only to gambling availability but also to the status of problematic sport betting.
Collapse
Affiliation(s)
- Damien Brevers
- Louvain for Experimental Psychopathology Research Group (LEP), Psychological Sciences Research InstituteUCLouvainLouvain‐la‐NeuveBelgium
- Department of Behavioural and Cognitive Sciences, Institute for Health and BehaviourUniversity of LuxembourgEsch‐sur‐AlzetteLuxembourg
| | - Chris Baeken
- Department of PsychiatryUZ BrusselBrusselsBelgium
- Department of Head and Skin, Ghent Experimental Psychiatry (GHEP) LabGhent University Hospital, Ghent UniversityGhentBelgium
- Department of Electrical EngineeringEindhoven University of TechnologyEindhovenThe Netherlands
| | - Antoine Bechara
- Department of PsychologyUniversity of Southern CaliforniaCaliforniaLos AngelesUSA
| | - Qinghua He
- Faculty of PsychologySouthwest UniversityChongqingChina
| | - Pierre Maurage
- Louvain for Experimental Psychopathology Research Group (LEP), Psychological Sciences Research InstituteUCLouvainLouvain‐la‐NeuveBelgium
| | - Guillaume Sescousse
- Lyon Neuroscience Research Center—INSERM U1028—CNRS UMR5292, PSYR2 TeamUniversity of LyonLyonFrance
| | - Claus Vögele
- Department of Behavioural and Cognitive Sciences, Institute for Health and BehaviourUniversity of LuxembourgEsch‐sur‐AlzetteLuxembourg
| | - Joël Billieux
- Institute of PsychologyUniversity of LausanneLausanneSwitzerland
- Centre for Excessive Gambling, Addiction MedicineLausanne University Hospitals (CHUV)LausanneSwitzerland
| |
Collapse
|
13
|
Halahakoon DC, Kaltenboeck A, Martens M, Geddes JG, Harmer CJ, Cowen P, Browning M. Pramipexole Enhances Reward Learning by Preserving Value Estimates. Biol Psychiatry 2024; 95:286-296. [PMID: 37330165 DOI: 10.1016/j.biopsych.2023.05.023] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 05/02/2023] [Accepted: 05/29/2023] [Indexed: 06/19/2023]
Abstract
BACKGROUND Dopamine D2-like agonists show promise as treatments for depression. They are thought to act by enhancing reward learning; however, the mechanisms by which they achieve this are not clear. Reinforcement learning accounts describe 3 distinct candidate mechanisms: increased reward sensitivity, increased inverse decision-temperature, and decreased value decay. As these mechanisms produce equivalent effects on behavior, arbitrating between them requires measurement of how expectations and prediction errors are altered. We characterized the effects of 2 weeks of the D2-like agonist pramipexole on reward learning and used functional magnetic resonance imaging measures of expectation and prediction error to assess which of these 3 mechanistic processes were responsible for the behavioral effects. METHODS Forty healthy volunteers (50% female) were randomized to 2 weeks of pramipexole (titrated to 1 mg/day) or placebo in a double-blind, between-subject design. Participants completed a probabilistic instrumental learning task before and after the pharmacological intervention, with functional magnetic resonance imaging data collected at the second visit. Asymptotic choice accuracy and a reinforcement learning model were used to assess reward learning. RESULTS Pramipexole increased choice accuracy in the reward condition with no effect on losses. Participants who received pramipexole had increased blood oxygen level-dependent response in the orbital frontal cortex during the expectation of win trials but decreased blood oxygen level-dependent response to reward prediction errors in the ventromedial prefrontal cortex. This pattern of results indicates that pramipexole enhances choice accuracy by reducing the decay of estimated values during reward learning. CONCLUSIONS The D2-like receptor agonist pramipexole enhances reward learning by preserving learned values. This is a plausible mechanism for pramipexole's antidepressant effect.
Collapse
Affiliation(s)
- Don Chamith Halahakoon
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom; Oxford Health National Health Service Foundation Trust, Warneford Hospital, Oxford, United Kingdom
| | - Alexander Kaltenboeck
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom; Clinical Division of Social Psychiatry, Department of Psychiatry and Psychotherapy, Medical University of Vienna, Vienna, Austria
| | - Marieke Martens
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom
| | - John G Geddes
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom; Oxford Health National Health Service Foundation Trust, Warneford Hospital, Oxford, United Kingdom
| | - Catherine J Harmer
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom; Oxford Health National Health Service Foundation Trust, Warneford Hospital, Oxford, United Kingdom
| | - Philip Cowen
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom; Oxford Health National Health Service Foundation Trust, Warneford Hospital, Oxford, United Kingdom
| | - Michael Browning
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom; Oxford Health National Health Service Foundation Trust, Warneford Hospital, Oxford, United Kingdom.
| |
Collapse
|
14
|
Batsikadze G, Pakusch J, Klein M, Ernst TM, Thieme A, Nicksirat SA, Steiner KM, Nio E, Genc E, Maderwald S, Deuschl C, Merz CJ, Quick HH, Mark MD, Timmann D. Mild Deficits in Fear Learning: Evidence from Humans and Mice with Cerebellar Cortical Degeneration. eNeuro 2024; 11:ENEURO.0365-23.2023. [PMID: 38176906 PMCID: PMC10897646 DOI: 10.1523/eneuro.0365-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Revised: 11/21/2023] [Accepted: 12/08/2023] [Indexed: 01/06/2024] Open
Abstract
Functional brain imaging studies in humans suggest involvement of the cerebellum in fear conditioning but do not allow conclusions about the functional significance. The main aim of the present study was to examine whether patients with cerebellar degeneration show impaired fear conditioning and whether this is accompanied by alterations in cerebellar cortical activations. To this end, a 2 d differential fear conditioning study was conducted in 20 cerebellar patients and 21 control subjects using a 7 tesla (7 T) MRI system. Fear acquisition and extinction training were performed on day 1, followed by recall on day 2. Cerebellar patients learned to differentiate between the CS+ and CS-. Acquisition and consolidation of learned fear, however, was slowed. Additionally, extinction learning appeared to be delayed. The fMRI signal was reduced in relation to the prediction of the aversive stimulus and altered in relation to its unexpected omission. Similarly, mice with cerebellar cortical degeneration (spinocerebellar ataxia type 6, SCA6) were able to learn the fear association, but retrieval of fear memory was reduced. In sum, cerebellar cortical degeneration led to mild abnormalities in the acquisition of learned fear responses in both humans and mice, particularly manifesting postacquisition training. Future research is warranted to investigate the basis of altered fMRI signals related to fear learning.
Collapse
Affiliation(s)
- Giorgi Batsikadze
- Department of Neurology and Center for Translational Neuro- and Behavioral Sciences (C-TNBS), Essen University Hospital, University of Duisburg-Essen, 45147 Essen, Germany
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, 45141 Essen, Germany
| | - Johanna Pakusch
- Behavioral Neuroscience, Ruhr University Bochum, 44801 Bochum, Germany
| | - Michael Klein
- Department of Neurology and Center for Translational Neuro- and Behavioral Sciences (C-TNBS), Essen University Hospital, University of Duisburg-Essen, 45147 Essen, Germany
| | - Thomas Michael Ernst
- Department of Neurology and Center for Translational Neuro- and Behavioral Sciences (C-TNBS), Essen University Hospital, University of Duisburg-Essen, 45147 Essen, Germany
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, 45141 Essen, Germany
| | - Andreas Thieme
- Department of Neurology and Center for Translational Neuro- and Behavioral Sciences (C-TNBS), Essen University Hospital, University of Duisburg-Essen, 45147 Essen, Germany
| | - Seyed Ali Nicksirat
- Department of Neurology and Center for Translational Neuro- and Behavioral Sciences (C-TNBS), Essen University Hospital, University of Duisburg-Essen, 45147 Essen, Germany
| | - Katharina Marie Steiner
- Department of Neurology and Center for Translational Neuro- and Behavioral Sciences (C-TNBS), Essen University Hospital, University of Duisburg-Essen, 45147 Essen, Germany
- LVR-Hospital Essen, Department of Psychiatry and Psychotherapy, Medical Faculty, University of Duisburg-Essen, 45147 Essen, Germany
| | - Enzo Nio
- Department of Neurology and Center for Translational Neuro- and Behavioral Sciences (C-TNBS), Essen University Hospital, University of Duisburg-Essen, 45147 Essen, Germany
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, 45141 Essen, Germany
| | - Erhan Genc
- Department of Psychology and Neurosciences, Leibniz Research Centre for Working Environment and Human Factors, Technical University of Dortmund (IfADo), 44139 Dortmund, Germany
| | - Stefan Maderwald
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, 45141 Essen, Germany
| | - Cornelius Deuschl
- Institute of Diagnostic and Interventional Radiology and Neuroradiology and C-TNBS, Essen University Hospital, University of Duisburg-Essen, 45147 Essen, Germany
| | - Christian Josef Merz
- Department of Cognitive Psychology, Institute of Cognitive Neuroscience, Ruhr University Bochum, 44801 Bochum, Germany
| | - Harald H Quick
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, 45141 Essen, Germany
- High-Field and Hybrid MR Imaging, Essen University Hospital, University of Duisburg-Essen, 45147 Essen, Germany
| | - Melanie D Mark
- Behavioral Neuroscience, Ruhr University Bochum, 44801 Bochum, Germany
| | - Dagmar Timmann
- Department of Neurology and Center for Translational Neuro- and Behavioral Sciences (C-TNBS), Essen University Hospital, University of Duisburg-Essen, 45147 Essen, Germany
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, 45141 Essen, Germany
| |
Collapse
|
15
|
Yang X, Song Y, Zou Y, Li Y, Zeng J. Neural correlates of prediction error in patients with schizophrenia: evidence from an fMRI meta-analysis. Cereb Cortex 2024; 34:bhad471. [PMID: 38061699 DOI: 10.1093/cercor/bhad471] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Revised: 11/07/2023] [Accepted: 11/09/2023] [Indexed: 01/19/2024] Open
Abstract
Abnormal processes of learning from prediction errors, i.e. the discrepancies between expectations and outcomes, are thought to underlie motivational impairments in schizophrenia. Although dopaminergic abnormalities in the mesocorticolimbic reward circuit have been found in patients with schizophrenia, the pathway through which prediction error signals are processed in schizophrenia has yet to be elucidated. To determine the neural correlates of prediction error processing in schizophrenia, we conducted a meta-analysis of whole-brain neuroimaging studies that investigated prediction error signal processing in schizophrenia patients and healthy controls. A total of 14 studies (324 schizophrenia patients and 348 healthy controls) using the reinforcement learning paradigm were included. Our meta-analysis showed that, relative to healthy controls, schizophrenia patients showed increased activity in the precentral gyrus and middle frontal gyrus and reduced activity in the mesolimbic circuit, including the striatum, thalamus, amygdala, hippocampus, anterior cingulate cortex, insula, superior temporal gyrus, and cerebellum, when processing prediction errors. We also found hyperactivity in frontal areas and hypoactivity in mesolimbic areas when encoding prediction error signals in schizophrenia patients, potentially indicating abnormal dopamine signaling of reward prediction error and suggesting failure to represent the value of alternative responses during prediction error learning and decision making.
Collapse
Affiliation(s)
- Xun Yang
- School of Public Policy and Administration, Chongqing University, No. 174, Shazhengjie, Shapingba, Chongqing, China
| | - Yuan Song
- School of Public Policy and Administration, Chongqing University, No. 174, Shazhengjie, Shapingba, Chongqing, China
| | - Yuhan Zou
- School of Economics and Business Administration, Chongqing University, No. 174, Shazhengjie, Shapingba, Chongqing, China
| | - Yilin Li
- Psychology and Neuroscience Department, University of St Andrews, Forbes 1 DRA, Buchanan Garden, St Andrews, Fife, United Kingdom
| | - Jianguang Zeng
- School of Economics and Business Administration, Chongqing University, No. 174, Shazhengjie, Shapingba, Chongqing, China
| |
Collapse
|
16
|
Grill F, Guitart-Masip M, Johansson J, Stiernman L, Axelsson J, Nyberg L, Rieckmann A. Dopamine release in human associative striatum during reversal learning. Nat Commun 2024; 15:59. [PMID: 38167691 PMCID: PMC10762220 DOI: 10.1038/s41467-023-44358-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2022] [Accepted: 12/11/2023] [Indexed: 01/05/2024] Open
Abstract
The dopaminergic system is firmly implicated in reversal learning but human measurements of dopamine release as a correlate of reversal learning success are lacking. Dopamine release and hemodynamic brain activity in response to unexpected changes in action-outcome probabilities are here explored using simultaneous dynamic [11C]Raclopride PET-fMRI and computational modelling of behavior. When participants encounter reversed reward probabilities during a card guessing game, dopamine release is observed in associative striatum. Individual differences in absolute reward prediction error and sensitivity to errors are associated with peak dopamine receptor occupancy. The fMRI response to perseverance errors at the onset of a reversal spatially overlap with the site of dopamine release. Trial-by-trial fMRI correlates of absolute prediction errors show a response in striatum and association cortices, closely overlapping with the location of dopamine release, and separable from a valence signal in ventral striatum. The results converge to implicate striatal dopamine release in associative striatum as a central component of reversal learning, possibly signifying the need for increased cognitive control when new stimuli-responses should be learned.
Collapse
Affiliation(s)
- Filip Grill
- Department of Diagnostics and Intervention, Diagnostic Radiology, Umeå University, Umeå, Sweden.
- Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden.
| | - Marc Guitart-Masip
- Aging Research Center, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden
- Center for Psychiatry Research, Region Stockholm, Stockholm, Sweden
- Center for Cognitive and Computational Neuropsychiatry (CCNP), Karolinska Institutet, Stockholm, Sweden
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
| | - Jarkko Johansson
- Department of Diagnostics and Intervention, Diagnostic Radiology, Umeå University, Umeå, Sweden
- Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden
| | - Lars Stiernman
- Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden
- Department of Medical and Translational Biology, Umeå University, Umeå, Sweden
| | - Jan Axelsson
- Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden
- Department of Diagnostics and Intervention, Radiation Physics, Umeå University, Umeå, Sweden
| | - Lars Nyberg
- Department of Diagnostics and Intervention, Diagnostic Radiology, Umeå University, Umeå, Sweden
- Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden
- Department of Medical and Translational Biology, Umeå University, Umeå, Sweden
| | - Anna Rieckmann
- Department of Diagnostics and Intervention, Diagnostic Radiology, Umeå University, Umeå, Sweden.
- Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden.
- Department of Medical and Translational Biology, Umeå University, Umeå, Sweden.
- Institute for Psychology, University of the Bundeswehr Munich, Neubiberg, Germany.
| |
Collapse
|
17
|
Algermissen J, Swart JC, Scheeringa R, Cools R, den Ouden HEM. Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases. Nat Commun 2024; 15:19. [PMID: 38168089 PMCID: PMC10762147 DOI: 10.1038/s41467-023-44632-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 12/22/2023] [Indexed: 01/05/2024] Open
Abstract
Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.
Collapse
Affiliation(s)
- Johannes Algermissen
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| | - Jennifer C Swart
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - René Scheeringa
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, Essen, Germany
| | - Roshan Cools
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Department of Psychiatry, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Hanneke E M den Ouden
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| |
Collapse
|
18
|
Halahakoon DC, Browning M. Pramipexole for the Treatment of Depression: Efficacy and Mechanisms. Curr Top Behav Neurosci 2024; 66:49-65. [PMID: 37982928 DOI: 10.1007/7854_2023_458] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2023]
Abstract
Dopaminergic mechanisms are a plausible treatment target for patients with clinical depression but are relatively underexplored in conventional antidepressant medications. There is continuing interest in the potential antidepressant effects of the dopamine receptor agonist, pramipexole, with data from both case series and controlled trials indicating that this agent may produce benefit for patients with difficult-to-treat depression. Pramipexole's therapeutic utility in depression is likely to be expressed through alterations in reward mechanisms which are strongly influenced by dopamine pathways and are known to function abnormally in depressed patients. Our work in healthy participants using brain imaging in conjunction with computational modelling suggests that repeated pramipexole facilitates reward learning by inhibiting value decay. This mechanism needs to be confirmed in larger clinical trials in depressed patients. Such studies will also allow assessment of whether baseline performance in reward learning in depression predicts therapeutic response to pramipexole treatment.
Collapse
Affiliation(s)
- Don Chamith Halahakoon
- Department of Psychiatry, Warneford Hospital, Oxford, UK
- Oxford Health NHS Foundation Trust, Oxford, UK
| | - Michael Browning
- Department of Psychiatry, Warneford Hospital, Oxford, UK.
- Oxford Health NHS Foundation Trust, Oxford, UK.
| |
Collapse
|
19
|
Carvalheiro J, Philiastides MG. Distinct spatiotemporal brainstem pathways of outcome valence during reward- and punishment-based learning. Cell Rep 2023; 42:113589. [PMID: 38100353 DOI: 10.1016/j.celrep.2023.113589] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Revised: 10/05/2023] [Accepted: 11/30/2023] [Indexed: 12/17/2023] Open
Abstract
Learning to seek rewards and avoid punishments, based on positive and negative choice outcomes, is essential for human survival. Yet, the neural underpinnings of outcome valence in the human brainstem and the extent to which they differ in reward and punishment learning contexts remain largely elusive. Here, using simultaneously acquired electroencephalography and functional magnetic resonance imaging data, we show that during reward learning the substantia nigra (SN)/ventral tegmental area (VTA) and locus coeruleus are initially activated following negative outcomes, while the VTA subsequently re-engages exhibiting greater responses for positive than negative outcomes, consistent with an early arousal/avoidance response and a later value-updating process, respectively. During punishment learning, we show that distinct raphe nucleus and SN subregions are activated only by negative outcomes with a sustained post-outcome activity across time, supporting the involvement of these brainstem subregions in avoidance behavior. Finally, we demonstrate that the coupling of these brainstem structures with other subcortical and cortical areas helps to shape participants' serial choice behavior in each context.
Collapse
Affiliation(s)
- Joana Carvalheiro
- School of Psychology and Neuroscience, University of Glasgow, Glasgow G12 8QB, UK; Centre for Cognitive Neuroimaging, University of Glasgow, Glasgow G12 8QB, UK.
| | - Marios G Philiastides
- School of Psychology and Neuroscience, University of Glasgow, Glasgow G12 8QB, UK; Centre for Cognitive Neuroimaging, University of Glasgow, Glasgow G12 8QB, UK.
| |
Collapse
|
20
|
Chase HW. A novel technique for delineating the effect of variation in the learning rate on the neural correlates of reward prediction errors in model-based fMRI. Front Psychol 2023; 14:1211528. [PMID: 38187436 PMCID: PMC10768009 DOI: 10.3389/fpsyg.2023.1211528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Accepted: 11/28/2023] [Indexed: 01/09/2024] Open
Abstract
Introduction Computational models play an increasingly important role in describing variation in neural activation in human neuroimaging experiments, including evaluating individual differences in the context of psychiatric neuroimaging. In particular, reinforcement learning (RL) techniques have been widely adopted to examine neural responses to reward prediction errors and stimulus or action values, and how these might vary as a function of clinical status. However, there is a lack of consensus around the importance of the precision of free parameter estimation for these methods, particularly with regard to the learning rate. In the present study, I introduce a novel technique which may be used within a general linear model (GLM) to model the effect of mis-estimation of the learning rate on reward prediction error (RPE)-related neural responses. Methods Simulations employed a simple RL algorithm, which was used to generate hypothetical neural activations that would be expected to be observed in functional magnetic resonance imaging (fMRI) studies of RL. Similar RL models were incorporated within a GLM-based analysis method including derivatives, with individual differences in the resulting GLM-derived beta parameters being evaluated with respect to the free parameters of the RL model or being submitted to other validation analyses. Results Initial simulations demonstrated that the conventional approach to fitting RL models to RPE responses is more likely to reflect individual differences in a reinforcement efficacy construct (lambda) rather than learning rate (alpha). The proposed method, adding a derivative regressor to the GLM, provides a second regressor which reflects the learning rate. Validation analyses were performed including examining another comparable method which yielded highly similar results, and a demonstration of sensitivity of the method in presence of fMRI-like noise. Conclusion Overall, the findings underscore the importance of the lambda parameter for interpreting individual differences in RPE-coupled neural activity, and validate a novel neural metric of the modulation of such activity by individual differences in the learning rate. The method is expected to find application in understanding aberrant reinforcement learning across different psychiatric patient groups including major depression and substance use disorder.
Collapse
Affiliation(s)
- Henry W. Chase
- Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, PA, United States
| |
Collapse
|
21
|
Hoy CW, Quiroga-Martinez DR, Sandoval E, King-Stephens D, Laxer KD, Weber P, Lin JJ, Knight RT. Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex. Nat Commun 2023; 14:8520. [PMID: 38129440 PMCID: PMC10739882 DOI: 10.1038/s41467-023-44248-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2022] [Accepted: 12/05/2023] [Indexed: 12/23/2023] Open
Abstract
The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.
Collapse
Affiliation(s)
- Colin W Hoy
- Department of Neurology, University of California, San Francisco, San Francisco, CA, USA.
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA.
| | - David R Quiroga-Martinez
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
- Center for Music in the Brain, Aarhus University & The Royal Academy of Music, Aarhus, Denmark
| | - Eduardo Sandoval
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
| | - David King-Stephens
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
- Department of Neurology, Yale School of Medicine, New Haven, CT, USA
| | - Kenneth D Laxer
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
| | - Peter Weber
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
| | - Jack J Lin
- Department of Neurology, University of California, Davis, Davis, CA, USA
- Center for Mind and Brain, University of California, Davis, Davis, CA, USA
| | - Robert T Knight
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
- Department of Psychology, University of California, Berkeley, Berkeley, CA, USA
| |
Collapse
|
22
|
Clairis N, Lopez-Persem A. Debates on the dorsomedial prefrontal/dorsal anterior cingulate cortex: insights for future research. Brain 2023; 146:4826-4844. [PMID: 37530487 PMCID: PMC10690029 DOI: 10.1093/brain/awad263] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Revised: 07/19/2023] [Accepted: 07/22/2023] [Indexed: 08/03/2023] Open
Abstract
The dorsomedial prefrontal cortex/dorsal anterior cingulate cortex (dmPFC/dACC) is a brain area subject to many theories and debates over its function(s). Even its precise anatomical borders are subject to much controversy. In the past decades, the dmPFC/dACC has been associated with more than 15 different cognitive processes, which sometimes appear quite unrelated (e.g. body perception, cognitive conflict). As a result, understanding what the dmPFC/dACC does has become a real challenge for many neuroscientists. Several theories of this brain area's function(s) have been developed, leading to successive and competitive publications bearing different models, which sometimes contradict each other. During the last two decades, the lively scientific exchanges around the dmPFC/dACC have promoted fruitful research in cognitive neuroscience. In this review, we provide an overview of the anatomy of the dmPFC/dACC, summarize the state of the art of functions that have been associated with this brain area and present the main theories aiming at explaining the dmPFC/dACC function(s). We explore the commonalities and the arguments between the different theories. Finally, we explain what can be learned from these debates for future investigations of the dmPFC/dACC and other brain regions' functions.
Collapse
Affiliation(s)
- Nicolas Clairis
- Laboratory of Behavioral Genetics (LGC)- Brain Mind Institute (BMI)- Sciences de la Vie (SV), École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland
| | - Alizée Lopez-Persem
- FrontLab, Institut du Cerveau - Paris Brain Institute - ICM, Inserm, CNRS, Sorbonne University, AP HP, Hôpital de la Pitié Salpêtrière, 75013 Paris, France
| |
Collapse
|
23
|
Ulrich M, Rüger A, Durner V, Grön G, Graf H. Reward is not reward: Differential impacts of primary and secondary rewards on expectation, outcome, and prediction error in the human brain's reward processing regions. Neuroimage 2023; 283:120440. [PMID: 37923280 DOI: 10.1016/j.neuroimage.2023.120440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Revised: 10/16/2023] [Accepted: 11/01/2023] [Indexed: 11/07/2023] Open
Abstract
According to their nature, rewarding stimuli are classified as primary (e.g., food, sex) and secondary (e.g., money) rewards. Neuroimaging studies have provided valuable insights in neural reward processing and its various aspects including reward expectation, outcome and prediction error encoding. However, there is only limited evidence of whether the two different types of rewards are processed in common or distinct brain areas, in particular when considering the different functions of reward processing. We analyzed a sample of 42 healthy, male participants using task-based functional magnetic resonance imaging (fMRI) during a variant of the monetary incentive delay task. We aimed to investigate the effects of three different rewarding stimuli-two primary (food and sex) and one secondary (money)-on the various functions of reward processing. To provide a thorough description, we focused on 12 brain regions of interest and utilized the Bayes factor bound (BFB) to express stimulus-related main effects and pairwise differences at different levels of evidence, ranging from weak to decisive. Our results revealed a dominance of sexually charged stimuli in engaging the brain's reward structures for all investigated aspects of reward processing. Nevertheless, the ventral tegmental area, amygdala, ventral caudate, ventromedial prefrontal cortex, subgenual anterior cingulate cortex, and lateral orbitofrontal cortex were activated by both primary and secondary reward outcomes. For other reward processing functions, i.e., reward expectation and the prediction error, effects of the different stimuli were weaker, and effects from one reward type cannot easily be generalized to the other.
Collapse
Affiliation(s)
- Martin Ulrich
- Section Neuropsychology and Functional Neuroimaging, Department of Psychiatry and Psychotherapy III, Ulm University, Leimgrubenweg 12-14, 89075 Ulm, Germany.
| | - Alexander Rüger
- Section Neuropsychology and Functional Neuroimaging, Department of Psychiatry and Psychotherapy III, Ulm University, Leimgrubenweg 12-14, 89075 Ulm, Germany
| | - Verena Durner
- Section Neuropsychology and Functional Neuroimaging, Department of Psychiatry and Psychotherapy III, Ulm University, Leimgrubenweg 12-14, 89075 Ulm, Germany
| | - Georg Grön
- Section Neuropsychology and Functional Neuroimaging, Department of Psychiatry and Psychotherapy III, Ulm University, Leimgrubenweg 12-14, 89075 Ulm, Germany
| | - Heiko Graf
- Section Neuropsychology and Functional Neuroimaging, Department of Psychiatry and Psychotherapy III, Ulm University, Leimgrubenweg 12-14, 89075 Ulm, Germany
| |
Collapse
|
24
|
Mihara M, Izumika R, Tsukiura T. Remembering unexpected beauty: Contributions of the ventral striatum to the processing of reward prediction errors regarding the facial attractiveness in face memory. Neuroimage 2023; 282:120408. [PMID: 37838105 DOI: 10.1016/j.neuroimage.2023.120408] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Revised: 10/05/2023] [Accepted: 10/12/2023] [Indexed: 10/16/2023] Open
Abstract
The COVID-19 pandemic has led people to predict facial attractiveness from partially covered faces. Differences in the predicted and observed facial attractiveness (i.e., masked and unmasked faces, respectively) are defined as reward prediction error (RPE) in a social context. Cognitive neuroscience studies have elucidated the neural mechanisms underlying RPE-induced memory improvements in terms of monetary rewards. However, little is known about the mechanisms underlying RPE-induced memory modulation in terms of social rewards. To elucidate this, the present functional magnetic resonance imaging (fMRI) study investigated activity and functional connectivity during face encoding. In encoding trials, participants rated the predicted attractiveness of faces covered except for around the eyes (prediction phase) and then rated the observed attractiveness of these faces without any cover (outcome phase). The difference in ratings between these phases was defined as RPE in facial attractiveness, and RPE was categorized into positive RPE (increased RPE from the prediction to outcome phases), negative RPE (decreased RPE from the prediction to outcome phases), and non-RPE (no difference in RPE between the prediction and outcome phases). During retrieval, participants were presented with individual faces that had been seen and unseen in the encoding trials, and were required to judge whether or not each face had been seen in the encoding trials. Univariate activity in the ventral striatum (VS) exhibited a linear increase with increased RPE in facial attractiveness. In the multivariate pattern analysis (MVPA), activity patterns in the VS and surrounding areas (extended VS) significantly discriminated between positive/negative RPE and non-RPE. In the functional connectivity analysis, significant functional connectivity between the extended VS and the hippocampus was observed most frequently in positive RPE. Memory improvements by face-based RPE could be involved in functional networks between the extended VS (representing RPE) and the hippocampus, and the interaction could be modulated by RPE values in a social context.
Collapse
Affiliation(s)
- Moe Mihara
- Department of Cognitive, Behavioral and Health Sciences, Graduate School of Human and Environmental Studies, Kyoto University, Yoshida-Nihonmatsu-Cho Sakyo-ku, Kyoto 606-8501, Japan
| | - Reina Izumika
- Department of Cognitive, Behavioral and Health Sciences, Graduate School of Human and Environmental Studies, Kyoto University, Yoshida-Nihonmatsu-Cho Sakyo-ku, Kyoto 606-8501, Japan
| | - Takashi Tsukiura
- Department of Cognitive, Behavioral and Health Sciences, Graduate School of Human and Environmental Studies, Kyoto University, Yoshida-Nihonmatsu-Cho Sakyo-ku, Kyoto 606-8501, Japan.
| |
Collapse
|
25
|
Collomb-Clerc A, Gueguen MCM, Minotti L, Kahane P, Navarro V, Bartolomei F, Carron R, Regis J, Chabardès S, Palminteri S, Bastin J. Human thalamic low-frequency oscillations correlate with expected value and outcomes during reinforcement learning. Nat Commun 2023; 14:6534. [PMID: 37848435 PMCID: PMC10582006 DOI: 10.1038/s41467-023-42380-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Accepted: 10/09/2023] [Indexed: 10/19/2023] Open
Abstract
Reinforcement-based adaptive decision-making is believed to recruit fronto-striatal circuits. A critical node of the fronto-striatal circuit is the thalamus. However, direct evidence of its involvement in human reinforcement learning is lacking. We address this gap by analyzing intra-thalamic electrophysiological recordings from eight participants while they performed a reinforcement learning task. We found that in both the anterior thalamus (ATN) and dorsomedial thalamus (DMTN), low frequency oscillations (LFO, 4-12 Hz) correlated positively with expected value estimated from computational modeling during reward-based learning (after outcome delivery) or punishment-based learning (during the choice process). Furthermore, LFO recorded from ATN/DMTN were also negatively correlated with outcomes so that both components of reward prediction errors were signaled in the human thalamus. The observed differences in the prediction signals between rewarding and punishing conditions shed light on the neural mechanisms underlying action inhibition in punishment avoidance learning. Our results provide insight into the role of thalamus in reinforcement-based decision-making in humans.
Collapse
Affiliation(s)
- Antoine Collomb-Clerc
- Univ. Grenoble Alpes, Inserm, U1216, CHU Grenoble Alpes, Grenoble Institut Neurosciences, 38000, Grenoble, France
| | - Maëlle C M Gueguen
- Univ. Grenoble Alpes, Inserm, U1216, CHU Grenoble Alpes, Grenoble Institut Neurosciences, 38000, Grenoble, France
- Department of Psychiatry, Brain Health Institute and University Behavioral Health Care, Rutgers University-New Brunswick, Piscataway, NJ, USA
| | - Lorella Minotti
- Univ. Grenoble Alpes, Inserm, U1216, CHU Grenoble Alpes, Grenoble Institut Neurosciences, 38000, Grenoble, France
- Neurology Department, University Hospital of Grenoble, Grenoble, France
| | - Philippe Kahane
- Univ. Grenoble Alpes, Inserm, U1216, CHU Grenoble Alpes, Grenoble Institut Neurosciences, 38000, Grenoble, France
- Neurology Department, University Hospital of Grenoble, Grenoble, France
| | - Vincent Navarro
- Sorbonne Université, Paris Brain Institute - Institut du Cerveau, ICM, INSERM, CNRS, AP-HP, Pitié-Salpêtrière Hospital, Paris, France
| | - Fabrice Bartolomei
- Timone University Hospital, Sleep Unit, Epileptology and Cerebral Rhythmology, University Hospital of Marseille, Marseille, France
- Aix Marseille University, Inserm, Institut de Neurosciences des Systèmes, Marseille, France
| | - Romain Carron
- Aix Marseille University, Inserm, Institut de Neurosciences des Systèmes, Marseille, France
- Timone University Hospital, Department of functional and stereotactic neurosurgery, University Hospital of Marseille, Marseille, France
| | - Jean Regis
- Neurosurgery Department, University Hospital of Marseille, Marseille, France
| | - Stephan Chabardès
- Univ. Grenoble Alpes, Inserm, U1216, CHU Grenoble Alpes, Grenoble Institut Neurosciences, 38000, Grenoble, France
- Neurosurgery Department, University Hospital of Grenoble, Grenoble, France
| | - Stefano Palminteri
- Laboratoire de Neurosciences Cognitives Computationnelles, Département d'Etudes Cognitives, ENS, PSL, INSERM, Paris, France
| | - Julien Bastin
- Univ. Grenoble Alpes, Inserm, U1216, CHU Grenoble Alpes, Grenoble Institut Neurosciences, 38000, Grenoble, France.
| |
Collapse
|
26
|
Gadassi Polack R, Mollick JA, Keren H, Joormann J, Watts R. Neural responses to reward valence and magnitude from pre- to early adolescence. Neuroimage 2023; 275:120166. [PMID: 37178821 PMCID: PMC10311119 DOI: 10.1016/j.neuroimage.2023.120166] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 04/04/2023] [Accepted: 05/10/2023] [Indexed: 05/15/2023] Open
Abstract
BACKGROUND Neural activation during reward processing is thought to underlie critical behavioral changes that take place during the transition to adolescence (e.g., learning, risk-taking). Though literature on the neural basis of reward processing in adolescence is booming, important gaps remain. First, more information is needed regarding changes in functional neuroanatomy in early adolescence. Another gap is understanding whether sensitivity to different aspects of the incentive (e.g., magnitude and valence) changes during the transition into adolescence. We used fMRI from a large sample of preadolescent children to characterize neural responses to incentive valence vs. magnitude during anticipation and feedback, and their change over a period of two years. METHODS Data were taken from the Adolescent Cognitive and Brain DevelopmentSM (ABCD®) study release 3.0. Children completed the Monetary Incentive Delay task at baseline (ages 9-10) and year 2 follow-up (ages 11-12). Based on data from two sites (N = 491), we identified activation-based Regions of Interest (ROIs; e.g., striatum, prefrontal regions, etc.) that were sensitive to trial type (win $5, win $0.20, neutral, lose $0.20, lose $5) during anticipation and feedback phases. Then, in an independent subsample (N = 1470), we examined whether these ROIs were sensitive to valence and magnitude and whether that sensitivity changed over two years. RESULTS Our results show that most ROIs involved in reward processing (including the striatum, prefrontal cortex, and insula) are specialized, i.e., mainly sensitive to either incentive valence or magnitude, and this sensitivity was consistent over a 2-year period. The effect sizes of time and its interactions were significantly smaller (0.002≤η2≤0.02) than the effect size of trial type (0.06≤η2≤0.30). Interestingly, specialization was moderated by reward processing phase but was stable across development. Biological sex and pubertal status differences were few and inconsistent. Developmental changes were mostly evident during success feedback, where neural reactivity increased over time. CONCLUSIONS Our results suggest sub-specialization to valence vs. magnitude within many ROIs of the reward circuitry. Additionally, in line with theoretical models of adolescent development, our results suggest that the ability to benefit from success increases from pre- to early adolescence. These findings can inform educators and clinicians and facilitate empirical research of typical and atypical motivational behaviors during a critical time of development.
Collapse
Affiliation(s)
- Reuma Gadassi Polack
- Psychology Department, Yale University, United States; Psychiatry Department, Yale University, United States; School of Behavioral Sciences, Tel Aviv-Yaffo Academic College, Israel.
| | | | - Hanna Keren
- Faculty of Medicine, Bar-Ilan University, Israel
| | | | - Richard Watts
- Psychology Department, Yale University, United States
| |
Collapse
|
27
|
Dugré JR, Potvin S. Altered functional connectivity of the amygdala across variants of callous-unemotional traits: A resting-state fMRI study in children and adolescents. J Psychiatr Res 2023; 163:32-42. [PMID: 37201236 DOI: 10.1016/j.jpsychires.2023.05.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/24/2022] [Revised: 02/28/2023] [Accepted: 05/01/2023] [Indexed: 05/20/2023]
Abstract
Over the past years, research has shown that primary (high callousness and low anxiety) and secondary (high callousness and anxiety) variants of CU traits may be associated with opposite amygdala activity (hypo- and hyper-reactivity, respectively). However, their differences in amygdala functional connectivity remains largely unexplored. We conducted a Latent Profile Analysis on a large sample of adolescents (n = 1416) to identify homogeneous subgroups with different levels of callousness and anxiety. We then performed a seed-to-voxel connectivity analysis on resting-state fMRI data to compare subgroups on connectivity patterns of the amygdala. We examined the results in relation to conduct problems to identify potential neural risk factors. The Latent Profile Analysis revealed four subgroups, including the primary and secondary variants, anxious, and typically developing adolescents. The seed-to-voxel analyses showed that the primary variant was mainly characterized by increased connectivity between the left amygdala and left thalamus. The secondary variant exhibited deficient connectivity between the amygdala and the dorsomedial prefrontal cortex, temporo-parietal junction, premotor, and postcentral gyrus. Both variants showed increased connectivity between the left amygdala and the right thalamus but exhibited opposite functional connectivity between the left amygdala and the parahippocampal gyrus. Dimensional analyses indicated that conduct problems may play a mediating role between callousness and amygdala-dmPFC functional connectivity across youths with already high levels of callousness. Our study highlights that both variants differ in the functional connectivity of the amygdala. Our results support the importance of disentangling the heterogeneity of adolescents at risk for conduct problems in neuroimaging.
Collapse
Affiliation(s)
- Jules R Dugré
- Research Center of the Institut Universitaire en Santé Mentale de Montréal, Montreal, Canada; Department of Psychiatry and Addictology, Faculty of Medicine, University of Montreal, Montreal, Canada.
| | - Stéphane Potvin
- Research Center of the Institut Universitaire en Santé Mentale de Montréal, Montreal, Canada; Department of Psychiatry and Addictology, Faculty of Medicine, University of Montreal, Montreal, Canada.
| |
Collapse
|
28
|
Tsai CG, Fu YF, Li CW. Prediction errors arising from switches between major and minor modes in music: An fMRI study. Brain Cogn 2023; 169:105987. [PMID: 37126951 DOI: 10.1016/j.bandc.2023.105987] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2023] [Revised: 04/15/2023] [Accepted: 04/17/2023] [Indexed: 05/03/2023]
Abstract
The major and minor modes in Western music have positive and negative connotations, respectively. The present fMRI study examined listeners' neural responses to switches between major and minor modes. We manipulated the final chords of J. S. Bach's keyboard pieces so that each major-mode passage ended with either the major (Major-Major) or minor (Major-Minor) tonic chord, and each minor-mode passage ended with either the minor (Minor-Minor) or major (Minor-Major) tonic chord. If the final major and minor chords have positive and negative reward values respectively, the Major-Minor and Minor-Major stimuli would cause negative and positive reward prediction errors (RPEs) respectively in a listener's brain. We found that activity in a frontoparietal network was significantly higher for Major-Minor than for Major-Major. Based on previous research, these results support the idea that a major-to-minor switch causes negative RPE. The contrast of Minor-Major minus Minor-Minor yielded activation in the ventral insula and visual cortex, speaking against the idea that a minor-to-major switch causes positive RPE. We discuss our results in relation to executive functions and the emotional connotations of major versus minor modes.
Collapse
Affiliation(s)
- Chen-Gia Tsai
- Graduate Institute of Musicology, National Taiwan University, Taipei, Taiwan; Neurobiology and Cognitive Science Center, National Taiwan University, Taipei, Taiwan
| | - Yi-Fan Fu
- Department of Bio-Industry Communication and Development, National Taiwan University, Taipei, Taiwan
| | - Chia-Wei Li
- Department of Radiology, Wan Fang Hospital, Taipei Medical University, Taipei, Taiwan.
| |
Collapse
|
29
|
Chen X, Affourtit J, Ryskin R, Regev TI, Norman-Haignere S, Jouravlev O, Malik-Moraleda S, Kean H, Varley R, Fedorenko E. The human language system, including its inferior frontal component in "Broca's area," does not support music perception. Cereb Cortex 2023; 33:7904-7929. [PMID: 37005063 PMCID: PMC10505454 DOI: 10.1093/cercor/bhad087] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2022] [Revised: 01/02/2023] [Accepted: 01/03/2023] [Indexed: 04/04/2023] Open
Abstract
Language and music are two human-unique capacities whose relationship remains debated. Some have argued for overlap in processing mechanisms, especially for structure processing. Such claims often concern the inferior frontal component of the language system located within "Broca's area." However, others have failed to find overlap. Using a robust individual-subject fMRI approach, we examined the responses of language brain regions to music stimuli, and probed the musical abilities of individuals with severe aphasia. Across 4 experiments, we obtained a clear answer: music perception does not engage the language system, and judgments about music structure are possible even in the presence of severe damage to the language network. In particular, the language regions' responses to music are generally low, often below the fixation baseline, and never exceed responses elicited by nonmusic auditory conditions, like animal sounds. Furthermore, the language regions are not sensitive to music structure: they show low responses to both intact and structure-scrambled music, and to melodies with vs. without structural violations. Finally, in line with past patient investigations, individuals with aphasia, who cannot judge sentence grammaticality, perform well on melody well-formedness judgments. Thus, the mechanisms that process structure in language do not appear to process music, including music syntax.
Collapse
Affiliation(s)
- Xuanyi Chen
- Department of Cognitive Sciences, Rice University, TX 77005, United States
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, United States
- McGovern Institute for Brain Research, MIT, Cambridge, MA 02139, United States
| | - Josef Affourtit
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, United States
- McGovern Institute for Brain Research, MIT, Cambridge, MA 02139, United States
| | - Rachel Ryskin
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, United States
- McGovern Institute for Brain Research, MIT, Cambridge, MA 02139, United States
- Department of Cognitive & Information Sciences, University of California, Merced, Merced, CA 95343, United States
| | - Tamar I Regev
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, United States
- McGovern Institute for Brain Research, MIT, Cambridge, MA 02139, United States
| | - Samuel Norman-Haignere
- Department of Biostatistics & Computational Biology, University of Rochester Medical Center, Rochester, NY, United States
- Department of Neuroscience, University of Rochester Medical Center, Rochester, NY, United States
- Department of Biomedical Engineering, University of Rochester, Rochester, NY, United States
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY, United States
| | - Olessia Jouravlev
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, United States
- McGovern Institute for Brain Research, MIT, Cambridge, MA 02139, United States
- Department of Cognitive Science, Carleton University, Ottawa, ON, Canada
| | - Saima Malik-Moraleda
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, United States
- McGovern Institute for Brain Research, MIT, Cambridge, MA 02139, United States
- The Program in Speech and Hearing Bioscience and Technology, Harvard University, Cambridge, MA 02138, United States
| | - Hope Kean
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, United States
- McGovern Institute for Brain Research, MIT, Cambridge, MA 02139, United States
| | - Rosemary Varley
- Psychology & Language Sciences, UCL, London, WCN1 1PF, United Kingdom
| | - Evelina Fedorenko
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, United States
- McGovern Institute for Brain Research, MIT, Cambridge, MA 02139, United States
- The Program in Speech and Hearing Bioscience and Technology, Harvard University, Cambridge, MA 02138, United States
| |
Collapse
|
30
|
Zeng J, You L, Yang F, Luo Y, Yu S, Yan J, Liu M, Yang X. A meta-analysis of the neural substrates of monetary reward anticipation and outcome in alcohol use disorder. Hum Brain Mapp 2023; 44:2841-2861. [PMID: 36852619 PMCID: PMC10089105 DOI: 10.1002/hbm.26249] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Revised: 01/23/2023] [Accepted: 02/13/2023] [Indexed: 03/01/2023] Open
Abstract
The capacity to anticipate and detect rewarding outcomes is fundamental for the development of adaptive decision-making and goal-oriented behavior. Delineating the neural correlates of different stages of reward processing is imperative for understanding the neurobiological mechanism underlying alcohol use disorder (AUD). To examine the neural correlates of monetary anticipation and outcome in AUD patients, we performed two separate voxel-wise meta-analyses of functional neuroimaging studies, including 12 studies investigating reward anticipation and 7 studies investigating reward outcome using the monetary incentive delay task. During the anticipation stage, AUD patients displayed decreased activation in response to monetary cues in mesocortical-limbic circuits and sensory areas, including the ventral striatum (VS), insula, hippocampus, inferior occipital gyrus, supramarginal gyrus, lingual gyrus and fusiform gyrus. During the outcome stage, AUD patients exhibited reduced activation in the dorsal striatum, VS and insula, and increased activation in the orbital frontal cortex and medial temporal area. Our findings suggest that different activation patterns are associated with nondrug rewards during different reward processing stages, potentially reflecting a changed sensitivity to monetary reward in AUD.
Collapse
Affiliation(s)
- Jianguang Zeng
- School of Economics and Business AdministrationChongqing UniversityChongqingChina
| | - Lantao You
- School of Economics and Business AdministrationChongqing UniversityChongqingChina
| | - Fan Yang
- Department of Ultrasonography, West China Second University HospitalSichuan UniversityChengduChina
- Chengdu Chenghua District Maternal and Child Health HospitalSichuan UniversityChengduChina
| | - Ya Luo
- Department of Psychiatry, State Key Lab of BiotherapyWest China Hospital of Sichuan UniversityChengduChina
| | - Shuxian Yu
- School of Economics and Business AdministrationChongqing UniversityChongqingChina
| | - Jiangnan Yan
- School of Economics and Business AdministrationChongqing UniversityChongqingChina
| | - Mengqi Liu
- Department of RadiologyThe First Affiliated Hospital of Chongqing Medical UniversityChongqingChina
| | - Xun Yang
- School of Public AffairsChongqing UniversityChongqingChina
| |
Collapse
|
31
|
Jansen M, Lockwood PL, Cutler J, de Bruijn ERA. l-DOPA and oxytocin influence the neurocomputational mechanisms of self-benefitting and prosocial reinforcement learning. Neuroimage 2023; 270:119983. [PMID: 36848972 DOI: 10.1016/j.neuroimage.2023.119983] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 02/03/2023] [Accepted: 02/23/2023] [Indexed: 02/27/2023] Open
Abstract
Humans learn through reinforcement, particularly when outcomes are unexpected. Recent research suggests similar mechanisms drive how we learn to benefit other people, that is, how we learn to be prosocial. Yet the neurochemical mechanisms underlying such prosocial computations remain poorly understood. Here, we investigated whether pharmacological manipulation of oxytocin and dopamine influence the neurocomputational mechanisms underlying self-benefitting and prosocial reinforcement learning. Using a double-blind placebo-controlled cross-over design, we administered intranasal oxytocin (24 IU), dopamine precursor l-DOPA (100 mg + 25 mg carbidopa), or placebo over three sessions. Participants performed a probabilistic reinforcement learning task with potential rewards for themselves, another participant, or no one, during functional magnetic resonance imaging. Computational models of reinforcement learning were used to calculate prediction errors (PEs) and learning rates. Participants behavior was best explained by a model with different learning rates for each recipient, but these were unaffected by either drug. On the neural level, however, both drugs blunted PE signaling in the ventral striatum and led to negative signaling of PEs in the anterior mid-cingulate cortex, dorsolateral prefrontal cortex, inferior parietal gyrus, and precentral gyrus, compared to placebo, and regardless of recipient. Oxytocin (versus placebo) administration was additionally associated with opposing tracking of self-benefitting versus prosocial PEs in dorsal anterior cingulate cortex, insula and superior temporal gyrus. These findings suggest that both l-DOPA and oxytocin induce a context-independent shift from positive towards negative tracking of PEs during learning. Moreover, oxytocin may have opposing effects on PE signaling when learning to benefit oneself versus another.
Collapse
Affiliation(s)
- Myrthe Jansen
- Department of Clinical Psychology, Leiden University, the Netherlands; Leiden Institute for Brain and Cognition (LIBC), Leiden, the Netherlands.
| | - Patricia L Lockwood
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK; Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK; Centre for Developmental Science, School of Psychology, University of Birmingham, UK
| | - Jo Cutler
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK; Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK; Centre for Developmental Science, School of Psychology, University of Birmingham, UK
| | - Ellen R A de Bruijn
- Department of Clinical Psychology, Leiden University, the Netherlands; Leiden Institute for Brain and Cognition (LIBC), Leiden, the Netherlands
| |
Collapse
|
32
|
Dugré JR, Potvin S. Neural bases of frustration-aggression theory: A multi-domain meta-analysis of functional neuroimaging studies. J Affect Disord 2023; 331:64-76. [PMID: 36924847 DOI: 10.1016/j.jad.2023.03.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Revised: 02/01/2023] [Accepted: 03/06/2023] [Indexed: 03/18/2023]
Abstract
BACKGROUND Early evidence suggests that unexpected non-reward may increase the risk for aggressive behaviors. Despite the growing interest in understanding brain functions that may be implicated in aggressive behaviors, the neural processes underlying such frustrative events remain largely unknown. Furthermore, meta-analytic results have produced discrepant results, potentially due to substantial differences in the definition of anger/aggression constructs. METHODS Therefore, we conducted a coordinate-based meta-analysis, using the activation likelihood estimation algorithm, on neuroimaging studies examining reward omission and retaliatory behaviors in healthy subjects. Conjunction analyses were further examined to discover overlapping brain activations across these meta-analytic maps. RESULTS Frustrative non-reward deactivated the orbitofrontal cortex, ventral striatum and posterior cingulate cortex, whereas increased activations were observed in midcingulo-insular regions. Retaliatory behaviors recruited the left fronto-insular and anterior midcingulate cortices, the dorsal caudate and the primary somatosensory cortex. Conjunction analyses revealed that both strongly activated midcingulo-insular regions. LIMITATIONS Spatial overlap between neural correlates of frustration and retaliatory behaviors was conducted using a conjunction analysis. Therefore, neurobiological markers underlying the temporal sequence of the frustration-aggression theory should be interpreted with caution. CONCLUSIONS Nonetheless, our results underscore the role of anterior midcingulate/pre-supplementary motor area and fronto-insular cortex in both frustration and retaliatory behaviors. A neurobiological framework for understanding frustration-based impulsive aggression is provided.
Collapse
Affiliation(s)
- Jules R Dugré
- Centre de recherche de l'Institut Universitaire en Santé Mentale de Montréal, Montréal, Canada; Department of Psychiatry and Addiction, Faculty of Medicine, University of Montreal, Montréal, Canada.
| | - Stéphane Potvin
- Centre de recherche de l'Institut Universitaire en Santé Mentale de Montréal, Montréal, Canada; Department of Psychiatry and Addiction, Faculty of Medicine, University of Montreal, Montréal, Canada.
| |
Collapse
|
33
|
Fornari L, Ioumpa K, Nostro AD, Evans NJ, De Angelis L, Speer SPH, Paracampo R, Gallo S, Spezio M, Keysers C, Gazzola V. Neuro-computational mechanisms and individual biases in action-outcome learning under moral conflict. Nat Commun 2023; 14:1218. [PMID: 36878911 PMCID: PMC9988878 DOI: 10.1038/s41467-023-36807-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2021] [Accepted: 02/17/2023] [Indexed: 03/08/2023] Open
Abstract
Learning to predict action outcomes in morally conflicting situations is essential for social decision-making but poorly understood. Here we tested which forms of Reinforcement Learning Theory capture how participants learn to choose between self-money and other-shocks, and how they adapt to changes in contingencies. We find choices were better described by a reinforcement learning model based on the current value of separately expected outcomes than by one based on the combined historical values of past outcomes. Participants track expected values of self-money and other-shocks separately, with the substantial individual difference in preference reflected in a valuation parameter balancing their relative weight. This valuation parameter also predicted choices in an independent costly helping task. The expectations of self-money and other-shocks were biased toward the favored outcome but fMRI revealed this bias to be reflected in the ventromedial prefrontal cortex while the pain-observation network represented pain prediction errors independently of individual preferences.
Collapse
Affiliation(s)
- Laura Fornari
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Kalliopi Ioumpa
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Alessandra D Nostro
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Nathan J Evans
- School of Psychology, University of Queensland, Brisbane, QLD, Australia
| | - Lorenzo De Angelis
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Sebastian P H Speer
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Riccardo Paracampo
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Selene Gallo
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Michael Spezio
- Psychology, Neuroscience, & Data Science, Scripps College, 1030 Columbia Ave, CA 91711, Claremont, CA, USA
| | - Christian Keysers
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands.,Department of Psychology, University of Amsterdam, Nieuwe Achtergracht 129-B, 1018 WT, Amsterdam, The Netherlands
| | - Valeria Gazzola
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands. .,Department of Psychology, University of Amsterdam, Nieuwe Achtergracht 129-B, 1018 WT, Amsterdam, The Netherlands.
| |
Collapse
|
34
|
Wieland L, Ebrahimi C, Katthagen T, Panitz M, Luettgau L, Heinz A, Schlagenhauf F, Sjoerds Z. Acute stress alters probabilistic reversal learning in healthy male adults. Eur J Neurosci 2023; 57:824-839. [PMID: 36656136 DOI: 10.1111/ejn.15916] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Revised: 01/12/2023] [Accepted: 01/14/2023] [Indexed: 01/20/2023]
Abstract
Behavioural adaptation is a fundamental cognitive ability, ensuring survival by allowing for flexible adjustment to changing environments. In laboratory settings, behavioural adaptation can be measured with reversal learning paradigms requiring agents to adjust reward learning to stimulus-action-outcome contingency changes. Stress is found to alter flexibility of reward learning, but effect directionality is mixed across studies. Here, we used model-based functional MRI (fMRI) in a within-subjects design to investigate the effect of acute psychosocial stress on flexible behavioural adaptation. Healthy male volunteers (n = 28) did a reversal learning task during fMRI in two sessions, once after the Trier Social Stress Test (TSST), a validated psychosocial stress induction method, and once after a control condition. Stress effects on choice behaviour were investigated using multilevel generalized linear models and computational models describing different learning processes that potentially generated the data. Computational models were fitted using a hierarchical Bayesian approach, and model-derived reward prediction errors (RPE) were used as fMRI regressors. We found that acute psychosocial stress slightly increased correct response rates. Model comparison revealed that double-update learning with altered choice temperature under stress best explained the observed behaviour. In the brain, model-derived RPEs were correlated with BOLD signals in striatum and ventromedial prefrontal cortex (vmPFC). Striatal RPE signals for win trials were stronger during stress compared with the control condition. Our study suggests that acute psychosocial stress could enhance reversal learning and RPE brain responses in healthy male participants and provides a starting point to explore these effects further in a more diverse population.
Collapse
Affiliation(s)
- Lara Wieland
- Department of Psychiatry and Neurosciences, CCM, Charité-Universitätsmedizin Berlin, Berlin, Germany.,Einstein Center for Neurosciences Berlin, Charité-Universitätsmedizin Berlin, Berlin, Germany.,Bernstein Center for Computational Neuroscience, Berlin, Germany
| | - Claudia Ebrahimi
- Department of Psychiatry and Neurosciences, CCM, Charité-Universitätsmedizin Berlin, Berlin, Germany
| | - Teresa Katthagen
- Department of Psychiatry and Neurosciences, CCM, Charité-Universitätsmedizin Berlin, Berlin, Germany
| | - Martin Panitz
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Lennart Luettgau
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.,Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London, UK
| | - Andreas Heinz
- Department of Psychiatry and Neurosciences, CCM, Charité-Universitätsmedizin Berlin, Berlin, Germany
| | - Florian Schlagenhauf
- Department of Psychiatry and Neurosciences, CCM, Charité-Universitätsmedizin Berlin, Berlin, Germany.,Einstein Center for Neurosciences Berlin, Charité-Universitätsmedizin Berlin, Berlin, Germany.,Bernstein Center for Computational Neuroscience, Berlin, Germany.,Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Zsuzsika Sjoerds
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.,Cognitive Psychology Unit, Institute of Psychology & Leiden Institute for Brain and Cognition, Leiden University, Leiden, the Netherlands
| |
Collapse
|
35
|
Blank H, Alink A, Büchel C. Multivariate functional neuroimaging analyses reveal that strength-dependent face expectations are represented in higher-level face-identity areas. Commun Biol 2023; 6:135. [PMID: 36725984 PMCID: PMC9892564 DOI: 10.1038/s42003-023-04508-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Accepted: 01/19/2023] [Indexed: 02/03/2023] Open
Abstract
Perception is an active inference in which prior expectations are combined with sensory input. It is still unclear how the strength of prior expectations is represented in the human brain. The strength, or precision, of a prior could be represented with its content, potentially in higher-level sensory areas. We used multivariate analyses of functional resonance imaging data to test whether expectation strength is represented together with the expected face in high-level face-sensitive regions. Participants were trained to associate images of scenes with subsequently presented images of different faces. Each scene predicted three faces, each with either low, intermediate, or high probability. We found that anticipation enhances the similarity of response patterns in the face-sensitive anterior temporal lobe to response patterns specifically associated with the image of the expected face. In contrast, during face presentation, activity increased for unexpected faces in a typical prediction error network, containing areas such as the caudate and the insula. Our findings show that strength-dependent face expectations are represented in higher-level face-identity areas, supporting hierarchical theories of predictive processing according to which higher-level sensory regions represent weighted priors.
Collapse
Affiliation(s)
- Helen Blank
- grid.13648.380000 0001 2180 3484Department of Systems Neuroscience, University Medical Center Hamburg-Eppendorf, 20246 Hamburg, Germany
| | - Arjen Alink
- grid.13648.380000 0001 2180 3484Department of Systems Neuroscience, University Medical Center Hamburg-Eppendorf, 20246 Hamburg, Germany
| | - Christian Büchel
- grid.13648.380000 0001 2180 3484Department of Systems Neuroscience, University Medical Center Hamburg-Eppendorf, 20246 Hamburg, Germany
| |
Collapse
|
36
|
Experiential values are underweighted in decisions involving symbolic options. Nat Hum Behav 2023; 7:611-626. [PMID: 36604497 DOI: 10.1038/s41562-022-01496-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2022] [Accepted: 11/04/2022] [Indexed: 01/07/2023]
Abstract
Standard models of decision-making assume each option is associated with subjective value, regardless of whether this value is inferred from experience (experiential) or explicitly instructed probabilistic outcomes (symbolic). In this study, we present results that challenge the assumption of unified representation of experiential and symbolic value. Across nine experiments, we presented participants with hybrid decisions between experiential and symbolic options. Participants' choices exhibited a pattern consistent with a systematic neglect of the experiential values. This normatively irrational decision strategy held after accounting for alternative explanations, and persisted even when it bore an economic cost. Overall, our results demonstrate that experiential and symbolic values are not symmetrically considered in hybrid decisions, suggesting they recruit different representational systems that may be assigned different priority levels in the decision process. These findings challenge the dominant models commonly used in value-based decision-making research.
Collapse
|
37
|
Moughrabi N, Botsford C, Gruichich TS, Azar A, Heilicher M, Hiser J, Crombie KM, Dunsmoor JE, Stowe Z, Cisler JM. Large-scale neural network computations and multivariate representations during approach-avoidance conflict decision-making. Neuroimage 2022; 264:119709. [PMID: 36283543 PMCID: PMC9835092 DOI: 10.1016/j.neuroimage.2022.119709] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 10/20/2022] [Accepted: 10/21/2022] [Indexed: 11/09/2022] Open
Abstract
Many real-world situations require navigating decisions for both reward and threat. While there has been significant progress in understanding mechanisms of decision-making and mediating neurocircuitry separately for reward and threat, there is limited understanding of situations where reward and threat contingencies compete to create approach-avoidance conflict (AAC). Here, we leverage computational learning models, independent component analysis (ICA), and multivariate pattern analysis (MVPA) approaches to understand decision-making during a novel task that embeds concurrent reward and threat learning and manipulates congruency between reward and threat probabilities. Computational modeling supported a modified reinforcement learning model where participants integrated reward and threat value into a combined total value according to an individually varying policy parameter, which was highly predictive of decisions to approach reward vs avoid threat during trials where the highest reward option was also the highest threat option (i.e., approach-avoidance conflict). ICA analyses demonstrated unique roles for salience, frontoparietal, medial prefrontal, and inferior frontal networks in differential encoding of reward vs threat prediction error and value signals. The left frontoparietal network uniquely encoded degree of conflict between reward and threat value at the time of choice. MVPA demonstrated that delivery of reward and threat could accurately be decoded within salience and inferior frontal networks, respectively, and that decisions to approach reward vs avoid threat were predicted by the relative degree to which these reward vs threat representations were active at the time of choice. This latter result suggests that navigating AAC decisions involves generating mental representations for possible decision outcomes, and relative activation of these representations may bias subsequent decision-making towards approaching reward or avoiding threat accordingly.
Collapse
Affiliation(s)
- Nicole Moughrabi
- Department of Psychiatry and Behavioral Sciences, University of Texas at Austin
| | - Chloe Botsford
- Department of Psychiatry, University of Wisconsin-Madison
| | | | - Ameera Azar
- Department of Psychiatry and Behavioral Sciences, University of Texas at Austin
| | | | - Jaryd Hiser
- Department of Psychiatry, University of Wisconsin-Madison
| | - Kevin M Crombie
- Department of Psychiatry and Behavioral Sciences, University of Texas at Austin
| | - Joseph E Dunsmoor
- Department of Psychiatry and Behavioral Sciences, University of Texas at Austin; Institute for Early Life Adversity Research, University of Texas at Austin
| | - Zach Stowe
- Department of Psychiatry, University of Wisconsin-Madison
| | - Josh M Cisler
- Department of Psychiatry and Behavioral Sciences, University of Texas at Austin; Institute for Early Life Adversity Research, University of Texas at Austin.
| |
Collapse
|
38
|
Pisauro MA, Fouragnan EF, Arabadzhiyska DH, Apps MAJ, Philiastides MG. Neural implementation of computational mechanisms underlying the continuous trade-off between cooperation and competition. Nat Commun 2022; 13:6873. [PMID: 36369180 PMCID: PMC9652314 DOI: 10.1038/s41467-022-34509-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2021] [Accepted: 10/27/2022] [Indexed: 11/13/2022] Open
Abstract
Social interactions evolve continuously. Sometimes we cooperate, sometimes we compete, while at other times we strategically position ourselves somewhere in between to account for the ever-changing social contexts around us. Research on social interactions often focuses on a binary dichotomy between competition and cooperation, ignoring people's evolving shifts along a continuum. Here, we develop an economic game - the Space Dilemma - where two players change their degree of cooperativeness over time in cooperative and competitive contexts. Using computational modelling we show how social contexts bias choices and characterise how inferences about others' intentions modulate cooperativeness. Consistent with the modelling predictions, brain regions previously linked to social cognition, including the temporo-parietal junction, dorso-medial prefrontal cortex and the anterior cingulate gyrus, encode social prediction errors and context-dependent signals, correlating with shifts along a cooperation-competition continuum. These results provide a comprehensive account of the computational and neural mechanisms underlying the continuous trade-off between cooperation and competition.
Collapse
Affiliation(s)
- M A Pisauro
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK.
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK.
| | - E F Fouragnan
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
- Brain Research Imaging Center and School of Psychology, Faculty of Health, University of Plymouth, Plymouth, UK
| | - D H Arabadzhiyska
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
| | - M A J Apps
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
| | - M G Philiastides
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
| |
Collapse
|
39
|
Lee M, Lori A, Langford NA, Rilling JK. Enhanced endogenous oxytocin signaling in the brain modulates neural responses to social misalignment and promotes conformity in humans: A multi-locus genetic profile approach. Psychoneuroendocrinology 2022; 144:105869. [PMID: 35868206 PMCID: PMC9553010 DOI: 10.1016/j.psyneuen.2022.105869] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Revised: 07/08/2022] [Accepted: 07/11/2022] [Indexed: 11/26/2022]
Abstract
The neuropeptide oxytocin (OT) is known to promote social conformity. However, the specific neurocognitive mechanisms underlying OT-induced conformity remain unclear. We aimed to address this gap by examining how genetic variation in the oxytocin receptor gene (OXTR) is linked with behavioral conformity and its underlying neural systems. Specifically, we utilized the genotype-tissue expression database (GTEx) to create a novel multi-locus genetic profile score (MPS) that reflects the level of OXTR expression in the human brain. A total of 194 participants (Neuroimaging N = 50, Behavioral N = 144) performed a novel conformity task in which they viewed a series of word pairs depicting various moral values and virtues widely recognized in the United States. In each trial, participants indicated the relative importance of these words and subsequently learned about the majority opinion. Participants later rated the same word pairs a second time. Changes in participants' ratings between the first and second sessions were measured and analyzed with respect to social feedback, blood oxygen level-dependent (BOLD) functional magnetic resonance imaging (fMRI) signals, and OXTR MPS. We found that participants adjusted their ratings in accordance with the majority opinions. Social misalignment between self and others activated brain areas such as the striatum and the posterior medial frontal cortex (pMFC). However, unlike most findings from previous studies, activation in the pMFC during the inconsistent social feedback negatively, rather than positively, predicted behavioral conformity. Notably, those with higher OXTR MPS had reduced pMFC activation in the face of social misalignment, which led to greater conformity. Our findings suggest that OT may promote conformity by dampening the conflict-related signals in the pMFC. They also show that OXTR MPS may be useful for studying the effect of genes on highly complex human social traits, such as conformity.
Collapse
Affiliation(s)
- Minwoo Lee
- Department of Anthropology, Emory University, USA
| | - Adriana Lori
- Department of Psychiatry and Behavioral Science, Emory University, USA
| | - Nicole A. Langford
- Department of Psychiatry and Behavioral Science, Emory University, USA,Nell Hodgson Woodruff School of Nursing, Emory University, USA
| | - James K. Rilling
- Department of Anthropology, Emory University, USA,Department of Psychiatry and Behavioral Science, Emory University, USA,Center for Behavioral Neuroscience, Emory University, USA,Emory National Primate Research Center, Emory University, USA,Center for Translational Social Neuroscience, Emory University, USA,Corresponding author at: Department of Anthropology, Emory University, USA. (J.K. Rilling)
| |
Collapse
|
40
|
Cecchi R, Vinckier F, Hammer J, Marusic P, Nica A, Rheims S, Trebuchon A, Barbeau EJ, Denuelle M, Maillard L, Minotti L, Kahane P, Pessiglione M, Bastin J. Intracerebral mechanisms explaining the impact of incidental feedback on mood state and risky choice. eLife 2022; 11:72440. [PMID: 35822700 PMCID: PMC9348847 DOI: 10.7554/elife.72440] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2021] [Accepted: 07/12/2022] [Indexed: 11/13/2022] Open
Abstract
Identifying factors whose fluctuations are associated with choice inconsistency is a major issue for rational decision theory. Here, we investigated the neuro-computational mechanisms through which mood fluctuations may bias human choice behavior. Intracerebral EEG data were collected in a large group of subjects (n=30) while they were performing interleaved quiz and choice tasks that were designed to examine how a series of unrelated feedbacks affect decisions between safe and risky options. Neural baseline activity preceding choice onset was confronted first to mood level, estimated by a computational model integrating the feedbacks received in the quiz task, and then to the weighting of option attributes, in a computational model predicting risk attitude in the choice task. Results showed that (1) elevated broadband gamma activity (BGA) in the ventromedial prefrontal cortex (vmPFC) and dorsal anterior insula (daIns) was respectively signaling periods of high and low mood, (2) increased vmPFC and daIns BGA respectively promoted and tempered risk taking by overweighting gain vs. loss prospects. Thus, incidental feedbacks induce brain states that correspond to different moods and bias the evaluation of risky options. More generally, these findings might explain why people experiencing positive (or negative) outcome in some part of their life tend to expect success (or failure) in any other.
Collapse
Affiliation(s)
| | | | - Jiri Hammer
- University Hospital in Motol, Prague, Czech Republic
| | - Petr Marusic
- University Hospital in Motol, Prague, Czech Republic
| | - Anca Nica
- Centre Hospitalier Universitaire de Rennes, Rennes, France
| | | | | | - Emmanuel J Barbeau
- Brain and Cognition Research Centre (CerCo), CNRS, University of Toulouse Paul Sabatier, Toulouse, France
| | - Marie Denuelle
- Centre Hospitalier Universitaire de Toulouse, Toulouse, France
| | | | - Lorella Minotti
- Centre Hospitalier Universitaire de Grenoble, Grenoble, France
| | - Philippe Kahane
- Centre Hospitalier Universitaire de Grenoble, Grenoble, France
| | | | | |
Collapse
|
41
|
Haarsma J, Kok P, Browning M. The promise of layer-specific neuroimaging for testing predictive coding theories of psychosis. Schizophr Res 2022; 245:68-76. [PMID: 33199171 PMCID: PMC9241988 DOI: 10.1016/j.schres.2020.10.009] [Citation(s) in RCA: 18] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/30/2020] [Revised: 10/03/2020] [Accepted: 10/28/2020] [Indexed: 12/24/2022]
Abstract
Predictive coding potentially provides an explanatory model for understanding the neurocognitive mechanisms of psychosis. It proposes that cognitive processes, such as perception and inference, are implemented by a hierarchical system, with the influence of each level being a function of the estimated precision of beliefs at that level. However, predictive coding models of psychosis are insufficiently constrained-any phenomenon can be explained in multiple ways by postulating different changes to precision at different levels of processing. One reason for the lack of constraint in these models is that the core processes are thought to be implemented by the function of specific cortical layers, and the technology to measure layer specific neural activity in humans has until recently been lacking. As a result, our ability to constrain the models with empirical data has been limited. In this review we provide a brief overview of predictive processing models of psychosis and then describe the potential for newly developed, layer specific neuroimaging techniques to test and thus constrain these models. We conclude by discussing the most promising avenues for this research as well as the technical and conceptual challenges which may limit its application.
Collapse
Affiliation(s)
- J. Haarsma
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom,Department of Psychiatry, University of Oxford, Oxford, United Kingdom,Corresponding author at: Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom.
| | - P. Kok
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
| | - M. Browning
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom,Oxford Health NHS Trust, Oxford, United Kingdom
| |
Collapse
|
42
|
Corlett PR, Mollick JA, Kober H. Meta-analysis of human prediction error for incentives, perception, cognition, and action. Neuropsychopharmacology 2022; 47:1339-1349. [PMID: 35017672 PMCID: PMC9117315 DOI: 10.1038/s41386-021-01264-3] [Citation(s) in RCA: 35] [Impact Index Per Article: 17.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Revised: 12/23/2021] [Accepted: 12/28/2021] [Indexed: 12/30/2022]
Abstract
Prediction errors (PEs) are a keystone for computational neuroscience. Their association with midbrain neural firing has been confirmed across species and has inspired the construction of artificial intelligence that can outperform humans. However, there is still much to learn. Here, we leverage the wealth of human PE data acquired in the functional neuroimaging setting in service of a deeper understanding, using an MKDA (multi-level kernel-based density) meta-analysis. Studies were identified with Google Scholar, and we included studies with healthy adult participants that reported activation coordinates corresponding to PEs published between 1999-2018. Across 264 PE studies that have focused on reward, punishment, action, cognition, and perception, consistent with domain-general theoretical models of prediction error we found midbrain PE signals during cognitive and reward learning tasks, and an insula PE signal for perceptual, social, cognitive, and reward prediction errors. There was evidence for domain-specific error signals--in the visual hierarchy during visual perception, and the dorsomedial prefrontal cortex during social inference. We assessed bias following prior neuroimaging meta-analyses and used family-wise error correction for multiple comparisons. This organization of computation by region will be invaluable in building and testing mechanistic models of cognitive function and dysfunction in machines, humans, and other animals. Limitations include small sample sizes and ROI masking in some included studies, which we addressed by weighting each study by sample size, and directly comparing whole brain vs. ROI-based results.
Collapse
Affiliation(s)
| | | | - Hedy Kober
- Department of Psychiatry, Yale University, New Haven, CT, USA.
| |
Collapse
|
43
|
Martinez-Saito M, Gorina E. Learning under social versus nonsocial uncertainty: A meta-analytic approach. Hum Brain Mapp 2022; 43:4185-4206. [PMID: 35620870 PMCID: PMC9374892 DOI: 10.1002/hbm.25948] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2021] [Revised: 04/08/2022] [Accepted: 05/04/2022] [Indexed: 01/10/2023] Open
Abstract
Much of the uncertainty that clouds our understanding of the world springs from the covert values and intentions held by other people. Thus, it is plausible that specialized mechanisms that compute learning signals under uncertainty of exclusively social origin operate in the brain. To test this hypothesis, we scoured academic databases for neuroimaging studies involving learning under uncertainty, and performed a meta‐analysis of brain activation maps that compared learning in the face of social versus nonsocial uncertainty. Although most of the brain activations associated with learning error signals were shared between social and nonsocial conditions, we found some evidence for functional segregation of error signals of exclusively social origin during learning in limited regions of ventrolateral prefrontal cortex and insula. This suggests that most behavioral adaptations to navigate social environments are reused from frontal and subcortical areas processing generic value representation and learning, but that a specialized circuitry might have evolved in prefrontal regions to deal with social context representation and strategic action.
Collapse
Affiliation(s)
| | - Elena Gorina
- Department of Cognitive and Brain Sciences, Hebrew University of Jerusalem, Jerusalem, Israel
| |
Collapse
|
44
|
Dennison JB, Sazhin D, Smith DV. Decision neuroscience and neuroeconomics: Recent progress and ongoing challenges. WILEY INTERDISCIPLINARY REVIEWS. COGNITIVE SCIENCE 2022; 13:e1589. [PMID: 35137549 PMCID: PMC9124684 DOI: 10.1002/wcs.1589] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/05/2020] [Revised: 11/28/2021] [Accepted: 12/21/2021] [Indexed: 01/10/2023]
Abstract
In the past decade, decision neuroscience and neuroeconomics have developed many new insights in the study of decision making. This review provides an overarching update on how the field has advanced in this time period. Although our initial review a decade ago outlined several theoretical, conceptual, methodological, empirical, and practical challenges, there has only been limited progress in resolving these challenges. We summarize significant trends in decision neuroscience through the lens of the challenges outlined for the field and review examples where the field has had significant, direct, and applicable impacts across economics and psychology. First, we review progress on topics including reward learning, explore-exploit decisions, risk and ambiguity, intertemporal choice, and valuation. Next, we assess the impacts of emotion, social rewards, and social context on decision making. Then, we follow up with how individual differences impact choices and new exciting developments in the prediction and neuroforecasting of future decisions. Finally, we consider how trends in decision-neuroscience research reflect progress toward resolving past challenges, discuss new and exciting applications of recent research, and identify new challenges for the field. This article is categorized under: Psychology > Reasoning and Decision Making Psychology > Emotion and Motivation.
Collapse
Affiliation(s)
- Jeffrey B Dennison
- Department of Psychology, Temple University, Philadelphia, Pennsylvania, USA
| | - Daniel Sazhin
- Department of Psychology, Temple University, Philadelphia, Pennsylvania, USA
| | - David V Smith
- Department of Psychology, Temple University, Philadelphia, Pennsylvania, USA
| |
Collapse
|
45
|
Lee M, Lindo J, Rilling JK. Exploring gene-culture coevolution in humans by inferring neuroendophenotypes: A case study of the oxytocin receptor gene and cultural tightness. GENES, BRAIN, AND BEHAVIOR 2022; 21:e12783. [PMID: 35044077 PMCID: PMC8917075 DOI: 10.1111/gbb.12783] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/27/2021] [Revised: 10/25/2021] [Accepted: 11/15/2021] [Indexed: 01/17/2023]
Abstract
The gene-culture coevolution (GCC) framework has gained increasing prominence in the social and biological sciences. While most studies on human GCC concern the evolution of low-level physiological traits, attempts have also been made to apply GCC to complex human traits, including social behavior and cognition. One major methodological challenge in this endeavor is to reconstruct a specific biological pathway between the implicated genes and their distal phenotypes. Here, we introduce a novel approach that combines data on population genetics and expression quantitative trait loci to infer the specific intermediate phenotypes of genes in the brain. We suggest that such "neuroendophenotypes" will provide more detailed mechanistic insights into the GCC process. We present a case study where we explored a GCC dynamics between the oxytocin receptor gene (OXTR) and cultural tightness-looseness. By combining data from the 1000 Genomes project and the Gene-Tissue-Expression project (GTEx), we estimated and compared OXTR expression in 10 brain regions across five human superpopulations. We found that OXTR expression in the anterior cingulate cortex (ACC) was highly variable across populations, and this variation correlated with cultural tightness and socio-ecological threats worldwide. The mediation models also suggested possible GCC dynamics where the increased OXTR expression in the ACC mediates or emerges from the tight culture and higher socio-ecological threats. Formal selection scans further confirmed that OXTR alleles linked to enhanced receptor expression in the ACC underwent positive selection in East Asian countries with tighter social norms. We discuss the implications of our method in human GCC research.
Collapse
Affiliation(s)
- Minwoo Lee
- Department of Anthropology, Emory UniversityAtlantaGeorgiaUSA
| | - John Lindo
- Department of Anthropology, Emory UniversityAtlantaGeorgiaUSA
| | - James K. Rilling
- Department of Anthropology, Emory UniversityAtlantaGeorgiaUSA,Department of Psychiatry and Behavioral Science, Emory UniversityAtlantaGeorgiaUSA,Center for Behavioral Neuroscience, Emory UniversityAtlantaGeorgiaUSA,Yerkes National Primate Research Center, Emory UniversityAtlantaGeorgiaUSA,Center for Translational Social Neuroscience, Emory UniversityAtlantaGeorgiaUSA
| |
Collapse
|
46
|
Millman ZB, Schiffman J, Gold JM, Akouri-Shan L, Demro C, Fitzgerald J, Rakhshan Rouhakhtar PJ, Klaunig M, Rowland LM, Waltz JA. Linking Salience Signaling With Early Adversity and Affective Distress in Individuals at Clinical High Risk for Psychosis: Results From an Event-Related fMRI Study. SCHIZOPHRENIA BULLETIN OPEN 2022; 3:sgac039. [PMID: 35799887 PMCID: PMC9250803 DOI: 10.1093/schizbullopen/sgac039] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Evidence suggests dysregulation of the salience network in individuals with psychosis, but few studies have examined the intersection of stress exposure and affective distress with prediction error (PE) signals among youth at clinical high-risk (CHR). Here, 26 individuals at CHR and 19 healthy volunteers (HVs) completed a monetary incentive delay task in conjunction with fMRI. We compared these groups on the amplitudes of neural responses to surprising outcomes-PEs without respect to their valence-across the whole brain and in two regions of interest, the anterior insula and amygdala. We then examined relations of these signals to the severity of depression, anxiety, and trauma histories in the CHR group. Relative to HV, youth at CHR presented with aberrant PE-evoked activation of the temporoparietal junction and weaker deactivation of the precentral gyrus, posterior insula, and associative striatum. No between-group differences were observed in the amygdala or anterior insula. Among youth at CHR, greater trauma histories were correlated with stronger PE-evoked amygdala activation. No associations were found between affective symptoms and the neural responses to PE. Our results suggest that unvalenced PE signals may provide unique information about the neurobiology of CHR syndromes and that early adversity exposure may contribute to neurobiological heterogeneity in this group. Longitudinal studies of young people with a range of risk syndromes are needed to further disentangle the contributions of distinct aspects of salience signaling to the development of psychopathology.
Collapse
Affiliation(s)
- Zachary B Millman
- Psychotic Disorders Division, McLean Hospital, 115 Mill Street, Belmont, MA 02478, USA
- Department of Psychiatry, Harvard Medical School, 25 Shattuck Street, Boston, MA 02114, USA
| | - Jason Schiffman
- Department of Psychological Science, University of California, Irvine, 4201 Social and Behavioral Sciences Gateway, Irvine, CA 92697-7085, USA
- Department of Psychology, University of Maryland, Baltimore County, 1000 Hilltop Circle, Baltimore, MD 21250, USA
| | - James M Gold
- Maryland Psychiatric Research Center, University of Maryland School of Medicine, 55 Wade Avenue, Catonsville, MD 21228, USA
| | - LeeAnn Akouri-Shan
- Department of Psychology, University of Maryland, Baltimore County, 1000 Hilltop Circle, Baltimore, MD 21250, USA
| | - Caroline Demro
- Department of Psychology, University of Minnesota, 75 East River Parkway, Minneapolis, MN 55455, USA
| | - John Fitzgerald
- Department of Psychology, University of Maryland, Baltimore County, 1000 Hilltop Circle, Baltimore, MD 21250, USA
| | - Pamela J Rakhshan Rouhakhtar
- Department of Psychology, University of Maryland, Baltimore County, 1000 Hilltop Circle, Baltimore, MD 21250, USA
| | - Mallory Klaunig
- Department of Psychology, University of Maryland, Baltimore County, 1000 Hilltop Circle, Baltimore, MD 21250, USA
| | - Laura M Rowland
- Maryland Psychiatric Research Center, University of Maryland School of Medicine, 55 Wade Avenue, Catonsville, MD 21228, USA
| | - James A Waltz
- Maryland Psychiatric Research Center, University of Maryland School of Medicine, 55 Wade Avenue, Catonsville, MD 21228, USA
| |
Collapse
|
47
|
Kesby JP, Murray GK, Knolle F. Neural Circuitry of Salience and Reward Processing in Psychosis. BIOLOGICAL PSYCHIATRY GLOBAL OPEN SCIENCE 2021; 3:33-46. [PMID: 36712572 PMCID: PMC9874126 DOI: 10.1016/j.bpsgos.2021.12.003] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Revised: 11/25/2021] [Accepted: 12/01/2021] [Indexed: 02/01/2023] Open
Abstract
The processing of salient and rewarding stimuli is integral to engaging our attention, stimulating anticipation for future events, and driving goal-directed behaviors. Widespread impairments in these processes are observed in psychosis, which may be associated with worse functional outcomes or mechanistically linked to the development of symptoms. Here, we summarize the current knowledge of behavioral and functional neuroimaging in salience, prediction error, and reward. Although each is a specific process, they are situated in multiple feedback and feedforward systems integral to decision making and cognition more generally. We argue that the origin of salience and reward processing dysfunctions may be centered in the subcortex during the earliest stages of psychosis, with cortical abnormalities being initially more spared but becoming more prominent in established psychotic illness/schizophrenia. The neural circuits underpinning salience and reward processing may provide targets for delaying or preventing progressive behavioral and neurobiological decline.
Collapse
Affiliation(s)
- James P. Kesby
- Queensland Brain Institute, The University of Queensland, Brisbane, Queensland, Australia,QIMR Berghofer Medical Research Institute, Brisbane, Queensland, Australia,Address correspondence to James Kesby, Ph.D.
| | - Graham K. Murray
- Institute for Molecular Bioscience, The University of Queensland, Brisbane, Queensland, Australia,Department of Psychiatry, University of Cambridge, Cambridge, United Kingdom,Cambridgeshire and Peterborough NHS Foundation Trust, Cambridge, United Kingdom
| | - Franziska Knolle
- Department of Psychiatry, University of Cambridge, Cambridge, United Kingdom,Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Technical University of Munich, Munich, Germany,Franziska Knolle, Ph.D.
| |
Collapse
|
48
|
Liakoni V, Lehmann MP, Modirshanechi A, Brea J, Lutti A, Gerstner W, Preuschoff K. Brain signals of a Surprise-Actor-Critic model: Evidence for multiple learning modules in human decision making. Neuroimage 2021; 246:118780. [PMID: 34875383 DOI: 10.1016/j.neuroimage.2021.118780] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2021] [Revised: 08/03/2021] [Accepted: 12/04/2021] [Indexed: 11/25/2022] Open
Abstract
Learning how to reach a reward over long series of actions is a remarkable capability of humans, and potentially guided by multiple parallel learning modules. Current brain imaging of learning modules is limited by (i) simple experimental paradigms, (ii) entanglement of brain signals of different learning modules, and (iii) a limited number of computational models considered as candidates for explaining behavior. Here, we address these three limitations and (i) introduce a complex sequential decision making task with surprising events that allows us to (ii) dissociate correlates of reward prediction errors from those of surprise in functional magnetic resonance imaging (fMRI); and (iii) we test behavior against a large repertoire of model-free, model-based, and hybrid reinforcement learning algorithms, including a novel surprise-modulated actor-critic algorithm. Surprise, derived from an approximate Bayesian approach for learning the world-model, is extracted in our algorithm from a state prediction error. Surprise is then used to modulate the learning rate of a model-free actor, which itself learns via the reward prediction error from model-free value estimation by the critic. We find that action choices are well explained by pure model-free policy gradient, but reaction times and neural data are not. We identify signatures of both model-free and surprise-based learning signals in blood oxygen level dependent (BOLD) responses, supporting the existence of multiple parallel learning modules in the brain. Our results extend previous fMRI findings to a multi-step setting and emphasize the role of policy gradient and surprise signalling in human learning.
Collapse
Affiliation(s)
- Vasiliki Liakoni
- École Polytechnique Fédérale de Lausanne (EPFL), School of Computer and Communication Sciences and School of Life Sciences, Lausanne, Switzerland.
| | - Marco P Lehmann
- École Polytechnique Fédérale de Lausanne (EPFL), School of Computer and Communication Sciences and School of Life Sciences, Lausanne, Switzerland
| | - Alireza Modirshanechi
- École Polytechnique Fédérale de Lausanne (EPFL), School of Computer and Communication Sciences and School of Life Sciences, Lausanne, Switzerland
| | - Johanni Brea
- École Polytechnique Fédérale de Lausanne (EPFL), School of Computer and Communication Sciences and School of Life Sciences, Lausanne, Switzerland
| | - Antoine Lutti
- Laboratoire de recherche en neuroimagerie (LREN), Department of Clinical Neurosciences, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Wulfram Gerstner
- École Polytechnique Fédérale de Lausanne (EPFL), School of Computer and Communication Sciences and School of Life Sciences, Lausanne, Switzerland
| | - Kerstin Preuschoff
- Geneva Finance Research Institute & Interfaculty Center for Affective Sciences, University of Geneva, Geneva, Switzerland
| |
Collapse
|
49
|
Kalbe F, Schwabe L. Prediction Errors for Aversive Events Shape Long-Term Memory Formation through a Distinct Neural Mechanism. Cereb Cortex 2021; 32:3081-3097. [PMID: 34849622 DOI: 10.1093/cercor/bhab402] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Revised: 09/09/2021] [Accepted: 10/12/2021] [Indexed: 11/13/2022] Open
Abstract
Prediction errors (PEs) have been known for decades to guide associative learning, but their role in episodic memory formation has been discovered only recently. To identify the neural mechanisms underlying the impact of aversive PEs on long-term memory formation, we used functional magnetic resonance imaging, while participants saw a series of unique stimuli and estimated the probability that an aversive shock would follow. Our behavioral data showed that negative PEs (i.e., omission of an expected outcome) were associated with superior recognition of the predictive stimuli, whereas positive PEs (i.e., presentation of an unexpected outcome) impaired subsequent memory. While medial temporal lobe (MTL) activity during stimulus encoding was overall associated with enhanced memory, memory-enhancing effects of negative PEs were linked to even decreased MTL activation. Additional large-scale network analyses showed PE-related increases in crosstalk between the "salience network" and a frontoparietal network commonly implicated in memory formation for expectancy-congruent events. These effects could not be explained by mere changes in physiological arousal or the prediction itself. Our results suggest that the superior memory for events associated with negative aversive PEs is driven by a potentially distinct neural mechanism that might serve to set these memories apart from those with expected outcomes.
Collapse
Affiliation(s)
- Felix Kalbe
- Department of Cognitive Psychology, Institute of Psychology, Universität Hamburg, Hamburg 20146, Germany
| | - Lars Schwabe
- Department of Cognitive Psychology, Institute of Psychology, Universität Hamburg, Hamburg 20146, Germany
| |
Collapse
|
50
|
Inkster AB, Milton F, Edmunds CER, Benattayallah A, Wills AJ. Neural correlates of the inverse base rate effect. Hum Brain Mapp 2021; 43:1370-1380. [PMID: 34826165 PMCID: PMC8837595 DOI: 10.1002/hbm.25729] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2021] [Revised: 10/16/2021] [Accepted: 11/15/2021] [Indexed: 12/05/2022] Open
Abstract
The inverse base rate effect (IBRE) is a nonrational behavioral phenomenon in predictive learning. Canonically, participants learn that the AB stimulus compound leads to one outcome and that AC leads to another outcome, with AB being presented three times as often as AC. When subsequently presented with BC, the outcome associated with AC is preferentially selected, in opposition to the underlying base rates of the outcomes. The current leading explanation is based on error‐driven learning. A key component of this account is prediction error, a concept previously linked to a number of brain areas including the anterior cingulate, the striatum, and the dorsolateral prefrontal cortex. The present work is the first fMRI study to directly examine the IBRE. Activations were noted in brain areas linked to prediction error, including the caudate body, the anterior cingulate, the ventromedial prefrontal cortex, and the right dorsolateral prefrontal cortex. Analyzing the difference in activations for singular key stimuli (B and C), as well as frequency matched controls, supports the predictions made by the error‐driven learning account.
Collapse
Affiliation(s)
- Angus B Inkster
- Brain Research and Imaging Centre, University of Plymouth, Plymouth
| | | | | | | | - Andy J Wills
- Brain Research and Imaging Centre, University of Plymouth, Plymouth
| |
Collapse
|