1
|
Wolff S, Brechmann A. Dorsal posterior cingulate cortex responds to negative feedback information supporting learning and relearning of response policies. Cereb Cortex 2022; 33:5947-5956. [PMID: 36533512 DOI: 10.1093/cercor/bhac473] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2022] [Revised: 11/09/2022] [Accepted: 11/10/2022] [Indexed: 12/23/2022] Open
Abstract
Abstract
Many challenges in life come without explicit instructions. Instead, humans need to test, select, and adapt their behavioral responses based on feedback from the environment. While reward-centric accounts of feedback processing primarily stress the reinforcing aspect of positive feedback, feedback’s central function from an information-processing perspective is to offer an opportunity to correct errors, thus putting a greater emphasis on the informational content of negative feedback. Independent of its potential rewarding value, the informational value of performance feedback has recently been suggested to be neurophysiologically encoded in the dorsal portion of the posterior cingulate cortex (dPCC). To further test this association, we investigated multidimensional categorization and reversal learning by comparing negative and positive feedback in an event-related functional magnetic resonance imaging experiment. Negative feedback, compared with positive feedback, increased activation in the dPCC as well as in brain regions typically involved in error processing. Only in the dPCC, subarea d23, this effect was significantly enhanced in relearning, where negative feedback signaled the need to shift away from a previously established response policy. Together with previous findings, this result contributes to a more fine-grained functional parcellation of PCC subregions and supports the dPCC’s involvement in the adaptation to behaviorally relevant information from the environment.
Collapse
Affiliation(s)
- Susann Wolff
- Leibniz Institute for Neurobiology Combinatorial NeuroImaging, , Brenneckestr. 6, Magdeburg 39118 , Germany
| | - André Brechmann
- Leibniz Institute for Neurobiology Combinatorial NeuroImaging, , Brenneckestr. 6, Magdeburg 39118 , Germany
| |
Collapse
|
2
|
The involvement of the posterior parietal cortex in promotion and prevention focus. CURRENT PSYCHOLOGY 2022. [DOI: 10.1007/s12144-022-03731-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Abstract
AbstractMotivation plays a critical role in human cognitive function, while acting as a driving force for the necessary behavior to achieve a desired goal and success (i.e., achievement motivation). Based on the theoretical background of achievement motivation, this study designed an incentive delay task with four motivational orientations (i.e., promotion, prevention, mastery/self, and performance/other). To investigate whether people would have their behavioral patterns toward achievement motivation orientation, we applied an unsupervised clustering algorithm to classify individuals’ behavioral responses acquired from the task by categorizing certain behavioral similarities. As a result, this hierarchical clustering approach classified subjects into two distinctive subgroups: Group#1 (i.e., the pro/pre group, n = 52) and Group#2 (i.e., the self/other group, n = 48). Based on clustering, Group#1 showed significantly better performance with promotion/prevention orientations, whereas Group#2 exhibited significantly higher performance with self/other orientations. Structural brain analyses discovered increased gray matter volume and sulcal depth in the posterior parietal cortex (PPC) in the pro/pre group compared to the self/other group. With resting-state functional magnetic resonance imaging data, we found higher local brain fluctuations in the medial prefrontal cortex (mPFC) in the self/other group compared to the pro/pre group. Furthermore, mPFC seed-based functional connectivity showed significantly increased functional coupling with the posterior cingulate cortex in the self/other group relative to the pro/pre group. Taken together, these results shed light on structural and functional neural mechanisms related to achievement motivation and, furthermore, provide novel insights regarding PPC’s role in motivational processing toward promotion- and prevention-focused orientation.
Collapse
|
3
|
Tashjian SM, Galván A. Frontopolar Cortex Response to Positive Feedback Relates to Nonincentivized Task Persistence. Cereb Cortex 2021; 32:2293-2309. [PMID: 34581407 DOI: 10.1093/cercor/bhab317] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023] Open
Abstract
When individuals make decisions whether to persist at a task, their decision-making is informed by whether success is pending or accomplished. If pending, the brain facilitates behavioral persistence; if the goal is accomplished or no longer desired, the brain enables switching away from the current task. Feedback, which is known to differentially engage reward neurocircuitry, may modulate goal-directed behavior such as task persistence. However, prior studies are confounded by offering external incentives for persistence. This study tested whether neural response to feedback differed as a function of nonincentivized task persistence in 99 human participants ages 13-30 (60 females). Individuals who persisted engaged the frontopolar cortex (FPC) to a greater extent during receipt of task-relevant positive feedback compared with negative feedback. For individuals who quit, task-irrelevant monetary reward engaged the FPC to a greater extent compared with positive feedback. FPC activation in response to positive feedback is identified as a key contributor to task persistence.
Collapse
Affiliation(s)
- Sarah M Tashjian
- Department of Psychology, University of California at Los Angeles, Los Angeles, CA 90095, USA
| | - Adriana Galván
- Department of Psychology, University of California at Los Angeles, Los Angeles, CA 90095, USA.,Brain Research Institute, University of California at Los Angeles, Los Angeles, CA 90095, USA
| |
Collapse
|
4
|
Dobryakova E, Zuckerman S, Sandry J. Neural correlates of extrinsic and intrinsic outcome processing during learning in individuals with TBI: a pilot investigation. Brain Imaging Behav 2021; 16:344-354. [PMID: 34406636 DOI: 10.1007/s11682-021-00508-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/15/2021] [Indexed: 10/20/2022]
Abstract
Outcome processing, the ability to learn from feedback, is an important component of adaptive behavior and rehabilitation. Evidence from healthy adults implicates the striatum and dopamine in outcome processing. Animal research shows that damage to dopaminergic pathways in the brain can lead to a disruption of dopamine tone and transmission. Such evidence thus suggests that persons with TBI experience deficits in outcome processing. However, no research has directly investigated outcome processing and associated neural mechanisms in TBI. Here, we examine outcome processing in individuals with TBI during learning. Given that TBI negatively impacts striatal and dopaminergic systems, we hypothesize that individuals with TBI exhibit deficits in learning from outcomes. To test this hypothesis, individuals with moderate-to-severe TBI and healthy adults were presented with a declarative paired-associate word learning task. Outcomes indicating performance accuracy were presented immediately during task performance and in the form of either monetary or performance-based feedback. Two types of feedback provided the opportunity to test whether extrinsic and intrinsic motivational aspects of outcome presentation play a role during learning and outcome processing. Our results show that individuals with TBI exhibited impaired learning from feedback compared to healthy participants. Additionally, individuals with TBI exhibited increased activation in the striatum during outcome processing. The results of this study suggest that outcome processing and learning from immediate outcomes is impaired in individuals with TBI and might be related to inefficient use of neural resources during task performance as reflected by increased activation of the striatum.
Collapse
Affiliation(s)
- Ekaterina Dobryakova
- Center for Traumatic Brain Injury Research, Kessler Foundation, 120 Eagle Rock Ave., East Hanover, NJ, 07936, USA. .,Department of Physical Medicine and Rehabilitation, Rutgers-New Jersey Medical School, Newark, NJ, USA.
| | - Suzanne Zuckerman
- Center for Traumatic Brain Injury Research, Kessler Foundation, 120 Eagle Rock Ave., East Hanover, NJ, 07936, USA
| | - Joshua Sandry
- Psychology Department, Montclair State University, 1 Normal Ave., Montclair, NJ, USA
| |
Collapse
|
5
|
Xin Y, Xu P, Aleman A, Luo Y, Feng T. Intrinsic prefrontal organization underlies associations between achievement motivation and delay discounting. Brain Struct Funct 2020; 225:511-518. [PMID: 31932869 DOI: 10.1007/s00429-019-01982-x] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2019] [Accepted: 10/30/2019] [Indexed: 01/30/2023]
Abstract
Achievement motivation is a core component of human decision making. However, neural mechanisms that link achievement motivation and intertemporal choice have not yet been elucidated. Here, we examined neural pathways underlying the relationship between achievement motivation and intertemporal choice using a delay discounting task and resting-state functional magnetic resonance imaging on 86 healthy subjects. Behaviorally, delay discounting rate was positively correlated with achievement motivation. Functional coupling of the dorsolateral prefrontal cortex (dlPFC) with the medial prefrontal cortex (mPFC), medial orbitofrontal cortex and ventral striatum was positively correlated with achievement motivation. Notably, the mediation analysis showed that the impact of achievement motivation on delay discounting was mediated by intrinsic connectivity between the dlPFC and mPFC. Our findings suggest that intrinsic organization within the prefrontal cortex plays a key role in linking achievement motivation and intertemporal choice.
Collapse
Affiliation(s)
- Yuanyuan Xin
- Research Center of Psychology and Social Development, Faculty of Psychology, Southwest University, Chongqing, 400715, China.,Shenzhen Key Laboratory of Affective and Social Cognitive Science, Shenzhen University, Shenzhen, 518060, China.,Department of Neuroscience, University Medical Center Groningen, University of Groningen, 9713 AW, Groningen, The Netherlands
| | - Pengfei Xu
- Shenzhen Key Laboratory of Affective and Social Cognitive Science, Shenzhen University, Shenzhen, 518060, China.,Center for Emotion and Brain, Shenzhen Institute of Neuroscience, Shenzhen, 518057, China.,Department of Neuroscience, University Medical Center Groningen, University of Groningen, 9713 AW, Groningen, The Netherlands
| | - André Aleman
- Shenzhen Key Laboratory of Affective and Social Cognitive Science, Shenzhen University, Shenzhen, 518060, China.,Department of Neuroscience, University Medical Center Groningen, University of Groningen, 9713 AW, Groningen, The Netherlands
| | - Yuejia Luo
- Shenzhen Key Laboratory of Affective and Social Cognitive Science, Shenzhen University, Shenzhen, 518060, China. .,Center for Emotion and Brain, Shenzhen Institute of Neuroscience, Shenzhen, 518057, China.
| | - Tingyong Feng
- Research Center of Psychology and Social Development, Faculty of Psychology, Southwest University, Chongqing, 400715, China. .,Key Laboratory of Cognition and Personality, Ministry of Education, Chongqing, China.
| |
Collapse
|
6
|
Ballard I, Miller EM, Piantadosi ST, Goodman ND, McClure SM. Beyond Reward Prediction Errors: Human Striatum Updates Rule Values During Learning. Cereb Cortex 2019; 28:3965-3975. [PMID: 29040494 DOI: 10.1093/cercor/bhx259] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2017] [Accepted: 09/13/2017] [Indexed: 11/13/2022] Open
Abstract
Humans naturally group the world into coherent categories defined by membership rules. Rules can be learned implicitly by building stimulus-response associations using reinforcement learning or by using explicit reasoning. We tested if the striatum, in which activation reliably scales with reward prediction error, would track prediction errors in a task that required explicit rule generation. Using functional magnetic resonance imaging during a categorization task, we show that striatal responses to feedback scale with a "surprise" signal derived from a Bayesian rule-learning model and are inconsistent with RL prediction error. We also find that striatum and caudal inferior frontal sulcus (cIFS) are involved in updating the likelihood of discriminative rules. We conclude that the striatum, in cooperation with the cIFS, is involved in updating the values assigned to categorization rules when people learn using explicit reasoning.
Collapse
Affiliation(s)
- Ian Ballard
- Stanford Neurosciences Graduate Training Program, Stanford University, Stanford, CA, USA
| | - Eric M Miller
- Department of Psychology, Stanford University, Stanford, CA, USA
| | - Steven T Piantadosi
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY, USA
| | - Noah D Goodman
- Department of Psychology, Stanford University, Stanford, CA, USA
| | - Samuel M McClure
- Department of Psychology, Arizona State University, Tempe, AZ, USA
| |
Collapse
|
7
|
DiMenichi BC, Ceceli AO, Bhanji JP, Tricomi E. Effects of Expressive Writing on Neural Processing During Learning. Front Hum Neurosci 2019; 13:389. [PMID: 31803032 PMCID: PMC6869814 DOI: 10.3389/fnhum.2019.00389] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2018] [Accepted: 10/17/2019] [Indexed: 12/01/2022] Open
Abstract
Expressive writing about past negative events has been shown to lead to a slew of positive outcomes. However, little is known about why writing about something negative would have positive effects. While some have posited that telling a narrative of a past negative event or current anxiety "frees up" cognitive resources, allowing individuals to focus more on the task at hand, there is little neural evidence suggesting that expressive writing has an effect on cognitive load. Moreover, little is known about how individual differences in the content of expressive writing could affect neural processing and the cognitive benefits writing confers. In our experiment, we compared brain activity in a group that had engaged in expressive writing vs. a control group, during performance on a feedback-based paired-associate word-learning task. We found that across groups, differential activation in the dorsal striatum in response to positive vs. negative feedback significantly predicted better later memory. Moreover, writing about a past failure resulted in more activation relative to the control group during the learning task in the mid-cingulate cortex (MCC), an area of the brain crucial to processing negative emotion. While our results do not provide support for the assertion that expressive writing alters attentional processing, our findings suggest that choosing to write about particularly intense past negative experiences like a difficult past failure may have resulted in changes in neural activation during task processing.
Collapse
Affiliation(s)
| | | | | | - Elizabeth Tricomi
- Department of Psychology, Rutgers University, Newark, NJ, United States
| |
Collapse
|
8
|
Bejjani C, DePasque S, Tricomi E. Intelligence mindset shapes neural learning signals and memory. Biol Psychol 2019; 146:107715. [PMID: 31212006 DOI: 10.1016/j.biopsycho.2019.06.003] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2017] [Revised: 04/01/2019] [Accepted: 06/04/2019] [Indexed: 11/18/2022]
Abstract
Intelligence mindset, which denotes individual beliefs about whether intelligence is fixed versus malleable, shapes academic success, but the neural mechanisms underlying mindset-related differences in learning are unknown. Here, we probe the effects of individual differences in mindset on neural responses to negative feedback after a competence threat manipulation. We hypothesized that when their competence was threatened, participants with fixed mindsets would interpret further negative feedback as punishing. After receiving either no score or a competence-threatening IQ score, participants performed a learning task with feedback that emphasized either the evaluative or informational weight of negative feedback. Participants who experienced the competence threat had the strongest predictive relationships between mindset, performance, and caudate activation. The competence threat may have compounded the subjective punishment of negative feedback for fixed mindsets relative to growth mindsets, causing poorer learning from negative feedback in the evaluative context and inflexible striatal responses to negative feedback across feedback contexts.
Collapse
Affiliation(s)
- Christina Bejjani
- Department of Psychology, Rutgers University, Newark, NJ 07102, United States.
| | - Samantha DePasque
- Department of Psychology, Rutgers University, Newark, NJ 07102, United States
| | - Elizabeth Tricomi
- Department of Psychology, Rutgers University, Newark, NJ 07102, United States
| |
Collapse
|
9
|
Skottnik L, Sorger B, Kamp T, Linden D, Goebel R. Success and failure of controlling the real-time functional magnetic resonance imaging neurofeedback signal are reflected in the striatum. Brain Behav 2019; 9:e01240. [PMID: 30790474 PMCID: PMC6422826 DOI: 10.1002/brb3.1240] [Citation(s) in RCA: 36] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/03/2018] [Revised: 01/24/2019] [Accepted: 01/25/2019] [Indexed: 12/27/2022] Open
Abstract
INTRODUCTION Over the last decades, neurofeedback has been applied in variety of research contexts and therapeutic interventions. Despite this extensive use, its neural mechanisms are still under debate. Several scientific advances have suggested that different networks become jointly active during neurofeedback, including regions generally involved in self-regulation, regions related to the specific mental task driving the neurofeedback and regions generally involved in feedback learning (Sitaram et al., 2017, Nature Reviews Neuroscience, 18, 86). METHODS To investigate the neural mechanisms specific to neurofeedback but independent from general effects of self-regulation, we compared brain activation as measured with functional magnetic resonance imaging (fMRI) across different mental tasks involving gradual self-regulation with and without providing neurofeedback. Ten participants freely chose one self-regulation task and underwent two training sessions during fMRI scanning, one with and one without receiving neurofeedback. During neurofeedback sessions, feedback signals were provided in real-time based on activity in task-related, individually defined target regions. In both sessions, participants aimed at reaching and holding low, medium, or high brain-activation levels in the target region. RESULTS During gradual self-regulation with neurofeedback, a network of cortical control regions as well as regions implicated in reward and feedback processing were activated. Self-regulation with feedback was accompanied by stronger activation within the striatum across different mental tasks. Additional time-resolved single-trial analysis revealed that neurofeedback performance was positively correlated with a delayed brain response in the striatum that reflected the accuracy of self-regulation. CONCLUSION Overall, these findings support that neurofeedback contributes to self-regulation through task-general regions involved in feedback and reward processing.
Collapse
Affiliation(s)
- Leon Skottnik
- Department of Psychiatry and Neuropsychology, Maastricht University, Maastricht, Netherlands.,Department of Cognitive Neuroscience, Maastricht University, Maastricht, Netherlands.,Brain Innovation BV, Maastricht, Netherlands
| | - Bettina Sorger
- Department of Cognitive Neuroscience, Maastricht University, Maastricht, Netherlands
| | - Tabea Kamp
- Department of Cognitive Neuroscience, Maastricht University, Maastricht, Netherlands
| | - David Linden
- MRC Centre for Neuropsychiatric Genetics and Genomics, School of Medicine, Cardiff University, Cardiff, United Kingdom.,School of Mental Health and Neuroscience, Maastricht University, Maastricht, Netherlands
| | - Rainer Goebel
- Department of Cognitive Neuroscience, Maastricht University, Maastricht, Netherlands.,Brain Innovation BV, Maastricht, Netherlands.,Department of Neuroimaging and Neuromodeling, Netherlands Institute for Neuroscience, an institute of the Royal Netherlands Academy of Arts and Sciences (KNAW), Amsterdam, Netherlands
| |
Collapse
|
10
|
DePasque S, Galván A. Neurobiological responses in the adolescent striatum to being 'tested'. Soc Cogn Affect Neurosci 2019; 14:3-12. [PMID: 30535253 PMCID: PMC6318469 DOI: 10.1093/scan/nsy104] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2018] [Revised: 11/07/2018] [Accepted: 11/21/2018] [Indexed: 11/12/2022] Open
Abstract
While emerging research implicates the striatum in adolescents' ability to learn from feedback, little is known about how motivational contexts, such as emphasizing the evaluative nature of learning tasks, modulate adolescents' striatal learning. We used functional magnetic resonance imaging during a feedback-based learning task, in conjunction with a within-subject evaluative threat manipulation, to determine whether evaluation threat influences behavioral and neural responses to feedback in adolescents. On average, adolescents were less sensitive than adults to the evaluation threat. In the adolescents, the effect of evaluation threat on performance was tracked with the striatal response to performance feedback during the evaluation threat condition, such that greater striatal sensitivity correlated with greater gains in learning performance. Our findings suggest that variability in how adolescents respond to a contextual threat of evaluation and associated striatal sensitivity can facilitate enhanced learning.
Collapse
Affiliation(s)
- Samantha DePasque
- Department of Psychology, University of California, Los Angeles, 1285 Franz Hall, PO Box 951563, Los Angeles, CA, USA
| | - Adriana Galván
- Department of Psychology, University of California, Los Angeles, 1285 Franz Hall, PO Box 951563, Los Angeles, CA, USA
| |
Collapse
|
11
|
Reeve J, Lee W. A neuroscientific perspective on basic psychological needs. J Pers 2018; 87:102-114. [PMID: 29626342 DOI: 10.1111/jopy.12390] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2017] [Revised: 03/12/2018] [Accepted: 03/20/2018] [Indexed: 11/29/2022]
Abstract
OBJECTIVE Self-determination theory's (SDT) most basic propositions are, first, that all people possess an inherent set of psychological needs and, second, that autonomy, competence, and relatedness are the three critical psychological satisfactions needed to maintain and promote personal growth and well-being. In this article, we identify the neural basis of the psychological needs and, in doing so, seek to advance the integration of SDT and neuroscience. METHOD We examine the neural underpinnings of SDT-based motivational states and traits for autonomy, competence, and relatedness. To study motivational states, participants are exposed to situational conditions known to affect their psychological needs, and neuroscience methods (e.g., fMRI) are used to examine changes in their brain activity. To study motivational traits, participants complete self-report trait measures that are then correlated with their brain activity observed during need-satisfying activities. RESULTS For both motivational states and traits and across all three needs, intrinsic satisfaction is associated with striatum-based reward processing, anterior insula-based subjective feelings, and the integration of these subjective feelings with reward-based processing. CONCLUSIONS Psychological need satisfaction is associated with striatum activity, anterior insula activity, and the functional coactivation between these two brain areas. Given these findings, it is now clear that several opportunities exist to integrate SDT motivational constructs with neuroscientific study, so we suggest eight new questions for future research.
Collapse
Affiliation(s)
- Johnmarshall Reeve
- Department of Education, Brain and Motivation Research Institute, Korea University
| | - Woogul Lee
- Department of Education, Korea National University of Education
| |
Collapse
|
12
|
Pine A, Sadeh N, Ben-Yakov A, Dudai Y, Mendelsohn A. Knowledge acquisition is governed by striatal prediction errors. Nat Commun 2018; 9:1673. [PMID: 29700377 PMCID: PMC5919975 DOI: 10.1038/s41467-018-03992-5] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2017] [Accepted: 03/27/2018] [Indexed: 11/09/2022] Open
Abstract
Discrepancies between expectations and outcomes, or prediction errors, are central to trial-and-error learning based on reward and punishment, and their neurobiological basis is well characterized. It is not known, however, whether the same principles apply to declarative memory systems, such as those supporting semantic learning. Here, we demonstrate with fMRI that the brain parametrically encodes the degree to which new factual information violates expectations based on prior knowledge and beliefs-most prominently in the ventral striatum, and cortical regions supporting declarative memory encoding. These semantic prediction errors determine the extent to which information is incorporated into long-term memory, such that learning is superior when incoming information counters strong incorrect recollections, thereby eliciting large prediction errors. Paradoxically, by the same account, strong accurate recollections are more amenable to being supplanted by misinformation, engendering false memories. These findings highlight a commonality in brain mechanisms and computational rules that govern declarative and nondeclarative learning, traditionally deemed dissociable.
Collapse
Affiliation(s)
- Alex Pine
- Sagol Department of Neurobiology, University of Haifa, Haifa, 3498838, Israel.
- Department of Neurobiology, Weizmann Institute of Science, Rehovot, 76100, Israel.
| | - Noa Sadeh
- Department of Neurobiology, Weizmann Institute of Science, Rehovot, 76100, Israel
| | - Aya Ben-Yakov
- Department of Neurobiology, Weizmann Institute of Science, Rehovot, 76100, Israel
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, CB27EF, UK
| | - Yadin Dudai
- Department of Neurobiology, Weizmann Institute of Science, Rehovot, 76100, Israel
| | - Avi Mendelsohn
- Sagol Department of Neurobiology, University of Haifa, Haifa, 3498838, Israel.
- The Institute of Information Processing and Decision Making (IIPDM), University of Haifa, Haifa, Israel.
| |
Collapse
|
13
|
Oldham S, Murawski C, Fornito A, Youssef G, Yücel M, Lorenzetti V. The anticipation and outcome phases of reward and loss processing: A neuroimaging meta-analysis of the monetary incentive delay task. Hum Brain Mapp 2018; 39:3398-3418. [PMID: 29696725 PMCID: PMC6055646 DOI: 10.1002/hbm.24184] [Citation(s) in RCA: 261] [Impact Index Per Article: 43.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2017] [Revised: 03/28/2018] [Accepted: 04/09/2018] [Indexed: 12/19/2022] Open
Abstract
The processing of rewards and losses are crucial to everyday functioning. Considerable interest has been attached to investigating the anticipation and outcome phases of reward and loss processing, but results to date have been inconsistent. It is unclear if anticipation and outcome of a reward or loss recruit similar or distinct brain regions. In particular, while the striatum has widely been found to be active when anticipating a reward, whether it activates in response to the anticipation of losses as well remains ambiguous. Furthermore, concerning the orbitofrontal/ventromedial prefrontal regions, activation is often observed during reward receipt. However, it is unclear if this area is active during reward anticipation as well. We ran an Activation Likelihood Estimation meta‐analysis of 50 fMRI studies, which used the Monetary Incentive Delay Task (MIDT), to identify which brain regions are implicated in the anticipation of rewards, anticipation of losses, and the receipt of reward. Anticipating rewards and losses recruits overlapping areas including the striatum, insula, amygdala and thalamus, suggesting that a generalised neural system initiates motivational processes independent of valence. The orbitofrontal/ventromedial prefrontal regions were recruited only during the reward outcome, likely representing the value of the reward received. Our findings help to clarify the neural substrates of the different phases of reward and loss processing, and advance neurobiological models of these processes.
Collapse
Affiliation(s)
- Stuart Oldham
- Brain and Mental Health Research Hub, School of Psychological Sciences and the Monash Institute of Cognitive and Clinical Neurosciences (MICCN), Monash University, Clayton, Victoria, Australia
| | - Carsten Murawski
- Department of Finance, The University of Melbourne, Parkville, Victoria, Australia
| | - Alex Fornito
- Brain and Mental Health Research Hub, School of Psychological Sciences and the Monash Institute of Cognitive and Clinical Neurosciences (MICCN), Monash University, Clayton, Victoria, Australia
| | - George Youssef
- Cognitive Neuroscience Unit, School of Psychology, Deakin University, Geelong, Australia.,Centre for Adolescent Health, Murdoch Children's Research Institute, Parkville, Australia
| | - Murat Yücel
- Brain and Mental Health Research Hub, School of Psychological Sciences and the Monash Institute of Cognitive and Clinical Neurosciences (MICCN), Monash University, Clayton, Victoria, Australia
| | - Valentina Lorenzetti
- Brain and Mental Health Research Hub, School of Psychological Sciences and the Monash Institute of Cognitive and Clinical Neurosciences (MICCN), Monash University, Clayton, Victoria, Australia.,School of Psychology, Faculty of Health Sciences, Australian Catholic University, Fitzroy, Victoria, Australia.,Department of Psychological Sciences, Institute of Psychology Health and Society, University of Liverpool, Liverpool, United Kingdom
| |
Collapse
|
14
|
The Caudate Nucleus Mediates Learning of Stimulus-Control State Associations. J Neurosci 2017; 37:1028-1038. [PMID: 28123033 DOI: 10.1523/jneurosci.0778-16.2016] [Citation(s) in RCA: 55] [Impact Index Per Article: 7.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2016] [Revised: 11/23/2016] [Accepted: 12/09/2016] [Indexed: 11/21/2022] Open
Abstract
A longstanding dichotomy in cognitive psychology and neuroscience pits controlled, top-down driven behavior against associative, bottom-up driven behavior, where cognitive control processes allow us to override well-learned stimulus-response (S-R) associations. By contrast, some previous studies have raised the intriguing possibility of an integration between associative and controlled processing in the form of stimulus-control state (S-C) associations, the learned linkage of specific stimuli to particular control states, such as high attentional selectivity. The neural machinery mediating S-C learning remains poorly understood, however. Here, we combined human functional magnetic resonance imaging (fMRI) with a previously developed Stroop protocol that allowed us to dissociate reductions in Stroop interference based on S-R learning from those based on S-C learning. We modeled subjects' acquisition of S-C and S-R associations using an associative learning model and then used trial-by-trial S-C and S-R prediction error (PE) estimates in model-based behavioral and fMRI analyses. We found that PE estimates derived from S-C and S-R associations accounted for the reductions in behavioral Stroop interference effects in the S-C and S-R learning conditions, respectively. Moreover, model-based fMRI analyses identified the caudate nucleus as the key structure involved in selectively updating stimulus-control state associations. Complementary analyses also revealed a greater reliance on parietal cortex when using the learned S-R versus S-C associations to minimize Stroop interference. These results support the emerging view that generalizable control states can become associated with specific bottom-up cues, and they place the caudate nucleus of the dorsal striatum at the center of the neural stimulus-control learning machinery. SIGNIFICANCE STATEMENT Previous behavioral studies have demonstrated that control states, for instance, heightened attentional selectivity, can become directly associated with, and subsequently retrieved by, particular stimuli, thus breaking down the traditional dichotomy between top-down and bottom-up driven behavior. However, the neural mechanisms underlying this type of stimulus-control learning remain poorly understood. We therefore combined noninvasive human neuroimaging with a task that allowed us to dissociate the acquisition of stimulus-control associations from that of stimulus-response associations. The results revealed the caudate nucleus as the key brain structure involved in selectively driving stimulus-control learning. These data represent the first identification of the neural mechanisms of stimulus-specific control associations, and they significantly extend current conceptions of the type of learning processes mediated by the caudate.
Collapse
|
15
|
Dobryakova E, Hulst HE, Spirou A, Chiaravalloti ND, Genova HM, Wylie GR, DeLuca J. Fronto-striatal network activation leads to less fatigue in multiple sclerosis. Mult Scler 2017. [PMID: 28627957 DOI: 10.1177/1352458517717087] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023]
Abstract
BACKGROUND The fronto-striatal network has been implicated in both fatigue, a common multiple sclerosis (MS) symptom, and goal attainment, which has been shown to reduce fatigue in healthy individuals. OBJECTIVES To investigate whether stimulation of the fronto-striatal network through goal attainment (potential monetary gain) leads to fatigue reduction in MS and healthy control (HC) participants. METHODS In all, 14 healthy and 19 MS participants performed a gambling task during functional magnetic resonance imaging (fMRI). Participants were presented with an opportunity to receive monetary reward during the outcome condition of the task but not during the no outcome condition. Self-reported fatigue measures were obtained after each condition and outside of the scanner. Structural alterations were also examined. RESULTS A significant decrease in fatigue was observed after the outcome condition compared to the no outcome condition in both groups. Significantly greater activation was observed in the ventral striatum in association with the outcome condition compared to the no outcome condition in both groups. Ventromedial prefrontal cortex showed significantly greater activation during the no outcome condition compared to the outcome condition with greater difference between conditions in the HC group. CONCLUSION This is the first functional neuroimaging study showing that stimulation of the fronto-striatal network through goal attainment leads to decreased on-task fatigue in MS and healthy participants.
Collapse
Affiliation(s)
- Ekaterina Dobryakova
- Traumatic Brain Injury Research, Kessler Foundation, East Hanover, NJ, USA; Traumatic Brain Injury Research, Kessler Foundation, West Orange, NJ, USA/Department of Physical Medicine and Rehabilitation, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Hanneke E Hulst
- Section of Clinical Neuroscience, Department of Anatomy & Neurosciences, Neuroscience Amsterdam, VUmc MS Center Amsterdam, VU University Medical Center, Amsterdam, The Netherlands
| | - Angela Spirou
- Traumatic Brain Injury Research, Kessler Foundation, West Orange, NJ, USA
| | - Nancy D Chiaravalloti
- Traumatic Brain Injury Research, Kessler Foundation, West Orange, NJ, USA/Neuropsychology & Neuroscience Laboratory, Kessler Foundation, West Orange, NJ, USA/Department of Physical Medicine and Rehabilitation, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Helen M Genova
- Neuropsychology and Neuroscience Laboratory, Kessler Foundation, West Orange, NJ, USA/Department of Physical Medicine and Rehabilitation, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Glenn R Wylie
- Neuropsychology and Neuroscience Laboratory, Kessler Foundation, West Orange, NJ, USA/Department of Physical Medicine and Rehabilitation, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - John DeLuca
- Neuropsychology and Neuroscience Laboratory, Kessler Foundation, West Orange, NJ, USA/Department of Physical Medicine and Rehabilitation, Rutgers New Jersey Medical School, Newark, NJ, USA
| |
Collapse
|
16
|
DePasque S, Galván A. Frontostriatal development and probabilistic reinforcement learning during adolescence. Neurobiol Learn Mem 2017; 143:1-7. [PMID: 28450078 DOI: 10.1016/j.nlm.2017.04.009] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2016] [Revised: 01/19/2017] [Accepted: 04/22/2017] [Indexed: 11/30/2022]
Abstract
Adolescence has traditionally been viewed as a period of vulnerability to increased risk-taking and adverse outcomes, which have been linked to neurobiological maturation of the frontostriatal reward system. However, growing research on the role of developmental changes in the adolescent frontostriatal system in facilitating learning will provide a more nuanced view of adolescence. In this review, we discuss the implications of existing research on this topic for learning during adolescence, and suggest that the very neural changes that render adolescents vulnerable to social pressure and risky decision making may also stand to play a role in scaffolding the ability to learn from rewards and from performance-related feedback.
Collapse
Affiliation(s)
- Samantha DePasque
- Department of Psychology, University of California, Los Angeles, 1285 Franz Hall, Box 951563, Los Angeles, CA, United States
| | - Adriana Galván
- Department of Psychology, University of California, Los Angeles, 1285 Franz Hall, Box 951563, Los Angeles, CA, United States.
| |
Collapse
|
17
|
Modulation of ventral striatal activity by cognitive effort. Neuroimage 2016; 147:330-338. [PMID: 27989778 DOI: 10.1016/j.neuroimage.2016.12.029] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2016] [Revised: 11/07/2016] [Accepted: 12/10/2016] [Indexed: 11/23/2022] Open
Abstract
Effort discounting theory suggests that the value of a reward should be lower if it was effortful to obtain, whereas contrast theory suggests that the contrast between the costly effort and the reward makes the reward seem more valuable. To test these alternative hypotheses, we used functional magnetic resonance imaging (fMRI) as participants engaged in feedback-based learning that required low or high cognitive effort to obtain positive feedback, while the objective amount of information provided by feedback remained constant. In the low effort condition, a single image was presented with four response options. In the high effort condition, two images were presented, each with two response options, and correct feedback was presented only when participants responded correctly to both of the images. Accuracy was significantly lower for the high effort condition, and all participants reported that the high effort condition was more difficult. A region of the ventral striatum selected for sensitivity to feedback value also showed increased activation to feedback presentation associated with the high effort condition relative to the low effort condition, when controlling for activation from corresponding control conditions where feedback was random. These results suggest that increased cognitive effort produces corresponding increases in positive feedback-related ventral striatum activity, in line with the predictions made by contrast theory. The accomplishment of obtaining a hard-earned intrinsic reward, such as positive feedback, may be particularly likely to promote reward-related brain activity.
Collapse
|
18
|
Tricomi E, DePasque S. The Role of Feedback in Learning and Motivation. ADVANCES IN MOTIVATION AND ACHIEVEMENT 2016. [DOI: 10.1108/s0749-742320160000019015] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
19
|
Lempert KM, Tricomi E. The Value of Being Wrong: Intermittent Feedback Delivery Alters the Striatal Response to Negative Feedback. J Cogn Neurosci 2015; 28:261-74. [PMID: 26439265 DOI: 10.1162/jocn_a_00892] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Whereas positive feedback is both rewarding and informative, negative feedback can be construed as either punishing (because it is indicative of poor performance) or informative (because it may lead to goal attainment). In this neuroimaging experiment, we highlighted the informational value of negative feedback by intermixing trials with and without feedback. When performance feedback is expected, positive feedback triggers an increase in striatal activity, whereas negative feedback elicits a decrease in striatal activity. We predicted that, in contrast, when feedback receipt is unpredictable, the striatal response to negative feedback would increase. Participants performed a paired-associate learning task during fMRI scanning. In one condition ("blocked feedback"), the receipt of feedback was predictable--participants knew whether or not they would receive feedback for their responses. In another condition ("mixed feedback"), the receipt of feedback was unpredictable--on a random 50% of trials, participants received feedback, and they otherwise received no feedback. Negative feedback in the mixed feedback condition elicited more striatal activity than negative feedback in the blocked feedback condition. In contrast, feedback omission evoked more striatal activity when feedback delivery was expected, compared to when it was unpredictable. This pattern emerged from an increase in caudate activity in response to negative feedback in the mixed feedback condition and a decrease in ventral striatal activity in response to no feedback in this condition. These results suggest that, by emphasizing the informational value of negative feedback, an unpredictable feedback context alters the striatal response to negative feedback and to the omission of feedback.
Collapse
|
20
|
DiMenichi BC, Tricomi E. The power of competition: Effects of social motivation on attention, sustained physical effort, and learning. Front Psychol 2015; 6:1282. [PMID: 26388801 PMCID: PMC4554955 DOI: 10.3389/fpsyg.2015.01282] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2015] [Accepted: 08/11/2015] [Indexed: 11/24/2022] Open
Abstract
Competition has often been implicated as a means to improve effort-based learning and attention. Two experiments examined the effects of competition on effort and memory. In Experiment 1, participants completed a physical effort task in which they were rewarded for winning an overall percentage, or for winning a competition they believed was against another player. In Experiment 2, participants completed a memory task in which they were rewarded for remembering an overall percentage of shapes, or more shapes than a “competitor.” We found that, in the physical effort task, participants demonstrated faster reaction times (RTs)—a previous indicator of increased attention—in the competitive environment. Moreover, individual differences predicted the salience of competition’s effect. Furthermore, male participants showed faster RTs and greater sustained effort as a result of a competitive environment, suggesting that males may be more affected by competition in physical effort tasks. However, in Experiment 2, participants remembered fewer shapes when competing, and later recalled less of these shapes during a post-test, suggesting that competition was harmful in our memory task. The different results from these two experiments suggest that competition can improve attention in a physical effort task, yet caution the use of competition in memory tasks.
Collapse
|
21
|
Effects of intrinsic motivation on feedback processing during learning. Neuroimage 2015; 119:175-86. [PMID: 26112370 DOI: 10.1016/j.neuroimage.2015.06.046] [Citation(s) in RCA: 31] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2015] [Revised: 06/14/2015] [Accepted: 06/15/2015] [Indexed: 10/23/2022] Open
Abstract
Learning commonly requires feedback about the consequences of one's actions, which can drive learners to modify their behavior. Motivation may determine how sensitive an individual might be to such feedback, particularly in educational contexts where some students value academic achievement more than others. Thus, motivation for a task might influence the value placed on performance feedback and how effectively it is used to improve learning. To investigate the interplay between intrinsic motivation and feedback processing, we used functional magnetic resonance imaging (fMRI) during feedback-based learning before and after a novel manipulation based on motivational interviewing, a technique for enhancing treatment motivation in mental health settings. Because of its role in the reinforcement learning system, the striatum is situated to play a significant role in the modulation of learning based on motivation. Consistent with this idea, motivation levels during the task were associated with sensitivity to positive versus negative feedback in the striatum. Additionally, heightened motivation following a brief motivational interview was associated with increases in feedback sensitivity in the left medial temporal lobe. Our results suggest that motivation modulates neural responses to performance-related feedback, and furthermore that changes in motivation facilitate processing in areas that support learning and memory.
Collapse
|
22
|
The role of prediction and outcomes in adaptive cognitive control. ACTA ACUST UNITED AC 2015; 109:38-52. [PMID: 25698177 DOI: 10.1016/j.jphysparis.2015.02.001] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2014] [Revised: 01/18/2015] [Accepted: 02/07/2015] [Indexed: 02/04/2023]
Abstract
Humans adaptively perform actions to achieve their goals. This flexible behaviour requires two core abilities: the ability to anticipate the outcomes of candidate actions and the ability to select and implement actions in a goal-directed manner. The ability to predict outcomes has been extensively researched in reinforcement learning paradigms, but this work has often focused on simple actions that are not embedded in hierarchical and sequential structures that are characteristic of goal-directed human behaviour. On the other hand, the ability to select actions in accordance with high-level task goals, particularly in the presence of alternative responses and salient distractors, has been widely researched in cognitive control paradigms. Cognitive control research, however, has often paid less attention to the role of action outcomes. The present review attempts to bridge these accounts by proposing an outcome-guided mechanism for selection of extended actions. Our proposal builds on constructs from the hierarchical reinforcement learning literature, which emphasises the concept of reaching and evaluating informative states, i.e., states that constitute subgoals in complex actions. We develop an account of the neural mechanisms that allow outcome-guided action selection to be achieved in a network that relies on projections from cortical areas to the basal ganglia and back-projections from the basal ganglia to the cortex. These cortico-basal ganglia-thalamo-cortical 'loops' allow convergence - and thus integration - of information from non-adjacent cortical areas (for example between sensory and motor representations). This integration is essential in action sequences, for which achieving an anticipated sensory state signals the successful completion of an action. We further describe how projection pathways within the basal ganglia allow selection between representations, which may pertain to movements, actions, or extended action plans. The model lastly envisages a role for hierarchical projections from the striatum to dopaminergic midbrain areas that enable more rostral frontal areas to bias the selection of inputs from more posterior frontal areas via their respective representations in the basal ganglia.
Collapse
|