1
|
Dubinsky JM, Hamid AA. The neuroscience of active learning and direct instruction. Neurosci Biobehav Rev 2024; 163:105737. [PMID: 38796122 DOI: 10.1016/j.neubiorev.2024.105737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 05/13/2024] [Accepted: 05/20/2024] [Indexed: 05/28/2024]
Abstract
Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.
Collapse
Affiliation(s)
- Janet M Dubinsky
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA.
| | - Arif A Hamid
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
2
|
Qasim SE, Deswal A, Saez I, Gu X. Positive affect modulates memory by regulating the influence of reward prediction errors. COMMUNICATIONS PSYCHOLOGY 2024; 2:52. [PMID: 39242805 PMCID: PMC11332028 DOI: 10.1038/s44271-024-00106-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 05/28/2024] [Indexed: 09/09/2024]
Abstract
How our decisions impact our memories is not well understood. Reward prediction errors (RPEs), the difference between expected and obtained reward, help us learn to make optimal decisions-providing a signal that may influence subsequent memory. To measure this influence and how it might go awry in mood disorders, we recruited a large cohort of human participants to perform a decision-making task in which perceptually memorable stimuli were associated with probabilistic rewards, followed by a recognition test for those stimuli. Computational modeling revealed that positive RPEs enhanced both the accuracy of memory and the temporal efficiency of memory search, beyond the contribution of perceptual information. Critically, positive affect upregulated the beneficial effect of RPEs on memory. These findings demonstrate how affect selectively regulates the impact of RPEs on memory, providing a computational mechanism for biased memory in mood disorders.
Collapse
Affiliation(s)
- Salman E Qasim
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | | | - Ignacio Saez
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Department of Neurosurgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Department of Neurology, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Xiaosi Gu
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
| |
Collapse
|
3
|
Blum K, Braverman ER, Gold MS, Dennen CA, Baron D, Thanos PK, Hanna C, Elman I, Gondre-Lewis MC, Ashford JW, Newberg A, Madigan MA, Jafari N, Zeine F, Sunder K, Giordano J, Barh D, Gupta A, Carney P, Bowirrat A, Badgaiyan RD. Addressing cortex dysregulation in youth through brain health check coaching and prophylactic brain development. INNOSC THERANOSTICS & PHARMACOLOGICAL SCIENCES 2024; 7:1472. [PMID: 38766548 PMCID: PMC11100020 DOI: 10.36922/itps.1472] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 05/22/2024]
Abstract
The Carter Center has estimated that the addiction crisis in the United States (US), if continues to worsen at the same rate, may cost the country approximately 16 trillion dollars by 2030. In recent years, the well-being of youth has been compromised by not only the coronavirus disease 2019 pandemic but also the alarming global opioid crisis, particularly in the US. Each year, deadly opioid drugs claim hundreds of thousands of lives, contributing to an ever-rising death toll. In addition, maternal usage of opioids and other drugs during pregnancy could compromise the neurodevelopment of children. A high rate of DNA polymorphic antecedents compounds the occurrence of epigenetic insults involving methylation of specific essential genes related to normal brain function. These genetic antecedent insults affect healthy DNA and mRNA transcription, leading to a loss of proteins required for normal brain development and function in youth. Myelination in the frontal cortex, a process known to extend until the late 20s, delays the development of proficient executive function and decision-making abilities. Understanding this delay in brain development, along with the presence of potential high-risk antecedent polymorphic variants or alleles and generational epigenetics, provides a clear rationale for embracing the Brain Research Commission's suggestion to mimic fitness programs with an adaptable brain health check (BHC). Implementing the BHC within the educational systems in the US and other countries could serve as an effective initiative for proactive therapies aimed at reducing juvenile mental health problems and eventually criminal activities, addiction, and other behaviors associated with reward deficiency syndrome.
Collapse
Affiliation(s)
- Kenneth Blum
- Division of Addiction Research and Education, Center for Sports, Exercise and Global Mental Health, Western University of Health Sciences, Pomona, California, United States of America
- The Kenneth Blum Behavioral and Neurogenetic Institute LLC, Austin, Texas, United States of America
- Faculty of Education and Psychology, Institute of Psychology, Eötvös Loránd University Budapest, Budapest, Hungary
- Department of Molecular Biology and Adelson School of Medicine, Ariel University, Ariel, Israel
- Division of Personalized Medicine, Cross-Cultural Research and Educational Institute, San Clemente, California, United States of America
- Centre for Genomics and Applied Gene Technology, Institute of Integrative Omics and Applied Biotechnology, Purba Medinipur, West Bengal, India
- Division of Personalized Recovery Science, Transplicegen Therapeutics, Llc., Austin, Tx., United of States
- Department of Psychiatry, University of Vermont, Burlington, Vermont, United States of America
- Department of Psychiatry, Boonshoft School of Medicine, Wright State University, Dayton, Ohio, United States of America
- Division of Personalized Medicine, Ketamine Clinic of South Florida, Pompano Beach, Florida, United States of America
| | - Eric R. Braverman
- The Kenneth Blum Behavioral and Neurogenetic Institute LLC, Austin, Texas, United States of America
| | - Mark S. Gold
- Department of Psychiatry, Washington University School of Medicine, St. Louis, Missouri, United States of America
| | - Catherine A. Dennen
- Department of Family Medicine, Jefferson Health Northeast, Philadelphia, Pennsylvania, United States of America
| | - David Baron
- Division of Addiction Research and Education, Center for Sports, Exercise and Global Mental Health, Western University of Health Sciences, Pomona, California, United States of America
| | - Panayotis K. Thanos
- Department of Psychology and Behavioral Neuropharmacology and Neuroimaging Laboratory on Addictions, Research Institute on Addictions, University of Buffalo, Buffalo, New York, United States of America
| | - Colin Hanna
- Department of Psychology and Behavioral Neuropharmacology and Neuroimaging Laboratory on Addictions, Research Institute on Addictions, University of Buffalo, Buffalo, New York, United States of America
| | - Igor Elman
- Cambridge Health Alliance, Harvard Medical School, Cambridge, Massachusetts, United States of America
| | - Marjorie C. Gondre-Lewis
- Department of Anatomy, Howard University School of Medicine, Washington, D.C., United States of America
| | - J. Wesson Ashford
- Department of Psychiatry and Behavioral Sciences, Stanford University, Palo Alto, California, United States of America
| | - Andrew Newberg
- Department of Integrative Medicine and Nutritional Sciences, Thomas Jefferson University and Hospital, Philadelphia, Pennsylvania, United States of America
| | - Margaret A. Madigan
- The Kenneth Blum Behavioral and Neurogenetic Institute LLC, Austin, Texas, United States of America
| | - Nicole Jafari
- Division of Personalized Medicine, Cross-Cultural Research and Educational Institute, San Clemente, California, United States of America
- Department of Human Development, California State University at Long Beach, Long Beach, California, United States of America
| | - Foojan Zeine
- Department of Human Development, California State University at Long Beach, Long Beach, California, United States of America
- Awareness Integration Institute, San Clemente, California, United States of America
| | - Keerthy Sunder
- Department of Health Science, California State University at Long Beach, Long Beach, California, United States of America
- Department of Psychiatry, University California, UC Riverside School of Medicine, Riverside, California, United States of America
| | - John Giordano
- Division of Personalized Medicine, Ketamine Clinic of South Florida, Pompano Beach, Florida, United States of America
| | - Debmayla Barh
- Centre for Genomics and Applied Gene Technology, Institute of Integrative Omics and Applied Biotechnology, Purba Medinipur, West Bengal, India
| | - Ashim Gupta
- Future Biologics, Lawrenceville, Georgia, United States of America
| | - Paul Carney
- Division of Pediatric Neurology, University of Missouri Health Care-Columbia, Columbia, Missouri, United States of America
| | - Abdalla Bowirrat
- Department of Molecular Biology and Adelson School of Medicine, Ariel University, Ariel, Israel
| | - Rajendra D. Badgaiyan
- Department of Psychiatry, Mt. Sinai School of Medicine, New York City, New York, United States of America
| |
Collapse
|
4
|
Reicher V, Kovács T, Csibra B, Gácsi M. Potential interactive effect of positive expectancy violation and sleep on memory consolidation in dogs. Sci Rep 2024; 14:9487. [PMID: 38664506 PMCID: PMC11045790 DOI: 10.1038/s41598-024-60166-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2023] [Accepted: 04/19/2024] [Indexed: 04/28/2024] Open
Abstract
In dogs, as in humans, both emotional and learning pretreatment affect subsequent behaviour and sleep. Although learning often occurs in an emotional-social context, the emotion-learning interplay in such context remain mainly unknown. Aims were to assess the effects of Controlling versus Permissive (emotional factors) training (learning factors) styles on dogs' behaviour, learning performance, and sleep. Family dogs (N = 24) participated in two command learning sessions employing the two training styles with each session followed by assessment of learning performance, a 2-h-long non-invasive sleep EEG measurement, and a retest of learning performance. Pre- to post-sleep improvement in learning performance was evident in dogs that received the Permissive training during the second learning session, indicating that dogs that experienced a more rewarding situation than expected (positive expectancy violation) during the second training session showed improved learning success after their afternoon sleep. These results possibly indicate an interactive effect of expectancy violation and sleep on enhancing learning.
Collapse
Affiliation(s)
- Vivien Reicher
- Clinical and Developmental Neuropsychology Research Group, HUN-REN Research Centre for Natural Sciences, Budapest, Hungary.
- Department of Ethology, Eötvös Loránd University, Budapest, Hungary.
| | - Tímea Kovács
- Department of Ethology, Eötvös Loránd University, Budapest, Hungary
- Doctoral School of Biology, ELTE Eötvös Loránd University, Budapest, Hungary
| | - Barbara Csibra
- Department of Ethology, Eötvös Loránd University, Budapest, Hungary
- Doctoral School of Biology, ELTE Eötvös Loránd University, Budapest, Hungary
| | - Márta Gácsi
- Department of Ethology, Eötvös Loránd University, Budapest, Hungary
- HUN-REN-ELTE Comparative Ethology Research Group, Budapest, Hungary
| |
Collapse
|
5
|
Antony JW, Van Dam J, Massey JR, Barnett AJ, Bennion KA. Long-term, multi-event surprise correlates with enhanced autobiographical memory. Nat Hum Behav 2023; 7:2152-2168. [PMID: 37322234 DOI: 10.1038/s41562-023-01631-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Accepted: 05/16/2023] [Indexed: 06/17/2023]
Abstract
Neurobiological and psychological models of learning emphasize the importance of prediction errors (surprises) for memory formation. This relationship has been shown for individual momentary surprising events; however, it is less clear whether surprise that unfolds across multiple events and timescales is also linked with better memory of those events. We asked basketball fans about their most positive and negative autobiographical memories of individual plays, games and seasons, allowing surprise measurements spanning seconds, hours and months. We used advanced analytics on National Basketball Association play-by-play data and betting odds spanning 17 seasons, more than 22,000 games and more than 5.6 million plays to compute and align the estimated surprise value of each memory. We found that surprising events were associated with better recall of positive memories on the scale of seconds and months and negative memories across all three timescales. Game and season memories could not be explained by surprise at shorter timescales, suggesting that long-term, multi-event surprise correlates with memory. These results expand notions of surprise in models of learning and reinforce its relevance in real-world domains.
Collapse
Affiliation(s)
- James W Antony
- Department of Psychology and Child Development, California Polytechnic State University, San Luis Obispo, CA, USA.
| | - Jacob Van Dam
- Department of Psychology and Child Development, California Polytechnic State University, San Luis Obispo, CA, USA
| | - Jarett R Massey
- Department of Psychology and Child Development, California Polytechnic State University, San Luis Obispo, CA, USA
| | | | - Kelly A Bennion
- Department of Psychology and Child Development, California Polytechnic State University, San Luis Obispo, CA, USA
| |
Collapse
|
6
|
Zhang T, Zhou S, Bai X, Zhou F, Zhai Y, Long Y, Lu C. Neurocomputations on dual-brain signals underlie interpersonal prediction during a natural conversation. Neuroimage 2023; 282:120400. [PMID: 37783363 DOI: 10.1016/j.neuroimage.2023.120400] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 09/21/2023] [Accepted: 09/30/2023] [Indexed: 10/04/2023] Open
Abstract
Prediction on the partner's speech plays a key role in a smooth conversation. However, previous studies on this issue have been majorly conducted at the single-brain rather than dual-brain level, leaving the interpersonal prediction hypothesis untested. To fill this gap, this study combined a neurocomputational modeling approach with a natural conversation paradigm in which two salespersons persuaded a customer to buy their product with their haemodynamic signals being collected using functional near-infrared spectroscopy hyperscanning. First, the results showed a cognitive hierarchy in a natural conversation, with the lower-level process (i.e., pragmatic representation of the persuasion) in the salesperson interacting with the higher-level process (i.e., value representation of the product) in the customer. Next, we found that the right dorsal lateral prefrontal cortex (rdlPFC) and temporoparietal junction (rTPJ) were associated with the representation of the product's value in the customer, while the right inferior frontal cortex (rIFC) was associated with the representation of the pragmatic processes in the salesperson. Finally, neurocomputational modeling results supported the prediction of the salesperson's lower-level brain activity based on the customer's higher-level brain activity. Moreover, the updating weight of the prediction model based on the neural computation between the rIFC of the salesperson and the rTPJ of the customer was closely associated with the interaction context, whereas that based on the rIFC-rdlPFC was not. In summary, these findings provide initial support for the interpersonal prediction hypothesis at the dual-brain level and reveal a hierarchy for the interpersonal prediction process.
Collapse
Affiliation(s)
- Tengfei Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Siyuan Zhou
- Institute of Brain and Psychological Sciences, Sichuan Normal University, Chengdu 610066, PR China
| | - Xialu Bai
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Faxin Zhou
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Yu Zhai
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Yuhang Long
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Chunming Lu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China.
| |
Collapse
|
7
|
Xue J, Jiang T, Chen C, Murty VP, Li Y, Ding Z, Zhang M. The interactive effect of external rewards and self-determined choice on memory. PSYCHOLOGICAL RESEARCH 2023; 87:2101-2110. [PMID: 36869894 PMCID: PMC9984743 DOI: 10.1007/s00426-023-01807-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Accepted: 02/10/2023] [Indexed: 03/05/2023]
Abstract
Both external motivational incentives (e.g., monetary reward) and internal motivational incentives (e.g., self-determined choice) have been found to promote memory, but much less is known about how these two types of incentives interact with each other to affect memory. The current study (N = 108) examined how performance-dependent monetary rewards affected the role of self-determined choice in memory performance, also known as the choice effect. Using a modified and better controlled version of the choice paradigm and manipulating levels of reward, we demonstrated an interactive effect between monetary reward and self-determined choice on 1-day delayed memory performance. Specifically, the choice effect on memory decreased when we introduced the performance-dependent external rewards. These results are discussed in terms of understanding how external and internal motivators interact to impact learning and memory.
Collapse
Affiliation(s)
- Jingming Xue
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Beijing, 100101, People's Republic of China
- Faculty of Psychology, Beijing Normal University, Beijing, 100875, People's Republic of China
| | - Ting Jiang
- Faculty of Psychology, Beijing Normal University, Beijing, 100875, People's Republic of China
| | - Chuansheng Chen
- Department of Psychological Science, University of California, Irvine, CA, 92697, USA
| | - Vishnu P Murty
- Department of Psychology, Temple University, Philadelphia, PA, 19122, USA
| | - Yuxin Li
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Beijing, 100101, People's Republic of China
- Department of Psychology, University of Chinese Academy of Sciences, Beijing, 100049, People's Republic of China
| | - Zhuolei Ding
- Faculty of Psychology, Beijing Normal University, Beijing, 100875, People's Republic of China
| | - Mingxia Zhang
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Beijing, 100101, People's Republic of China.
- Department of Psychology, University of Chinese Academy of Sciences, Beijing, 100049, People's Republic of China.
- Institute of Psychology, Chinese Academy of Sciences, 16 Lincui Rd., Beijing, 100101, People's Republic of China.
| |
Collapse
|
8
|
Rouhani N, Niv Y, Frank MJ, Schwabe L. Multiple routes to enhanced memory for emotionally relevant events. Trends Cogn Sci 2023; 27:867-882. [PMID: 37479601 DOI: 10.1016/j.tics.2023.06.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Revised: 06/13/2023] [Accepted: 06/15/2023] [Indexed: 07/23/2023]
Abstract
Events associated with aversive or rewarding outcomes are prioritized in memory. This memory boost is commonly attributed to the elicited affective response, closely linked to noradrenergic and dopaminergic modulation of hippocampal plasticity. Herein we review and compare this 'affect' mechanism to an additional, recently discovered, 'prediction' mechanism whereby memories are strengthened by the extent to which outcomes deviate from expectations, that is, by prediction errors (PEs). The mnemonic impact of PEs is separate from the affective outcome itself and has a distinct neural signature. While both routes enhance memory, these mechanisms are linked to different - and sometimes opposing - predictions for memory integration. We discuss new findings that highlight mechanisms by which emotional events strengthen, integrate, and segment memory.
Collapse
Affiliation(s)
- Nina Rouhani
- Division of Biology and Biological Engineering and Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - Yael Niv
- Department of Psychology and Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Michael J Frank
- Department of Cognitive, Linguistic & Psychological Sciences and Carney Institute for Brain Science, Brown University, Providence, RI, USA
| | - Lars Schwabe
- Department of Cognitive Psychology, Institute of Psychology, Universität Hamburg, Hamburg, Germany.
| |
Collapse
|
9
|
Schultz H, Stoffregen H, Benoit RG. A reward effect on memory retention, consolidation, and generalization? Learn Mem 2023; 30:169-174. [PMID: 37679044 PMCID: PMC10519399 DOI: 10.1101/lm.053842.123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Accepted: 08/09/2023] [Indexed: 09/09/2023]
Abstract
Reward improves memory through both encoding and consolidation processes. In this preregistered study, we tested whether reward effects on memory generalize from high-rewarded items to low-rewarded but episodically related items. Fifty-nine human volunteers incidentally encoded associations between unique objects and repeated scenes. Some scenes typically yielded high reward, whereas others typically yielded low reward. Memory was tested immediately after encoding (n = 29) or the next day (n = 30). Overall, reward had only a limited influence on memory. It did not enhance consolidation and its effect did not generalize to episodically related stimuli. We thus contribute to understanding the boundary conditions of reward effects on memory.
Collapse
Affiliation(s)
- Heidrun Schultz
- Max Planck Research Group: Adaptive Memory, Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany
| | - Hanna Stoffregen
- Max Planck Research Group: Adaptive Memory, Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany
- International Max Planck Research School NeuroCom, 04103 Leipzig, Germany
| | - Roland G Benoit
- Max Planck Research Group: Adaptive Memory, Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany
- Department of Psychology and Neuroscience, University of Colorado Boulder, Boulder, Colorado 80309, USA
- Institute of Cognitive Science, University of Colorado Boulder, Boulder, Colorado 80309, USA
| |
Collapse
|
10
|
Rouhani N, Stanley D, Adolphs R. Collective events and individual affect shape autobiographical memory. Proc Natl Acad Sci U S A 2023; 120:e2221919120. [PMID: 37432994 PMCID: PMC10629560 DOI: 10.1073/pnas.2221919120] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2022] [Accepted: 06/06/2023] [Indexed: 07/13/2023] Open
Abstract
How do collective events shape how we remember our lives? We leveraged advances in natural language processing as well as a rich, longitudinal assessment of 1,000 Americans throughout 2020 to examine how memory is influenced by two prominent factors: surprise and emotion. Autobiographical memory for 2020 displayed a unique signature: There was a substantial bump in March, aligning with pandemic onset and lockdowns, consistent across three memory collections 1 y apart. We further investigated how emotion, using both immediate and retrieved measures, predicted the amount and content of autobiographical memory: Negative affect increased recall across all measures, whereas its more clinical indices, depression and posttraumatic stress disorder, selectively increased nonepisodic recall. Finally, in a separate cohort, we found pandemic news to be better remembered, surprising, and negative, while lockdowns compressed remembered time. Our work connects laboratory findings to the real world and delineates the effects of acute versus clinical signatures of negative emotion on memory.
Collapse
Affiliation(s)
- Nina Rouhani
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA91125
- Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA91125
| | - Damian Stanley
- Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA91125
- Derner School of Psychology, Adelphi University, New York, NY11530
| | | | - Ralph Adolphs
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA91125
- Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA91125
| |
Collapse
|
11
|
Pupillo F, Ortiz-Tudela J, Bruckner R, Shing YL. The effect of prediction error on episodic memory encoding is modulated by the outcome of the predictions. NPJ SCIENCE OF LEARNING 2023; 8:18. [PMID: 37248232 DOI: 10.1038/s41539-023-00166-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Accepted: 05/05/2023] [Indexed: 05/31/2023]
Abstract
Expectations can lead to prediction errors of varying degrees depending on the extent to which the information encountered in the environment conforms with prior knowledge. While there is strong evidence on the computationally specific effects of such prediction errors on learning, relatively less evidence is available regarding their effects on episodic memory. Here, we had participants work on a task in which they learned context/object-category associations of different strengths based on the outcomes of their predictions. We then used a reinforcement learning model to derive subject-specific trial-to-trial estimates of prediction error at encoding and link it to subsequent recognition memory. Results showed that model-derived prediction errors at encoding influenced subsequent memory as a function of the outcome of participants' predictions (correct vs. incorrect). When participants correctly predicted the object category, stronger prediction errors (as a consequence of weak expectations) led to enhanced memory. In contrast, when participants incorrectly predicted the object category, stronger prediction errors (as a consequence of strong expectations) led to impaired memory. These results highlight the important moderating role of choice outcome that may be related to interactions between the hippocampal and striatal dopaminergic systems.
Collapse
Affiliation(s)
- Francesco Pupillo
- Department of Psychology, Goethe University Frankfurt, Frankfurt, Germany.
- TS Social and Behavioral Sciences, Tilburg University, Tilburg, Netherlands.
| | | | - Rasmus Bruckner
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Berlin, Germany
| | - Yee Lee Shing
- Department of Psychology, Goethe University Frankfurt, Frankfurt, Germany
| |
Collapse
|
12
|
Wu X, Packard PA, García-Arch J, Bunzeck N, Fuentemilla L. Contextual incongruency triggers memory reinstatement and the disruption of neural stability. Neuroimage 2023; 273:120114. [PMID: 37080120 DOI: 10.1016/j.neuroimage.2023.120114] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Revised: 03/13/2023] [Accepted: 04/13/2023] [Indexed: 04/22/2023] Open
Abstract
Schemas, or internal representation models of the environment, are thought to be central in organising our everyday life behaviour by giving stability and predictiveness to the structure of the world. However, when an element from an unfolding event mismatches the schema-derived expectations, the coherent narrative is interrupted and an update to the current event model representation is required. Here, we asked whether the perceived incongruence of an item from an unfolding event and its impact on memory relied on the disruption of neural stability patterns preceded by the neural reactivation of the memory representations of the just-encoded event. Our study includes data from two different experiments whereby human participants (N = 33, 26 females and N = 18, 16 females, respectively) encoded images of objects preceded by trial-unique sequences of events depicting daily routine. We found that neural stability patterns gradually increased throughout the ongoing exposure to a schema-consistent episode, which was corroborated by the re-analysis of data from two other experiments, and that the brain stability pattern was interrupted when the encoding of an object of the event was incongruent with the ongoing schema. We found that the decrease in neural stability for low-congruence items was seen at ∼1000 ms from object encoding onset and that it was preceded by an enhanced N400 ERP and an increased degree of neural reactivation of the just-encoded episode. Current results offer new insights into the neural mechanisms and their temporal orchestration that are engaged during online encoding of schema-consistent episodic narratives and the detection of incongruencies.
Collapse
Affiliation(s)
- Xiongbo Wu
- Cognition and Brain Plasticity Group, Bellvitge Institute for Biomedical Research, Hospitalet de Llobregat 08907, Spain; Department of Cognition, Development and Educational Psychology, University of Barcelona, Barcelona 08035, Spain; Institute of Neurosciences, University of Barcelona, Barcelona 08035, Spain.
| | - Pau A Packard
- Multisensory Research Group, Center for Brain and Cognition, Pompeu Fabra University, Barcelona, Spain
| | - Josué García-Arch
- Cognition and Brain Plasticity Group, Bellvitge Institute for Biomedical Research, Hospitalet de Llobregat 08907, Spain; Department of Cognition, Development and Educational Psychology, University of Barcelona, Barcelona 08035, Spain; Institute of Neurosciences, University of Barcelona, Barcelona 08035, Spain
| | - Nico Bunzeck
- Department of Psychology, University of Lübeck, Lübeck 23562, Germany; Center of Brain, Behavior and Metabolism (CBBM), University of Lübeck, Lübeck 23562, Germany
| | - Lluís Fuentemilla
- Cognition and Brain Plasticity Group, Bellvitge Institute for Biomedical Research, Hospitalet de Llobregat 08907, Spain; Department of Cognition, Development and Educational Psychology, University of Barcelona, Barcelona 08035, Spain; Institute of Neurosciences, University of Barcelona, Barcelona 08035, Spain
| |
Collapse
|
13
|
Momennejad I. A rubric for human-like agents and NeuroAI. Philos Trans R Soc Lond B Biol Sci 2023; 378:20210446. [PMID: 36511409 PMCID: PMC9745874 DOI: 10.1098/rstb.2021.0446] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2022] [Accepted: 10/27/2022] [Indexed: 12/15/2022] Open
Abstract
Researchers across cognitive, neuro- and computer sciences increasingly reference 'human-like' artificial intelligence and 'neuroAI'. However, the scope and use of the terms are often inconsistent. Contributed research ranges widely from mimicking behaviour, to testing machine learning methods as neurally plausible hypotheses at the cellular or functional levels, or solving engineering problems. However, it cannot be assumed nor expected that progress on one of these three goals will automatically translate to progress in others. Here, a simple rubric is proposed to clarify the scope of individual contributions, grounded in their commitments to human-like behaviour, neural plausibility or benchmark/engineering/computer science goals. This is clarified using examples of weak and strong neuroAI and human-like agents, and discussing the generative, corroborate and corrective ways in which the three dimensions interact with one another. The author maintains that future progress in artificial intelligence will need strong interactions across the disciplines, with iterative feedback loops and meticulous validity tests-leading to both known and yet-unknown advances that may span decades to come. This article is part of a discussion meeting issue 'New approaches to 3D vision'.
Collapse
Affiliation(s)
- Ida Momennejad
- Microsoft Research NYC, Reinforcement Learning Station, 300 Lafayette, New York, NY 10012, USA
| |
Collapse
|
14
|
Hegedüs P, Sviatkó K, Király B, Martínez-Bellver S, Hangya B. Cholinergic activity reflects reward expectations and predicts behavioral responses. iScience 2022; 26:105814. [PMID: 36636356 PMCID: PMC9830220 DOI: 10.1016/j.isci.2022.105814] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Revised: 10/22/2022] [Accepted: 12/12/2022] [Indexed: 12/23/2022] Open
Abstract
Basal forebrain cholinergic neurons (BFCNs) play an important role in associative learning, suggesting that BFCNs may participate in processing stimuli that predict future outcomes. However, the impact of outcome probabilities on BFCN activity remained elusive. Therefore, we performed bulk calcium imaging and recorded spiking of identified cholinergic neurons from the basal forebrain of mice performing a probabilistic Pavlovian cued outcome task. BFCNs responded more to sensory cues that were often paired with reward. Reward delivery also activated BFCNs, with surprising rewards eliciting a stronger response, whereas punishments evoked uniform positive-going responses. We propose that BFCNs differentially weigh predictions of positive and negative reinforcement, reflecting divergent relative salience of forecasting appetitive and aversive outcomes, partially explained by a simple reinforcement learning model of a valence-weighed unsigned prediction error. Finally, the extent of cue-driven cholinergic activation predicted subsequent decision speed, suggesting that the expectation-gated cholinergic firing is instructive to reward-seeking behaviors.
Collapse
Affiliation(s)
- Panna Hegedüs
- Lendület Laboratory of Systems Neuroscience, Institute of Experimental Medicine, H-1083 Budapest, Hungary,János Szentágothai Doctoral School of Neurosciences, Semmelweis University, H-1085 Budapest, Hungary
| | - Katalin Sviatkó
- Lendület Laboratory of Systems Neuroscience, Institute of Experimental Medicine, H-1083 Budapest, Hungary,János Szentágothai Doctoral School of Neurosciences, Semmelweis University, H-1085 Budapest, Hungary
| | - Bálint Király
- Lendület Laboratory of Systems Neuroscience, Institute of Experimental Medicine, H-1083 Budapest, Hungary,Department of Biological Physics, Eötvös Loránd University, H-1117 Budapest, Hungary
| | - Sergio Martínez-Bellver
- Lendület Laboratory of Systems Neuroscience, Institute of Experimental Medicine, H-1083 Budapest, Hungary,Department of Anatomy and Human Embryology, Faculty of Medicine and Odontology, University of Valencia, 46010 Valencia, Spain
| | - Balázs Hangya
- Lendület Laboratory of Systems Neuroscience, Institute of Experimental Medicine, H-1083 Budapest, Hungary,Corresponding author
| |
Collapse
|
15
|
Elliott BL, D'Ardenne K, Murty VP, Brewer GA, McClure SM. Midbrain-Hippocampus Structural Connectivity Selectively Predicts Motivated Memory Encoding. J Neurosci 2022; 42:9426-9434. [PMID: 36332978 PMCID: PMC9794367 DOI: 10.1523/jneurosci.0945-22.2022] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2022] [Revised: 09/22/2022] [Accepted: 10/26/2022] [Indexed: 11/06/2022] Open
Abstract
Motivation is a powerful driver of learning and memory. Functional MRI studies show that interactions among the dopaminergic midbrain substantia nigra/ventral tegmental area (SN/VTA), hippocampus, and nucleus accumbens (NAc) are critical for motivated memory encoding. However, it is not known whether these effects are transient and purely functional, or whether individual differences in the structure of this circuit underlie motivated memory encoding. To quantify individual differences in structure, diffusion-weighted MRI and probabilistic tractography were used to quantify SN/VTA-striatum and SN/VTA-hippocampus pathways associated with motivated memory encoding in humans. Male and female participants completed a motivated source memory paradigm. During encoding, words were randomly assigned to one of three conditions, reward ($1.00), control ($0.00), or punishment (-$1.00). During retrieval, participants were asked to retrieve item and source information of the previously studied words and were rewarded or penalized according to their performance. Source memory for words assigned to both reward and punishment conditions was greater than those for control words, but there were no differences in item memory based on value. Anatomically, probabilistic tractography results revealed a heterogeneous, topological arrangement of the SN/VTA. Tract density measures of SN/VTA-hippocampus pathways were positively correlated with individual differences in reward-and-punishment-modulated memory performance, whereas density of SN/VTA-striatum pathways showed no association. This novel finding suggests that pathways emerging from the human SV/VTA are anatomically separable and functionally heterogeneous. Individual differences in structural connectivity of the dopaminergic hippocampus-VTA loop are selectively associated with motivated memory encoding.SIGNIFICANCE STATEMENT Functional MRI studies show that interactions among the SN/VTA, hippocampus, and NAc are critical for motivated memory encoding. This has led to competing theories that posit either SN/VTA-NAc reward prediction errors or SN/VTA-hippocampus signals underlie motivated memory encoding. Additionally, it is not known whether these effects are transient and purely functional or whether individual differences in the structure of these circuits underlie motivated memory encoding. Using diffusion-weighted MRI and probabilistic tractography, we show that tract density measures of SN/VTA-hippocampus pathways are positively correlated with motivated memory performance, whereas density of SN/VTA-striatum pathways show no association. This finding suggests that anatomic individual differences of the dopaminergic hippocampus-VTA loop are selectively associated with motivated memory encoding.
Collapse
Affiliation(s)
- Blake L Elliott
- Department of Psychology and Neuroscience, Temple University, Philadelphia, Pennsylvania 19122
| | | | - Vishnu P Murty
- Department of Psychology and Neuroscience, Temple University, Philadelphia, Pennsylvania 19122
| | - Gene A Brewer
- Department of Psychology, Arizona State University, Tempe, Arizona 85721
| | - Samuel M McClure
- Department of Psychology, Arizona State University, Tempe, Arizona 85721
| |
Collapse
|
16
|
Schultz H, Yoo J, Meshi D, Heekeren HR. Category-specific memory encoding in the medial temporal lobe and beyond: the role of reward. Learn Mem 2022; 29:379-389. [PMID: 36180131 PMCID: PMC9536755 DOI: 10.1101/lm.053558.121] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2021] [Accepted: 07/28/2022] [Indexed: 12/15/2022]
Abstract
The medial temporal lobe (MTL), including the hippocampus (HC), perirhinal cortex (PRC), and parahippocampal cortex (PHC), is central to memory formation. Reward enhances memory through interplay between the HC and substantia nigra/ventral tegmental area (SNVTA). While the SNVTA also innervates the MTL cortex and amygdala (AMY), their role in reward-enhanced memory is unclear. Prior research suggests category specificity in the MTL cortex, with the PRC and PHC processing object and scene memory, respectively. It is unknown, however, whether reward modulates category-specific memory processes. Furthermore, no study has demonstrated clear category specificity in the MTL for encoding processes contributing to subsequent recognition memory. To address these questions, we had 39 healthy volunteers (27 for all memory-based analyses) undergo functional magnetic resonance imaging while performing an incidental encoding task pairing objects or scenes with high or low reward, followed by a next-day recognition test. Behaviorally, high reward preferably enhanced object memory. Neural activity in the PRC and PHC reflected successful encoding of objects and scenes, respectively. Importantly, AMY encoding effects were selective for high-reward objects, with a similar pattern in the PRC. The SNVTA and HC showed no clear evidence of successful encoding. This behavioral and neural asymmetry may be conveyed through an anterior-temporal memory system, including the AMY and PRC, potentially in interplay with the ventromedial prefrontal cortex.
Collapse
Affiliation(s)
- Heidrun Schultz
- Department of Education and Psychology, Freie Universität Berlin, 14195 Berlin, Germany
- Center for Cognitive Neuroscience Berlin, Freie Universität Berlin, 14195 Berlin, Germany
- Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany
| | - Jungsun Yoo
- Department of Education and Psychology, Freie Universität Berlin, 14195 Berlin, Germany
- Center for Cognitive Neuroscience Berlin, Freie Universität Berlin, 14195 Berlin, Germany
- Department of Cognitive Sciences, University of California at Irvine, Irvine, California 92697, USA
| | - Dar Meshi
- Department of Education and Psychology, Freie Universität Berlin, 14195 Berlin, Germany
- Center for Cognitive Neuroscience Berlin, Freie Universität Berlin, 14195 Berlin, Germany
- Department of Advertising and Public Relations, Michigan State University, East Lansing, Michigan 48824, USA
| | - Hauke R Heekeren
- Department of Education and Psychology, Freie Universität Berlin, 14195 Berlin, Germany
- Center for Cognitive Neuroscience Berlin, Freie Universität Berlin, 14195 Berlin, Germany
- Executive University Board, Universität Hamburg, 20148 Hamburg, Germany
| |
Collapse
|
17
|
Patel SP, McCurdy MP, Frankenstein AN, Sklenar AM, Urban Levy P, Szpunar KK, Leshikar ED. The reciprocal relationship between episodic memory and future thinking: How the outcome of predictions is subsequently remembered. Brain Behav 2022; 12:e2603. [PMID: 36000544 PMCID: PMC9480898 DOI: 10.1002/brb3.2603] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/24/2021] [Revised: 02/10/2022] [Accepted: 02/14/2022] [Indexed: 11/09/2022] Open
Abstract
Evidence suggests that memory is involved in making simulations and predictions about the future (i.e., future thinking), but less work has examined how the outcome of those predictions (whether events play out as predicted or expected) subsequently affects episodic memory. In this investigation, we examine whether memory is better for outcomes that are consistent with predictions, or whether memory is enhanced for outcomes that are inconsistent with predictions, after the predicted event occurs. In this experiment, participants learned a core trait associated with social targets (e.g., high in extroversion), before making predictions about behaviors targets would perform. Participants then were shown behaviors the social targets actually performed (i.e., prediction outcome), which was either consistent or inconsistent with predictions. After that, participants completed a memory test (recognition; recall) for the prediction outcomes. For recognition, the results revealed better memory for outcomes that were consistent with traits associated with targets (i.e., trait-consistent outcomes), compared to outcomes that were inconsistent (i.e., trait-inconsistent outcomes). Finding a memory advantage for trait-consistent outcomes suggests that outcomes that are in line with the contents of memory (e.g., what one knows; schemas) are more readily remembered than those that are inconsistent with memory, which may reflect an adaptive memory process. For recall, memory did not differ between trait-consistent and trait-inconsistent outcomes. Altogether, the results of this experiment advance understanding of the reciprocal relationship between episodic memory and future thinking and show that outcome of predictions has an influence on subsequent episodic memory, at least as measured by recognition.
Collapse
|
18
|
Fountas Z, Sylaidi A, Nikiforou K, Seth AK, Shanahan M, Roseboom W. A Predictive Processing Model of Episodic Memory and Time Perception. Neural Comput 2022; 34:1501-1544. [PMID: 35671462 DOI: 10.1162/neco_a_01514] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Accepted: 03/06/2022] [Indexed: 11/04/2022]
Abstract
Human perception and experience of time are strongly influenced by ongoing stimulation, memory of past experiences, and required task context. When paying attention to time, time experience seems to expand; when distracted, it seems to contract. When considering time based on memory, the experience may be different than what is in the moment, exemplified by sayings like "time flies when you're having fun." Experience of time also depends on the content of perceptual experience-rapidly changing or complex perceptual scenes seem longer in duration than less dynamic ones. The complexity of interactions among attention, memory, and perceptual stimulation is a likely reason that an overarching theory of time perception has been difficult to achieve. Here, we introduce a model of perceptual processing and episodic memory that makes use of hierarchical predictive coding, short-term plasticity, spatiotemporal attention, and episodic memory formation and recall, and apply this model to the problem of human time perception. In an experiment with approximately 13,000 human participants, we investigated the effects of memory, cognitive load, and stimulus content on duration reports of dynamic natural scenes up to about 1 minute long. Using our model to generate duration estimates, we compared human and model performance. Model-based estimates replicated key qualitative biases, including differences by cognitive load (attention), scene type (stimulation), and whether the judgment was made based on current or remembered experience (memory). Our work provides a comprehensive model of human time perception and a foundation for exploring the computational basis of episodic memory within a hierarchical predictive coding framework.
Collapse
Affiliation(s)
- Zafeirios Fountas
- Emotech Labs, London, N1 7EU U.K.,Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3AR, U.K.
| | | | | | - Anil K Seth
- Department of Informatics and Sackler Centre for Consciousness Science, University of Sussex, Brighton, BN1 9RH, U.K.,Canadian Institute for Advanced Research Program on Brain, Mind, and Consciousness, Toronto, ON M5G 1M1, Canada
| | - Murray Shanahan
- Department of Computing, Imperial College London, London, SW7 2RH, U.K.
| | - Warrick Roseboom
- Department of Informatics and Sackler Centre for Consciousness Science, University of Sussex, Brighton BN1 9RH, U.K.
| |
Collapse
|
19
|
Abstract
Mood is an integrative and diffuse affective state that is thought to exert a pervasive effect on cognition and behavior. At the same time, mood itself is thought to fluctuate slowly as a product of feedback from interactions with the environment. Here we present a new computational theory of the valence of mood-the Integrated Advantage model-that seeks to account for this bidirectional interaction. Adopting theoretical formalisms from reinforcement learning, we propose to conceptualize the valence of mood as a leaky integral of an agent's appraisals of the Advantage of its actions. This model generalizes and extends previous models of mood wherein affective valence was conceptualized as a moving average of reward prediction errors. We give a full theoretical derivation of the Integrated Advantage model and provide a functional explanation of how an integrated-Advantage variable could be deployed adaptively by a biological agent to accelerate learning in complex and/or stochastic environments. Specifically, drawing on stochastic optimization theory, we propose that an agent can utilize our hypothesized form of mood to approximate a momentum-based update to its behavioral policy, thereby facilitating rapid learning of optimal actions. We then show how this model of mood provides a principled and parsimonious explanation for a number of contextual effects on mood from the affective science literature, including expectation- and surprise-related effects, counterfactual effects from information about foregone alternatives, action-typicality effects, and action/inaction asymmetry. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Collapse
Affiliation(s)
| | | | - Yael Niv
- Princeton Neuroscience Institute and Department of Psychology
| |
Collapse
|
20
|
Shen X, Ballard IC, Smith DV, Murty VP. Decision uncertainty during hypothesis testing enhances memory accuracy for incidental information. Learn Mem 2022; 29:93-99. [PMID: 35293323 PMCID: PMC8973392 DOI: 10.1101/lm.053458.121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2021] [Accepted: 02/25/2022] [Indexed: 11/24/2022]
Abstract
Humans actively seek information to reduce uncertainty, providing insight on how our decisions causally affect the world. While we know that episodic memories can help support future goal-oriented behaviors, little is known about how hypothesis testing during exploration influences episodic memory. To investigate this question, we designed a hypothesis testing paradigm, in which participants figured out rules to unlock treasure chests. Using this paradigm, we characterized how hypothesis testing during exploration influenced memory for the contents of the treasure chests. We found that there was an inverted U-shaped relationship between decision uncertainty and memory, such that memory was best when decision uncertainty was moderate. An exploratory analysis also showed that surprising outcomes lead to lower memory confidence independent of accuracy. These findings support a model in which moderate decision uncertainty during hypothesis testing enhances incidental information encoding.
Collapse
Affiliation(s)
- Xinxu Shen
- Department of Psychology, Temple University, Philadelphia, Pennsylvania 19122, USA
| | - Ian C Ballard
- Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, California 94720, USA
| | - David V Smith
- Department of Psychology, Temple University, Philadelphia, Pennsylvania 19122, USA
| | - Vishnu P Murty
- Department of Psychology, Temple University, Philadelphia, Pennsylvania 19122, USA
| |
Collapse
|
21
|
Lu Q, Hasson U, Norman KA. A neural network model of when to retrieve and encode episodic memories. eLife 2022; 11:e74445. [PMID: 35142289 PMCID: PMC9000961 DOI: 10.7554/elife.74445] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2021] [Accepted: 02/09/2022] [Indexed: 11/23/2022] Open
Abstract
Recent human behavioral and neuroimaging results suggest that people are selective in when they encode and retrieve episodic memories. To explain these findings, we trained a memory-augmented neural network to use its episodic memory to support prediction of upcoming states in an environment where past situations sometimes reoccur. We found that the network learned to retrieve selectively as a function of several factors, including its uncertainty about the upcoming state. Additionally, we found that selectively encoding episodic memories at the end of an event (but not mid-event) led to better subsequent prediction performance. In all of these cases, the benefits of selective retrieval and encoding can be explained in terms of reducing the risk of retrieving irrelevant memories. Overall, these modeling results provide a resource-rational account of why episodic retrieval and encoding should be selective and lead to several testable predictions.
Collapse
Affiliation(s)
- Qihong Lu
- Department of Psychology, Princeton UniversityPrincetonUnited States
- Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| | - Uri Hasson
- Department of Psychology, Princeton UniversityPrincetonUnited States
- Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| | - Kenneth A Norman
- Department of Psychology, Princeton UniversityPrincetonUnited States
- Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| |
Collapse
|
22
|
Rosenbaum GM, Grassie HL, Hartley CA. Valence biases in reinforcement learning shift across adolescence and modulate subsequent memory. eLife 2022; 11:e64620. [PMID: 35072624 PMCID: PMC8786311 DOI: 10.7554/elife.64620] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2020] [Accepted: 12/24/2021] [Indexed: 12/12/2022] Open
Abstract
As individuals learn through trial and error, some are more influenced by good outcomes, while others weight bad outcomes more heavily. Such valence biases may also influence memory for past experiences. Here, we examined whether valence asymmetries in reinforcement learning change across adolescence, and whether individual learning asymmetries bias the content of subsequent memory. Participants ages 8-27 learned the values of 'point machines,' after which their memory for trial-unique images presented with choice outcomes was assessed. Relative to children and adults, adolescents overweighted worse-than-expected outcomes during learning. Individuals' valence biases modulated incidental memory, such that those who prioritized worse- (or better-) than-expected outcomes during learning were also more likely to remember images paired with these outcomes, an effect reproduced in an independent dataset. Collectively, these results highlight age-related changes in the computation of subjective value and demonstrate that a valence-asymmetric valuation process influences how information is prioritized in episodic memory.
Collapse
Affiliation(s)
- Gail M Rosenbaum
- Department of Psychology, New York UniversityNew YorkUnited States
| | - Hannah L Grassie
- Department of Psychology, New York UniversityNew YorkUnited States
| | - Catherine A Hartley
- Department of Psychology, New York UniversityNew YorkUnited States
- Center for Neural Science, New York UniversityNew YorkUnited States
| |
Collapse
|
23
|
Nicholas J, Daw ND, Shohamy D. Uncertainty alters the balance between incremental learning and episodic memory. eLife 2022; 11:81679. [PMID: 36458809 PMCID: PMC9810331 DOI: 10.7554/elife.81679] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 12/01/2022] [Indexed: 12/04/2022] Open
Abstract
A key question in decision-making is how humans arbitrate between competing learning and memory systems to maximize reward. We address this question by probing the balance between the effects, on choice, of incremental trial-and-error learning versus episodic memories of individual events. Although a rich literature has studied incremental learning in isolation, the role of episodic memory in decision-making has only recently drawn focus, and little research disentangles their separate contributions. We hypothesized that the brain arbitrates rationally between these two systems, relying on each in circumstances to which it is most suited, as indicated by uncertainty. We tested this hypothesis by directly contrasting contributions of episodic and incremental influence to decisions, while manipulating the relative uncertainty of incremental learning using a well-established manipulation of reward volatility. Across two large, independent samples of young adults, participants traded these influences off rationally, depending more on episodic information when incremental summaries were more uncertain. These results support the proposal that the brain optimizes the balance between different forms of learning and memory according to their relative uncertainties and elucidate the circumstances under which episodic memory informs decisions.
Collapse
Affiliation(s)
- Jonathan Nicholas
- Department of Psychology, Columbia UniversityNew YorkUnited States,Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia UniversityNew YorkUnited States
| | - Nathaniel D Daw
- Department of Psychology, Princeton UniversityPrincetonUnited States,Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| | - Daphna Shohamy
- Department of Psychology, Columbia UniversityNew YorkUnited States,Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia UniversityNew YorkUnited States,The Kavli Institute for Brain Science, Columbia UniversityNew YorkUnited States
| |
Collapse
|
24
|
Monosov IE, Rushworth MFS. Interactions between ventrolateral prefrontal and anterior cingulate cortex during learning and behavioural change. Neuropsychopharmacology 2022; 47:196-210. [PMID: 34234288 PMCID: PMC8617208 DOI: 10.1038/s41386-021-01079-2] [Citation(s) in RCA: 23] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Revised: 05/27/2021] [Accepted: 06/15/2021] [Indexed: 02/06/2023]
Abstract
Hypotheses and beliefs guide credit assignment - the process of determining which previous events or actions caused an outcome. Adaptive hypothesis formation and testing are crucial in uncertain and changing environments in which associations and meanings are volatile. Despite primates' abilities to form and test hypotheses, establishing what is causally responsible for the occurrence of particular outcomes remains a fundamental challenge for credit assignment and learning. Hypotheses about what surprises are due to stochasticity inherent in an environment as opposed to real, systematic changes are necessary for identifying the environment's predictive features, but are often hard to test. We review evidence that two highly interconnected frontal cortical regions, anterior cingulate cortex and ventrolateral prefrontal area 47/12o, provide a biological substrate for linking two crucial components of hypothesis-formation and testing: the control of information seeking and credit assignment. Neuroimaging, targeted disruptions, and neurophysiological studies link an anterior cingulate - 47/12o circuit to generation of exploratory behaviour, non-instrumental information seeking, and interpretation of subsequent feedback in the service of credit assignment. Our observations support the idea that information seeking and credit assignment are linked at the level of neural circuits and explain why this circuit is important for ensuring behaviour is flexible and adaptive.
Collapse
Affiliation(s)
- Ilya E Monosov
- Department of Neuroscience, Washington University School of Medicine, St. Louis, MO, USA.
- Department of Biomedical Engineering, Washington University, St. Louis, MO, USA.
- Department of Electrical Engineering, Washington University, St. Louis, MO, USA.
- Department of Neurosurgery, Washington University, St. Louis, MO, USA.
- Pain Center, Washington University, St. Louis, MO, USA.
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, UK.
| |
Collapse
|
25
|
Loganathan K. Value-based cognition and drug dependency. Addict Behav 2021; 123:107070. [PMID: 34359016 DOI: 10.1016/j.addbeh.2021.107070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2021] [Revised: 07/03/2021] [Accepted: 07/26/2021] [Indexed: 10/20/2022]
Abstract
Value-based decision-making is thought to play an important role in drug dependency. Achieving elevated levels of euphoria or ameliorating dysphoria/pain may motivate goal-directed drug consumption in both drug-naïve and long-time users. In other words, drugs become viewed as the preferred means of attaining a desired internal state. The bias towards choosing drugs may affect one's cognition. Observed biases in learning, attention and memory systems within the brain gradually focus one's cognitive functions towards drugs and related cues to the exclusion of other stimuli. In this narrative review, the effects of drug use on learning, attention and memory are discussed with a particular focus on changes across brain-wide functional networks and the subsequent impact on behaviour. These cognitive changes are then incorporated into the cycle of addiction, an established model outlining the transition from casual drug use to chronic dependency. If drug use results in the elevated salience of drugs and their cues, the studies highlighted in this review strongly suggest that this salience biases cognitive systems towards the motivated pursuit of addictive drugs. This bias is observed throughout the cycle of addiction, possibly contributing to the persistent hold that addictive drugs have over the dependent. Taken together, the excessive valuation of drugs as the preferred means of achieving a desired internal state affects more than just decision-making, but also learning, attentional and mnemonic systems. This eventually narrows the focus of one's thoughts towards the pursuit and consumption of addictive drugs.
Collapse
|
26
|
Kalbe F, Schwabe L. Prediction Errors for Aversive Events Shape Long-Term Memory Formation through a Distinct Neural Mechanism. Cereb Cortex 2021; 32:3081-3097. [PMID: 34849622 DOI: 10.1093/cercor/bhab402] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Revised: 09/09/2021] [Accepted: 10/12/2021] [Indexed: 11/13/2022] Open
Abstract
Prediction errors (PEs) have been known for decades to guide associative learning, but their role in episodic memory formation has been discovered only recently. To identify the neural mechanisms underlying the impact of aversive PEs on long-term memory formation, we used functional magnetic resonance imaging, while participants saw a series of unique stimuli and estimated the probability that an aversive shock would follow. Our behavioral data showed that negative PEs (i.e., omission of an expected outcome) were associated with superior recognition of the predictive stimuli, whereas positive PEs (i.e., presentation of an unexpected outcome) impaired subsequent memory. While medial temporal lobe (MTL) activity during stimulus encoding was overall associated with enhanced memory, memory-enhancing effects of negative PEs were linked to even decreased MTL activation. Additional large-scale network analyses showed PE-related increases in crosstalk between the "salience network" and a frontoparietal network commonly implicated in memory formation for expectancy-congruent events. These effects could not be explained by mere changes in physiological arousal or the prediction itself. Our results suggest that the superior memory for events associated with negative aversive PEs is driven by a potentially distinct neural mechanism that might serve to set these memories apart from those with expected outcomes.
Collapse
Affiliation(s)
- Felix Kalbe
- Department of Cognitive Psychology, Institute of Psychology, Universität Hamburg, Hamburg 20146, Germany
| | - Lars Schwabe
- Department of Cognitive Psychology, Institute of Psychology, Universität Hamburg, Hamburg 20146, Germany
| |
Collapse
|
27
|
Talmi D, Kavaliauskaite D, Daw ND. In for a penny, in for a pound: examining motivated memory through the lens of retrieved context models. Learn Mem 2021; 28:445-456. [PMID: 34782403 DOI: 10.1101/lm.053470.121] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2021] [Accepted: 08/31/2021] [Indexed: 11/24/2022]
Abstract
When people encounter items that they believe will help them gain reward, they later remember them better than others. A recent model of emotional memory, the emotional context maintenance and retrieval model (eCMR), predicts that these effects would be stronger when stimuli that predict high and low reward can compete with each other during both encoding and retrieval. We tested this prediction in two experiments. Participants were promised £1 for remembering some pictures, but only a few pence for remembering others. Their recall of the content of the pictures they saw was tested after 1 min and, in experiment 2, also after 24 h. Memory at the immediate test showed effects of list composition. Recall of stimuli that predicted high reward was greater than of stimuli that predicted lower reward, but only when high- and low-reward items were studied and recalled together, not when they were studied and recalled separately. More high-reward items in mixed lists were forgotten over a 24-h retention interval compared with items studied in other conditions, but reward did not modulate the forgetting rate, a null effect that should be replicated in a larger sample. These results confirm eCMR's predictions, although further research is required to compare that model against alternatives.
Collapse
Affiliation(s)
- Deborah Talmi
- Department of Psychology, University of Cambridge, Cambridge CB2 3EB, United Kingdom
| | | | - Nathaniel D Daw
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey 08540, USA.,Department of Psychology, Princeton University, Princeton, New Jersey 08540, USA
| |
Collapse
|
28
|
Bein O, Plotkin NA, Davachi L. Mnemonic prediction errors promote detailed memories. Learn Mem 2021; 28:422-434. [PMID: 34663695 PMCID: PMC8525423 DOI: 10.1101/lm.053410.121] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2021] [Accepted: 08/16/2021] [Indexed: 12/19/2022]
Abstract
When our experience violates our predictions, it is adaptive to update our knowledge to promote a more accurate representation of the world and facilitate future predictions. Theoretical models propose that these mnemonic prediction errors should be encoded into a distinct memory trace to prevent interference with previous, conflicting memories. We investigated this proposal by repeatedly exposing participants to pairs of sequentially presented objects (A → B), thus evoking expectations. Then, we violated participants' expectations by replacing the second object in the pairs with a novel object (A → C). The following item memory test required participants to discriminate between identical old items and similar lures, thus testing detailed and distinctive item memory representations. In two experiments, mnemonic prediction errors enhanced item memory: Participants correctly identified more old items as old when those items violated expectations during learning, compared with items that did not violate expectations. This memory enhancement for C items was only observed when participants later showed intact memory for the related A → B pairs, suggesting that strong predictions are required to facilitate memory for violations. Following up on this, a third experiment reduced prediction strength prior to violation and subsequently eliminated the memory advantage of violations. Interestingly, mnemonic prediction errors did not increase gist-based mistakes of identifying old items as similar lures or identifying similar lures as old. Enhanced item memory in the absence of gist-based mistakes suggests that violations enhanced memory for items' details, which could be mediated via distinct memory traces. Together, these results advance our knowledge of how mnemonic prediction errors promote memory formation.
Collapse
Affiliation(s)
- Oded Bein
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey 08540, USA
| | - Natalie A Plotkin
- Department of Psychology, Columbia University, New York, New York 10027, USA
| | - Lila Davachi
- Department of Psychology, Columbia University, New York, New York 10027, USA
- Center for Clinical Research, The Nathan S. Kline Institute for Psychiatric Research, Orangeburg, New York 10962, USA
| |
Collapse
|
29
|
Nussenbaum K, Hartley CA. Developmental change in prefrontal cortex recruitment supports the emergence of value-guided memory. eLife 2021; 10:e69796. [PMID: 34542408 PMCID: PMC8452307 DOI: 10.7554/elife.69796] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2021] [Accepted: 08/23/2021] [Indexed: 12/18/2022] Open
Abstract
Prioritizing memory for valuable information can promote adaptive behavior across the lifespan, but it is unclear how the neurocognitive mechanisms that enable the selective acquisition of useful knowledge develop. Here, using a novel task coupled with functional magnetic resonance imaging, we examined how children, adolescents, and adults (N = 90) learn from experience what information is likely to be rewarding, and modulate encoding and retrieval processes accordingly. We found that the ability to use learned value signals to selectively enhance memory for useful information strengthened throughout childhood and into adolescence. Encoding and retrieval of high- vs. low-value information was associated with increased activation in striatal and prefrontal regions implicated in value processing and cognitive control. Age-related increases in value-based lateral prefrontal cortex modulation mediated the relation between age and memory selectivity. Our findings demonstrate that developmental increases in the strategic engagement of the prefrontal cortex support the emergence of adaptive memory.
Collapse
|
30
|
Moeller M, Grohn J, Manohar S, Bogacz R. An association between prediction errors and risk-seeking: Theory and behavioral evidence. PLoS Comput Biol 2021; 17:e1009213. [PMID: 34270552 PMCID: PMC8318232 DOI: 10.1371/journal.pcbi.1009213] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2021] [Revised: 07/28/2021] [Accepted: 06/23/2021] [Indexed: 11/19/2022] Open
Abstract
Reward prediction errors (RPEs) and risk preferences have two things in common: both can shape decision making behavior, and both are commonly associated with dopamine. RPEs drive value learning and are thought to be represented in the phasic release of striatal dopamine. Risk preferences bias choices towards or away from uncertainty; they can be manipulated with drugs that target the dopaminergic system. Based on the common neural substrate, we hypothesize that RPEs and risk preferences are linked on the level of behavior as well. Here, we develop this hypothesis theoretically and test it empirically. First, we apply a recent theory of learning in the basal ganglia to predict how RPEs influence risk preferences. We find that positive RPEs should cause increased risk-seeking, while negative RPEs should cause risk-aversion. We then test our behavioral predictions using a novel bandit task in which value and risk vary independently across options. Critically, conditions are included where options vary in risk but are matched for value. We find that our prediction was correct: participants become more risk-seeking if choices are preceded by positive RPEs, and more risk-averse if choices are preceded by negative RPEs. These findings cannot be explained by other known effects, such as nonlinear utility curves or dynamic learning rates.
Collapse
Affiliation(s)
- Moritz Moeller
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| | - Jan Grohn
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Sanjay Manohar
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Rafal Bogacz
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
31
|
Reward prediction errors drive declarative learning irrespective of agency. Psychon Bull Rev 2021; 28:2045-2056. [PMID: 34131890 DOI: 10.3758/s13423-021-01952-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/12/2021] [Indexed: 11/08/2022]
Abstract
Recent years have witnessed a steady increase in the number of studies investigating the role of reward prediction errors (RPEs) in declarative learning. Specifically, in several experimental paradigms, RPEs drive declarative learning, with larger and more positive RPEs enhancing declarative learning. However, it is unknown whether this RPE must derive from the participant's own response, or whether instead, any RPE is sufficient to obtain the learning effect. To test this, we generated RPEs in the same experimental paradigm where we combined an agency and a nonagency condition. We observed no interaction between RPE and agency, suggesting that any RPE (irrespective of its source) can drive declarative learning. This result holds implications for declarative learning theory.
Collapse
|
32
|
Unkelbach C, Koch A, Alves H. Explaining Negativity Dominance without Processing Bias. Trends Cogn Sci 2021; 25:429-430. [PMID: 33875383 DOI: 10.1016/j.tics.2021.04.005] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Accepted: 04/06/2021] [Indexed: 10/21/2022]
Abstract
In a recent study, Shin and Niv explain both negativity and positivity biases in social evaluations as a function of the diversity and low frequency of events. We discuss why negative information is indeed more diverse and less frequent, and highlight the implications beyond social evaluations.
Collapse
Affiliation(s)
| | - Alex Koch
- Booth School of Business, University of Chicago, Chicago, IL 60637, USA
| | - Hans Alves
- Faculty of Psychology, Ruhr-Universität Bochum, 44780 Bochum, Germany
| |
Collapse
|
33
|
Abstract
The central theme of this review is the dynamic interaction between information selection and learning. We pose a fundamental question about this interaction: How do we learn what features of our experiences are worth learning about? In humans, this process depends on attention and memory, two cognitive functions that together constrain representations of the world to features that are relevant for goal attainment. Recent evidence suggests that the representations shaped by attention and memory are themselves inferred from experience with each task. We review this evidence and place it in the context of work that has explicitly characterized representation learning as statistical inference. We discuss how inference can be scaled to real-world decisions by approximating beliefs based on a small number of experiences. Finally, we highlight some implications of this inference process for human decision-making in social environments.
Collapse
Affiliation(s)
- Angela Radulescu
- Department of Psychology, Princeton University, Princeton, New Jersey 08544, USA; .,Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey 08544, USA
| | - Yeon Soon Shin
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey 08544, USA
| | - Yael Niv
- Department of Psychology, Princeton University, Princeton, New Jersey 08544, USA; .,Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey 08544, USA
| |
Collapse
|
34
|
Individual differences in experienced and observational decision-making illuminate interactions between reinforcement learning and declarative memory. Sci Rep 2021; 11:5899. [PMID: 33723288 PMCID: PMC7971018 DOI: 10.1038/s41598-021-85322-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2020] [Accepted: 02/22/2021] [Indexed: 11/15/2022] Open
Abstract
Decision making can be shaped both by trial-and-error experiences and by memory of unique contextual information. Moreover, these types of information can be acquired either by means of active experience or by observing others behave in similar situations. The interactions between reinforcement learning parameters that inform decision updating and memory formation of declarative information in experienced and observational learning settings are, however, unknown. In the current study, participants took part in a probabilistic decision-making task involving situations that either yielded similar outcomes to those of an observed player or opposed them. By fitting alternative reinforcement learning models to each subject, we discerned participants who learned similarly from experience and observation from those who assigned different weights to learning signals from these two sources. Participants who assigned different weights to their own experience versus those of others displayed enhanced memory performance as well as subjective memory strength for episodes involving significant reward prospects. Conversely, memory performance of participants who did not prioritize their own experience over others did not seem to be influenced by reinforcement learning parameters. These findings demonstrate that interactions between implicit and explicit learning systems depend on the means by which individuals weigh relevant information conveyed via experience and observation.
Collapse
|
35
|
Rouhani N, Niv Y. Signed and unsigned reward prediction errors dynamically enhance learning and memory. eLife 2021; 10:e61077. [PMID: 33661094 PMCID: PMC8041467 DOI: 10.7554/elife.61077] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2020] [Accepted: 02/26/2021] [Indexed: 02/05/2023] Open
Abstract
Memory helps guide behavior, but which experiences from the past are prioritized? Classic models of learning posit that events associated with unpredictable outcomes as well as, paradoxically, predictable outcomes, deploy more attention and learning for those events. Here, we test reinforcement learning and subsequent memory for those events, and treat signed and unsigned reward prediction errors (RPEs), experienced at the reward-predictive cue or reward outcome, as drivers of these two seemingly contradictory signals. By fitting reinforcement learning models to behavior, we find that both RPEs contribute to learning by modulating a dynamically changing learning rate. We further characterize the effects of these RPE signals on memory and show that both signed and unsigned RPEs enhance memory, in line with midbrain dopamine and locus-coeruleus modulation of hippocampal plasticity, thereby reconciling separate findings in the literature.
Collapse
Affiliation(s)
- Nina Rouhani
- Chen Neuroscience Institute, California Institute of TechnologyPasadenaUnited States
| | - Yael Niv
- Department of Psychology, Princeton UniversityPrincetonUnited States
- Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| |
Collapse
|
36
|
Quent JA, Henson RN, Greve A. A predictive account of how novelty influences declarative memory. Neurobiol Learn Mem 2021; 179:107382. [PMID: 33476747 PMCID: PMC8024513 DOI: 10.1016/j.nlm.2021.107382] [Citation(s) in RCA: 34] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2020] [Revised: 11/08/2020] [Accepted: 01/10/2021] [Indexed: 01/13/2023]
Abstract
A rich body of studies in the human and non-human literature has examined the question how novelty influences memory. For a variety of different stimuli, ranging from simple objects and words to vastly complex scenarios, the literature reports that novelty improves memory in some cases, but impairs memory in other cases. In recent attempts to reconcile these conflicting findings, novelty has been divided into different subtypes, such as relative versus absolute novelty, or stimulus versus contextual novelty. Nevertheless, a single overarching theory of novelty and memory has been difficult to attain, probably due to the complexities in the interactions among stimuli, environmental factors (e.g., spatial and temporal context) and level of prior knowledge (but see Duszkiewicz et al., 2019; Kafkas & Montaldi, 2018b; Schomaker & Meeter, 2015). Here we describe how a predictive coding framework might be able to shed new light on different types of novelty and how they affect declarative memory in humans. More precisely, we consider how prior expectations modulate the influence of novelty on encoding episodes into memory, e.g., in terms of surprise, and how novelty/surprise affect memory for surrounding information. By reviewing a range of behavioural findings and their possible underlying neurobiological mechanisms, we highlight where a predictive coding framework succeeds and where it appears to struggle.
Collapse
Affiliation(s)
| | - Richard N Henson
- MRC Cognition and Brain Sciences Unit, University of Cambridge, United Kingdom; Department of Psychiatry, University of Cambridge, United Kingdom
| | - Andrea Greve
- MRC Cognition and Brain Sciences Unit, University of Cambridge, United Kingdom
| |
Collapse
|
37
|
Clark MD, Chang LJ. Surprise Signals Changing Affective Experiences in Naturalistic Sports Spectating. Neuron 2021; 109:199-201. [PMID: 33476562 PMCID: PMC7938857 DOI: 10.1016/j.neuron.2020.12.022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
What role does surprise play in a spectating experience? In this issue of Neuron, Antony et al. (2021) develop and validate a model of surprise to characterize the psychological and neurobiological processes underlying sports fans' experiences as they watch the final minutes of NCAA college basketball games.
Collapse
Affiliation(s)
- Marissa D Clark
- Computational Social and Affective Neuroscience Laboratory, Department of Psychological & Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Luke J Chang
- Computational Social and Affective Neuroscience Laboratory, Department of Psychological & Brain Sciences, Dartmouth College, Hanover, NH, USA.
| |
Collapse
|
38
|
Signed Reward Prediction Errors in the Ventral Striatum Drive Episodic Memory. J Neurosci 2020; 41:1716-1726. [PMID: 33334870 DOI: 10.1523/jneurosci.1785-20.2020] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2020] [Revised: 12/04/2020] [Accepted: 12/07/2020] [Indexed: 11/21/2022] Open
Abstract
Recent behavioral evidence implicates reward prediction errors (RPEs) as a key factor in the acquisition of episodic memory. Yet, important neural predictions related to the role of RPEs in episodic memory acquisition remain to be tested. Humans (both sexes) performed a novel variable-choice task where we experimentally manipulated RPEs and found support for key neural predictions with fMRI. Our results show that in line with previous behavioral observations, episodic memory accuracy increases with the magnitude of signed (i.e., better/worse-than-expected) RPEs (SRPEs). Neurally, we observe that SRPEs are encoded in the ventral striatum (VS). Crucially, we demonstrate through mediation analysis that activation in the VS mediates the experimental manipulation of SRPEs on episodic memory accuracy. In particular, SRPE-based responses in the VS (during learning) predict the strength of subsequent episodic memory (during recollection). Furthermore, functional connectivity between task-relevant processing areas (i.e., face-selective areas) and hippocampus and ventral striatum increased as a function of RPE value (during learning), suggesting a central role of these areas in episodic memory formation. Our results consolidate reinforcement learning theory and striatal RPEs as key factors subtending the formation of episodic memory.SIGNIFICANCE STATEMENT Recent behavioral research has shown that reward prediction errors (RPEs), a key concept of reinforcement learning theory, are crucial to the formation of episodic memories. In this study, we reveal the neural underpinnings of this process. Using fMRI, we show that signed RPEs (SRPEs) are encoded in the ventral striatum (VS), and crucially, that SRPE VS activity is responsible for the subsequent recollection accuracy of one-shot learned episodic memory associations.
Collapse
|
39
|
Ergo K, De Loof E, Debra G, Pastötter B, Verguts T. Failure to modulate reward prediction errors in declarative learning with theta (6 Hz) frequency transcranial alternating current stimulation. PLoS One 2020; 15:e0237829. [PMID: 33270685 PMCID: PMC7714179 DOI: 10.1371/journal.pone.0237829] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2020] [Accepted: 11/18/2020] [Indexed: 12/26/2022] Open
Abstract
Recent evidence suggests that reward prediction errors (RPEs) play an important role in declarative learning, but its neurophysiological mechanism remains unclear. Here, we tested the hypothesis that RPEs modulate declarative learning via theta-frequency oscillations, which have been related to memory encoding in prior work. For that purpose, we examined the interaction between RPE and transcranial Alternating Current Stimulation (tACS) in declarative learning. Using a between-subject (real versus sham stimulation group), single-blind stimulation design, 76 participants learned 60 Dutch-Swahili word pairs, while theta-frequency (6 Hz) tACS was administered over the medial frontal cortex (MFC). Previous studies have implicated MFC in memory encoding. We replicated our previous finding of signed RPEs (SRPEs) boosting declarative learning; with larger and more positive RPEs enhancing memory performance. However, tACS failed to modulate the SRPE effect in declarative learning and did not affect memory performance. Bayesian statistics supported evidence for an absence of effect. Our study confirms a role of RPE in declarative learning, but also calls for standardized procedures in transcranial electrical stimulation.
Collapse
Affiliation(s)
- Kate Ergo
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| | - Esther De Loof
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| | - Gillian Debra
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| | | | - Tom Verguts
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| |
Collapse
|
40
|
Behavioral, Physiological, and Neural Signatures of Surprise during Naturalistic Sports Viewing. Neuron 2020; 109:377-390.e7. [PMID: 33242421 DOI: 10.1016/j.neuron.2020.10.029] [Citation(s) in RCA: 50] [Impact Index Per Article: 12.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2020] [Revised: 08/07/2020] [Accepted: 10/22/2020] [Indexed: 12/13/2022]
Abstract
Surprise signals a discrepancy between past and current beliefs. It is theorized to be linked to affective experiences, the creation of particularly resilient memories, and segmentation of the flow of experience into discrete perceived events. However, the ability to precisely measure naturalistic surprise has remained elusive. We used advanced basketball analytics to derive a quantitative measure of surprise and characterized its behavioral, physiological, and neural correlates in human subjects observing basketball games. We found that surprise was associated with segmentation of ongoing experiences, as reflected by subjectively perceived event boundaries and shifts in neocortical patterns underlying belief states. Interestingly, these effects differed by whether surprising moments contradicted or bolstered current predominant beliefs. Surprise also positively correlated with pupil dilation, activation in subcortical regions associated with dopamine, game enjoyment, and long-term memory. These investigations support key predictions from event segmentation theory and extend theoretical conceptualizations of surprise to real-world contexts.
Collapse
|
41
|
Abstract
Stressful events are often vividly remembered. Although generally adaptive to survival, this emotional-memory enhancement may contribute to stress-related disorders. We tested here whether the enhanced memory for stressful events is due to the expectancy violation evoked by these events. Ninety-four men and women underwent a stressful or control episode. Critically, to manipulate the degree of expectancy violation, we gave participants either detailed or minimal information about the stressor. Although the subjective and hormonal stress responses were comparable in informed and uninformed participants, prior information about the stressor abolished the memory advantage for core features of the stressful event, tested 7 days later. Using functional near-infrared spectroscopy, we further linked the expectancy violation and memory formation under stress to the inferior temporal cortex. These data are the first to show that detailed information about an upcoming stressor and, by implication, a reduced expectancy violation attenuates the memory for stressful events.
Collapse
Affiliation(s)
- Felix Kalbe
- Department of Cognitive Psychology, Institute of Psychology, Universität Hamburg
| | - Stina Bange
- Department of Cognitive Psychology, Institute of Psychology, Universität Hamburg
| | - Annika Lutz
- Department of Cognitive Psychology, Institute of Psychology, Universität Hamburg
| | - Lars Schwabe
- Department of Cognitive Psychology, Institute of Psychology, Universität Hamburg
| |
Collapse
|
42
|
Monosov IE. How Outcome Uncertainty Mediates Attention, Learning, and Decision-Making. Trends Neurosci 2020; 43:795-809. [PMID: 32736849 PMCID: PMC8153236 DOI: 10.1016/j.tins.2020.06.009] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2020] [Revised: 06/16/2020] [Accepted: 06/24/2020] [Indexed: 01/24/2023]
Abstract
Animals and humans evolved sophisticated nervous systems that endowed them with the ability to form internal-models or beliefs and make predictions about the future to survive and flourish in a world in which future outcomes are often uncertain. Crucial to this capacity is the ability to adjust behavioral and learning policies in response to the level of uncertainty. Until recently, the neuronal mechanisms that could underlie such uncertainty-guided control have been largely unknown. In this review, I discuss newly discovered neuronal circuits in primates that represent uncertainty about future rewards and propose how they guide information-seeking, attention, decision-making, and learning to help us survive in an uncertain world. Lastly, I discuss the possible relevance of these findings to learning in artificial systems.
Collapse
Affiliation(s)
- Ilya E Monosov
- Department of Neuroscience and Neurosurgery, Washington University School of Medicine in St. Louis, MO, USA; Department of Biomedical Engineering, Washington University School of Medicine in St. Louis, MO, USA; Washington University Pain Center, Washington University School of Medicine in St. Louis, MO, USA.
| |
Collapse
|
43
|
van de Vijver I, Ligneul R. Relevance of working memory for reinforcement learning in older adults varies with timescale of learning. NEUROPSYCHOLOGY, DEVELOPMENT, AND COGNITION. SECTION B, AGING, NEUROPSYCHOLOGY AND COGNITION 2020; 27:654-676. [PMID: 31544587 DOI: 10.1080/13825585.2019.1664389] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/09/2018] [Accepted: 09/02/2019] [Indexed: 06/10/2023]
Abstract
In young adults, individual differences in working memory (WM) contribute to reinforcement learning (RL). Age-related RL changes, however, are mostly attributed to decreased reward prediction-error (RPE) signaling. Here, we investigated the contribution of WM to RL in young (18-35) and older (≥65) adults. Because WM supports maintenance across a limited timescale, we only expected a relation between RL and WM with short delays between stimulus repetitions. Our results demonstrated better learning with short than long delays. A week later, however, long-delay associations were remembered better. Computational modeling corroborated that during learning, WM was more engaged by young adults in the short-delay condition than in any other age-condition combination. Crucially, both model-derived and neuropsychological assessments of WM predicted short-delay learning in older adults, who further benefitted from using self-conceived learning strategies. Thus, depending on the timescale of learning, age-related RL changes may not only reflect decreased RPE signaling but also WM decline.
Collapse
Affiliation(s)
- Irene van de Vijver
- Behavioural Science Institute, Radboud University , Nijmegen, The Netherlands
- Department of Clinical Psychology, University of Amsterdam , Amsterdam, The Netherlands
- Amsterdam Brain and Cognition, University of Amsterdam , Amsterdam, The Netherlands
| | - Romain Ligneul
- Champalimaud Neuroscience Program, Champalimaud Foundation , Lisboa, Portugal
| |
Collapse
|
44
|
Murty VP, Fain MR, Hlutkowsky C, Perlman SB. Memory for social interactions throughout early childhood. Cognition 2020; 202:104324. [DOI: 10.1016/j.cognition.2020.104324] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2019] [Revised: 05/06/2020] [Accepted: 05/07/2020] [Indexed: 12/11/2022]
|
45
|
Sharp ME, Duncan K, Foerde K, Shohamy D. Dopamine is associated with prioritization of reward-associated memories in Parkinson's disease. Brain 2020; 143:2519-2531. [PMID: 32844197 DOI: 10.1093/brain/awaa182] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2019] [Revised: 04/08/2020] [Accepted: 04/16/2020] [Indexed: 01/23/2023] Open
Abstract
Patients with Parkinson's disease have reduced reward sensitivity related to dopaminergic neuron loss, which is associated with impairments in reinforcement learning. Increasingly, however, dopamine-dependent reward signals are recognized to play an important role beyond reinforcement learning. In particular, it has been shown that reward signals mediated by dopamine help guide the prioritization of events for long-term memory consolidation. Meanwhile, studies of memory in patients with Parkinson's disease have focused on overall memory capacity rather than what is versus what isn't remembered, leaving open questions about the effect of dopamine replacement on the prioritization of memories by reward and the time-dependence of this effect. The current study sought to fill this gap by testing the effect of reward and dopamine on memory in patients with Parkinson's disease. We tested the effect of dopamine modulation and reward on two forms of long-term memory: episodic memory for neutral objects and memory for stimulus-value associations. We measured both forms of memory in a single task, adapting a standard task of reinforcement learning with incidental episodic encoding events of trial-unique objects. Objects were presented on each trial at the time of feedback, which was either rewarding or not. Memory for the trial-unique images and for the stimulus-value associations, and the influence of reward on both, was tested immediately after learning and 2 days later. We measured performance in Parkinson's disease patients tested either ON or OFF their dopaminergic medications and in healthy older control subjects. We found that dopamine was associated with a selective enhancement of memory for reward-associated images, but that it did not influence overall memory capacity. Contrary to predictions, this effect did not differ between the immediate and delayed memory tests. We also found that while dopamine had an effect on reward-modulated episodic memory, there was no effect of dopamine on memory for stimulus-value associations. Our results suggest that impaired prioritization of cognitive resource allocation may contribute to the early cognitive deficits of Parkinson's disease.
Collapse
Affiliation(s)
- Madeleine E Sharp
- Department of Neurology and Neurosurgery, McGill University, Montreal, Quebec, Canada
| | - Katherine Duncan
- Department of Psychology, University of Toronto, Toronto, Canada
| | - Karin Foerde
- New York State Psychiatric Institute and Department of Psychiatry, Columbia University, New York, NY, USA
| | - Daphna Shohamy
- Department of Psychology, Columbia University, New York, NY, USA.,Zuckerman Mind, Brain, Behavior Institute, Columbia University, New York, NY, USA.,Kavli Institute for Brain Science, Columbia University, New York, NY, USA
| |
Collapse
|
46
|
Rouhani N, Norman KA, Niv Y, Bornstein AM. Reward prediction errors create event boundaries in memory. Cognition 2020; 203:104269. [PMID: 32563083 DOI: 10.1016/j.cognition.2020.104269] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2019] [Revised: 03/13/2020] [Accepted: 03/16/2020] [Indexed: 10/24/2022]
Abstract
We remember when things change. Particularly salient are experiences where there is a change in rewards, eliciting reward prediction errors (RPEs). How do RPEs influence our memory of those experiences? One idea is that this signal directly enhances the encoding of memory. Another, not mutually exclusive, idea is that the RPE signals a deeper change in the environment, leading to the mnemonic separation of subsequent experiences from what came before, thereby creating a new latent context and a more separate memory trace. We tested this in four experiments where participants learned to predict rewards associated with a series of trial-unique images. High-magnitude RPEs indicated a change in the underlying distribution of rewards. To test whether these large RPEs created a new latent context, we first assessed recognition priming for sequential pairs that included a high-RPE event or not (Exp. 1: n = 27 & Exp. 2: n = 83). We found evidence of recognition priming for the high-RPE event, indicating that the high-RPE event is bound to its predecessor in memory. Given that high-RPE events are themselves preferentially remembered (Rouhani, Norman, & Niv, 2018), we next tested whether there was an event boundary across a high-RPE event (i.e., excluding the high-RPE event itself; Exp. 3: n = 85). Here, sequential pairs across a high RPE no longer showed recognition priming whereas pairs within the same latent reward state did, providing initial evidence for an RPE-modulated event boundary. We then investigated whether RPE event boundaries disrupt temporal memory by asking participants to order and estimate the distance between two events that had either included a high-RPE event between them or not (Exp. 4). We found (n = 49) and replicated (n = 77) worse sequence memory for events across a high RPE. In line with our recognition priming results, we did not find sequence memory to be impaired between the high-RPE event and its predecessor, but instead found worse sequence memory for pairs across a high-RPE event. Moreover, greater distance between events at encoding led to better sequence memory for events across a low-RPE event, but not a high-RPE event, suggesting separate mechanisms for the temporal ordering of events within versus across a latent reward context. Altogether, these findings demonstrate that high-RPE events are both more strongly encoded, show intact links with their predecessor, and act as event boundaries that interrupt the sequential integration of events. We captured these effects in a variant of the Context Maintenance and Retrieval model (CMR; Polyn, Norman, & Kahana, 2009), modified to incorporate RPEs into the encoding process.
Collapse
Affiliation(s)
- Nina Rouhani
- Princeton Neuroscience Institute, Princeton University, United States of America; Department of Psychology, Princeton University, United States of America.
| | - Kenneth A Norman
- Princeton Neuroscience Institute, Princeton University, United States of America; Department of Psychology, Princeton University, United States of America
| | - Yael Niv
- Princeton Neuroscience Institute, Princeton University, United States of America; Department of Psychology, Princeton University, United States of America
| | - Aaron M Bornstein
- Department of Cognitive Sciences and Center for the Neurobiology of Learning and Memory, University of California, Irvine, United States of America
| |
Collapse
|
47
|
Shin YS, DuBrow S. Structuring Memory Through Inference‐Based Event Segmentation. Top Cogn Sci 2020; 13:106-127. [DOI: 10.1111/tops.12505] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2020] [Revised: 03/29/2019] [Accepted: 04/14/2020] [Indexed: 11/28/2022]
|
48
|
Bornstein AM, Pickard H. "Chasing the first high": memory sampling in drug choice. Neuropsychopharmacology 2020; 45:907-915. [PMID: 31896119 PMCID: PMC7162911 DOI: 10.1038/s41386-019-0594-2] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/04/2019] [Revised: 11/21/2019] [Accepted: 12/16/2019] [Indexed: 02/02/2023]
Abstract
Although vivid memories of drug experiences are prevalent within clinical contexts and addiction folklore ("chasing the first high"), little is known about the relevance of cognitive processes governing memory retrieval to substance use disorder. Drawing on recent work that identifies episodic memory's influence on decisions for reward, we propose a framework in which drug choices are biased by selective sampling of individual memories during two phases of addiction: (i) downward spiral into persistent use and (ii) relapse. Consideration of how memory retrieval influences the addiction process suggests novel treatment strategies. Rather than try to break learned associations between drug cues and drug rewards, treatment should aim to strengthen existing and/or create new associations between drug cues and drug-inconsistent rewards.
Collapse
Affiliation(s)
- Aaron M Bornstein
- Department of Cognitive Sciences, University of California, Irvine, CA, 92617, USA.
- Center for the Neurobiology of Learning and Memory, University of California, Irvine, CA, 92697, USA.
- Institute for Mathematical Behavioral Sciences, University of California, Irvine, CA, 92697, USA.
| | - Hanna Pickard
- Department of Philosophy, Johns Hopkins University, Baltimore, MD, 21218, USA.
- Berman Institute of Bioethics, Johns Hopkins University, Baltimore, MD, 21205, USA.
| |
Collapse
|
49
|
Aberg KC, Kramer EE, Schwartz S. Interplay between midbrain and dorsal anterior cingulate regions arbitrates lingering reward effects on memory encoding. Nat Commun 2020; 11:1829. [PMID: 32286275 PMCID: PMC7156375 DOI: 10.1038/s41467-020-15542-z] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2019] [Accepted: 03/12/2020] [Indexed: 12/17/2022] Open
Abstract
Rewarding events enhance memory encoding via dopaminergic influences on hippocampal plasticity. Phasic dopamine release depends on immediate reward magnitude, but presumably also on tonic dopamine levels, which may vary as a function of the average accumulation of reward over time. Using model-based fMRI in combination with a novel associative memory task, we show that immediate reward magnitude exerts a monotonically increasing influence on the nucleus accumbens, ventral tegmental area (VTA), and hippocampal activity during encoding, and enhances memory. By contrast, average reward levels modulate feedback-related responses in the VTA and hippocampus in a non-linear (inverted U-shape) fashion, with similar effects on memory performance. Additionally, the dorsal anterior cingulate cortex (dACC) monotonically tracks average reward levels, while VTA-dACC functional connectivity is non-linearly modulated (inverted U-shape) by average reward. We propose that the dACC computes the net behavioral impact of average reward and relays this information to memory circuitry via the VTA.
Collapse
Affiliation(s)
| | - Emily Elizabeth Kramer
- Program in Neurosciences and Mental Health, Hospital for Sick Children, Toronto, ON, Canada.,Institute of Medical Sciences, University of Toronto, Toronto, ON, Canada
| | - Sophie Schwartz
- Department of Neuroscience, University of Geneva, Geneva, Switzerland.,Swiss Center for Affective Sciences, University of Geneva, Geneva, Switzerland.,Geneva Neuroscience Center, University of Geneva, Geneva, Switzerland
| |
Collapse
|
50
|
Ergo K, De Loof E, Verguts T. Reward Prediction Error and Declarative Memory. Trends Cogn Sci 2020; 24:388-397. [PMID: 32298624 DOI: 10.1016/j.tics.2020.02.009] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2019] [Revised: 02/03/2020] [Accepted: 02/22/2020] [Indexed: 01/04/2023]
Abstract
Learning based on reward prediction error (RPE) was originally proposed in the context of nondeclarative memory. We postulate that RPE may support declarative memory as well. Indeed, recent years have witnessed a number of independent empirical studies reporting effects of RPE on declarative memory. We provide a brief overview of these studies, identify emerging patterns, and discuss open issues such as the role of signed versus unsigned RPEs in declarative learning.
Collapse
Affiliation(s)
- Kate Ergo
- Department of Experimental Psychology, Ghent University, Henri Dunantlaan 2, B-9000 Ghent, Belgium
| | - Esther De Loof
- Department of Experimental Psychology, Ghent University, Henri Dunantlaan 2, B-9000 Ghent, Belgium
| | - Tom Verguts
- Department of Experimental Psychology, Ghent University, Henri Dunantlaan 2, B-9000 Ghent, Belgium.
| |
Collapse
|