1
|
Nicholas J, Amlang C, Lin CYR, Montaser-Kouhsari L, Desai N, Pan MK, Kuo SH, Shohamy D. The Role of the Cerebellum in Learning to Predict Reward: Evidence from Cerebellar Ataxia. CEREBELLUM (LONDON, ENGLAND) 2024; 23:1355-1368. [PMID: 38066397 PMCID: PMC11161554 DOI: 10.1007/s12311-023-01633-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/02/2023] [Indexed: 01/25/2024]
Abstract
Recent findings in animals have challenged the traditional view of the cerebellum solely as the site of motor control, suggesting that the cerebellum may also be important for learning to predict reward from trial-and-error feedback. Yet, evidence for the role of the cerebellum in reward learning in humans is lacking. Moreover, open questions remain about which specific aspects of reward learning the cerebellum may contribute to. Here we address this gap through an investigation of multiple forms of reward learning in individuals with cerebellum dysfunction, represented by cerebellar ataxia cases. Nineteen participants with cerebellar ataxia and 57 age- and sex-matched healthy controls completed two separate tasks that required learning about reward contingencies from trial-and-error. To probe the selectivity of reward learning processes, the tasks differed in their underlying structure: while one task measured incremental reward learning ability alone, the other allowed participants to use an alternative learning strategy based on episodic memory alongside incremental reward learning. We found that individuals with cerebellar ataxia were profoundly impaired at reward learning from trial-and-error feedback on both tasks, but retained the ability to learn to predict reward based on episodic memory. These findings provide evidence from humans for a specific and necessary role for the cerebellum in incremental learning of reward associations based on reinforcement. More broadly, the findings suggest that alongside its role in motor learning, the cerebellum likely operates in concert with the basal ganglia to support reinforcement learning from reward.
Collapse
Affiliation(s)
- Jonathan Nicholas
- Department of Psychology, Columbia University, New York, NY, USA
- Zuckerman Mind Brain Behavior Institute, Columbia University, Quad 3D, 3227 Broadway, New York, NY, 10027, USA
| | - Christian Amlang
- Department of Neurology, Columbia University Medical Center, 650 W. 168th St, Rm 305, New York, NY, 10032, USA
- Initiative for Columbia Ataxia and Tremor, Columbia University Medical Center, New York, NY, USA
| | - Chi-Ying R Lin
- Department of Neurology, Baylor College of Medicine, Houston, TX, USA
| | | | - Natasha Desai
- Department of Neurology, Columbia University Medical Center, 650 W. 168th St, Rm 305, New York, NY, 10032, USA
- Initiative for Columbia Ataxia and Tremor, Columbia University Medical Center, New York, NY, USA
| | - Ming-Kai Pan
- Department of Medical Research, National Taiwan University Hospital, 100, Taipei, Taiwan
- Department and Graduate Institute of Pharmacology, National Taiwan University College of Medicine, 100, Taipei, Taiwan
- Cerebellar Research Center, National Taiwan University Hospital, Yun-Lin Branch, Yun-Lin, Taiwan
| | - Sheng-Han Kuo
- Department of Neurology, Columbia University Medical Center, 650 W. 168th St, Rm 305, New York, NY, 10032, USA.
- Initiative for Columbia Ataxia and Tremor, Columbia University Medical Center, New York, NY, USA.
| | - Daphna Shohamy
- Department of Psychology, Columbia University, New York, NY, USA.
- Zuckerman Mind Brain Behavior Institute, Columbia University, Quad 3D, 3227 Broadway, New York, NY, 10027, USA.
- Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| |
Collapse
|
2
|
Qasim SE, Deswal A, Saez I, Gu X. Positive affect modulates memory by regulating the influence of reward prediction errors. COMMUNICATIONS PSYCHOLOGY 2024; 2:52. [PMID: 39242805 PMCID: PMC11332028 DOI: 10.1038/s44271-024-00106-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 05/28/2024] [Indexed: 09/09/2024]
Abstract
How our decisions impact our memories is not well understood. Reward prediction errors (RPEs), the difference between expected and obtained reward, help us learn to make optimal decisions-providing a signal that may influence subsequent memory. To measure this influence and how it might go awry in mood disorders, we recruited a large cohort of human participants to perform a decision-making task in which perceptually memorable stimuli were associated with probabilistic rewards, followed by a recognition test for those stimuli. Computational modeling revealed that positive RPEs enhanced both the accuracy of memory and the temporal efficiency of memory search, beyond the contribution of perceptual information. Critically, positive affect upregulated the beneficial effect of RPEs on memory. These findings demonstrate how affect selectively regulates the impact of RPEs on memory, providing a computational mechanism for biased memory in mood disorders.
Collapse
Affiliation(s)
- Salman E Qasim
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | | | - Ignacio Saez
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Department of Neurosurgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Department of Neurology, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Xiaosi Gu
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
| |
Collapse
|
3
|
Montaser-Kouhsari L, Nicholas J, Gerraty RT, Shohamy D. Two routes to value-based decisions in Parkinson's disease: differentiating incremental reinforcement learning from episodic memory. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.03.592414. [PMID: 38746345 PMCID: PMC11092770 DOI: 10.1101/2024.05.03.592414] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2024]
Abstract
Patients with Parkinson's disease are impaired at incremental reward-based learning. It is typically assumed that this impairment reflects a loss of striatal dopamine. However, many open questions remain about the nature of reward-based learning deficits in Parkinson's. Recent studies have found that a combination of different cognitive and computational strategies contribute even to simple reward-based learning tasks, suggesting a possible role for episodic memory. These findings raise critical questions about how incremental learning and episodic memory interact to support learning from past experience and what their relative contributions are to impaired decision-making in Parkinson's disease. Here we addressed these questions by asking patients with Parkinson's disease (n=26) both on and off their dopamine replacement medication and age- and education-matched healthy controls (n=26) to complete a task designed to isolate the contributions of incremental learning and episodic memory to reward-based learning and decision-making. We found that Parkinson's patients performed as well as healthy controls when using episodic memory, but were impaired at incremental reward-based learning. Dopamine replacement medication remediated this deficit while enhancing subsequent episodic memory for the value of motivationally relevant stimuli. These results demonstrate that Parkinson's patients are impaired at learning about reward from trial-and-error when episodic memory is properly controlled for, and that learning based on the value of single experiences remains intact in patients with Parkinson's disease.
Collapse
|
4
|
Sherman BE, Turk-Browne NB, Goldfarb EV. Multiple Memory Subsystems: Reconsidering Memory in the Mind and Brain. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2024; 19:103-125. [PMID: 37390333 PMCID: PMC10756937 DOI: 10.1177/17456916231179146] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/02/2023]
Abstract
The multiple-memory-systems framework-that distinct types of memory are supported by distinct brain systems-has guided learning and memory research for decades. However, recent work challenges the one-to-one mapping between brain structures and memory types central to this taxonomy, with key memory-related structures supporting multiple functions across substructures. Here we integrate cross-species findings in the hippocampus, striatum, and amygdala to propose an updated framework of multiple memory subsystems (MMSS). We provide evidence for two organizational principles of the MMSS theory: First, opposing memory representations are colocated in the same brain structures; second, parallel memory representations are supported by distinct structures. We discuss why this burgeoning framework has the potential to provide a useful revision of classic theories of long-term memory, what evidence is needed to further validate the framework, and how this novel perspective on memory organization may guide future research.
Collapse
Affiliation(s)
| | | | - Elizabeth V Goldfarb
- Department of Psychology, Yale University
- Wu Tsai Institute, Yale University
- Department of Psychiatry, Yale University
- National Center for PTSD, West Haven, USA
| |
Collapse
|
5
|
Sanders DMW, Cowell RA. The locus of recognition memory signals in human cortex depends on the complexity of the memory representations. Cereb Cortex 2023; 33:9835-9849. [PMID: 37401000 DOI: 10.1093/cercor/bhad248] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Revised: 06/12/2023] [Accepted: 06/14/2023] [Indexed: 07/05/2023] Open
Abstract
According to a "Swiss Army Knife" model of the brain, cognitive functions such as episodic memory and face perception map onto distinct neural substrates. In contrast, representational accounts propose that each brain region is best explained not by which specialized function it performs, but by the type of information it represents with its neural firing. In a functional magnetic resonance imaging study, we asked whether the neural signals supporting recognition memory fall mandatorily within the medial temporal lobes (MTL), traditionally thought the seat of declarative memory, or whether these signals shift within cortex according to the content of the memory. Participants studied objects and scenes that were unique conjunctions of pre-defined visual features. Next, we tested recognition memory in a task that required mnemonic discrimination of both simple features and complex conjunctions. Feature memory signals were strongest in posterior visual regions, declining with anterior progression toward the MTL, while conjunction memory signals followed the opposite pattern. Moreover, feature memory signals correlated with feature memory discrimination performance most strongly in posterior visual regions, whereas conjunction memory signals correlated with conjunction memory discrimination most strongly in anterior sites. Thus, recognition memory signals shifted with changes in memory content, in line with representational accounts.
Collapse
Affiliation(s)
- D Merika W Sanders
- Department of Psychology, Harvard University, Cambridge, MA 02138, United States
| | - Rosemary A Cowell
- Institute of Cognitive Science, University of Colorado Boulder, Boulder, CO 80309, United States
- Department of Psychology & Neuroscience, University of Colorado Boulder, Boulder, CO 80309, United States
| |
Collapse
|
6
|
Bloom PA, Bartlett E, Kathios N, Algharazi S, Siegelman M, Shen F, Beresford L, DiMaggio-Potter ME, Singh A, Bennett S, Natarajan N, Lee H, Sajid S, Joyce E, Fischman R, Hutchinson S, Pan S, Tottenham N, Aly M. Effects of familiar music exposure on deliberate retrieval of remote episodic and semantic memories in healthy aging adults. Memory 2023; 31:428-456. [PMID: 36651851 DOI: 10.1080/09658211.2023.2166078] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/19/2023]
Abstract
Familiar music facilitates memory retrieval in adults with dementia. However, mechanisms behind this effect, and its generality, are unclear because of a lack of parallel work in healthy aging. Exposure to familiar music enhances spontaneous recall of memories directly cued by the music, but it is unknown whether such effects extend to deliberate recall more generally - e.g., to memories not directly linked to the music being played. It is also unclear whether familiar music boosts recall of specific episodes versus more generalised semantic memories, or whether effects are driven by domain-general mechanisms (e.g., improved mood). In a registered report study, we examined effects of familiar music on deliberate recall in healthy adults ages 65-80 years (N = 75) by presenting familiar music from earlier in life, unfamiliar music, and non-musical audio clips across three sessions. After each clip, we assessed free recall of remote memories for pre-selected events. Contrary to our hypotheses, we found no effects of music exposure on recall of prompted events, though familiar music evoked spontaneous memories most often. These results suggest that effects of familiar music on recall may be limited to memories specifically evoked in response to the music (Preprint and registered report protocol at https://osf.io/kjnwd/).
Collapse
Affiliation(s)
| | - Ella Bartlett
- Barnard College of Columbia University, New York, NY, USA
| | | | | | | | - Fan Shen
- Columbia University, New York, NY, USA
| | | | | | | | - Sarah Bennett
- Teachers College, Columbia University, New York, NY, USA
| | | | | | | | - Erin Joyce
- Teachers College, Columbia University, New York, NY, USA
| | | | | | - Sophie Pan
- Barnard College of Columbia University, New York, NY, USA
| | | | | |
Collapse
|
7
|
Pay attention and you might miss it: Greater learning during attentional lapses. Psychon Bull Rev 2022:10.3758/s13423-022-02226-6. [PMID: 36510094 DOI: 10.3758/s13423-022-02226-6] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/21/2022] [Indexed: 12/15/2022]
Abstract
Attentional lapses have been found to impair everything from basic perception to learning and memory. Yet, despite the well-documented costs of lapses on cognition, recent work suggests that lapses might unexpectedly confer some benefits. One potential benefit is that lapses broaden our learning to integrate seemingly irrelevant content that could later prove useful-a benefit that prior research focusing only on goal-relevant memory would miss. Here, we measure how fluctuations in sustained attention influence the learning of seemingly goal-irrelevant content that competes for attention with target content. Participants completed a correlated flanker task in which they categorized central targets (letters or numbers) while ignoring peripheral flanking symbols that shared hidden probabilistic relationships with the targets. We found that across participants, higher rates of attentional lapses correlated with greater learning of the target-flanker relationships. Moreover, within participants, learning was more evident during attentional lapses. These findings address long-standing theoretical debates and reveal a benefit of attentional lapses: they expand the scope of learning and decisions beyond the strictly relevant.
Collapse
|
8
|
Lu Q, Hasson U, Norman KA. A neural network model of when to retrieve and encode episodic memories. eLife 2022; 11:e74445. [PMID: 35142289 PMCID: PMC9000961 DOI: 10.7554/elife.74445] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2021] [Accepted: 02/09/2022] [Indexed: 11/23/2022] Open
Abstract
Recent human behavioral and neuroimaging results suggest that people are selective in when they encode and retrieve episodic memories. To explain these findings, we trained a memory-augmented neural network to use its episodic memory to support prediction of upcoming states in an environment where past situations sometimes reoccur. We found that the network learned to retrieve selectively as a function of several factors, including its uncertainty about the upcoming state. Additionally, we found that selectively encoding episodic memories at the end of an event (but not mid-event) led to better subsequent prediction performance. In all of these cases, the benefits of selective retrieval and encoding can be explained in terms of reducing the risk of retrieving irrelevant memories. Overall, these modeling results provide a resource-rational account of why episodic retrieval and encoding should be selective and lead to several testable predictions.
Collapse
Affiliation(s)
- Qihong Lu
- Department of Psychology, Princeton UniversityPrincetonUnited States
- Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| | - Uri Hasson
- Department of Psychology, Princeton UniversityPrincetonUnited States
- Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| | - Kenneth A Norman
- Department of Psychology, Princeton UniversityPrincetonUnited States
- Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| |
Collapse
|
9
|
Rosenbaum GM, Grassie HL, Hartley CA. Valence biases in reinforcement learning shift across adolescence and modulate subsequent memory. eLife 2022; 11:e64620. [PMID: 35072624 PMCID: PMC8786311 DOI: 10.7554/elife.64620] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2020] [Accepted: 12/24/2021] [Indexed: 12/12/2022] Open
Abstract
As individuals learn through trial and error, some are more influenced by good outcomes, while others weight bad outcomes more heavily. Such valence biases may also influence memory for past experiences. Here, we examined whether valence asymmetries in reinforcement learning change across adolescence, and whether individual learning asymmetries bias the content of subsequent memory. Participants ages 8-27 learned the values of 'point machines,' after which their memory for trial-unique images presented with choice outcomes was assessed. Relative to children and adults, adolescents overweighted worse-than-expected outcomes during learning. Individuals' valence biases modulated incidental memory, such that those who prioritized worse- (or better-) than-expected outcomes during learning were also more likely to remember images paired with these outcomes, an effect reproduced in an independent dataset. Collectively, these results highlight age-related changes in the computation of subjective value and demonstrate that a valence-asymmetric valuation process influences how information is prioritized in episodic memory.
Collapse
Affiliation(s)
- Gail M Rosenbaum
- Department of Psychology, New York UniversityNew YorkUnited States
| | - Hannah L Grassie
- Department of Psychology, New York UniversityNew YorkUnited States
| | - Catherine A Hartley
- Department of Psychology, New York UniversityNew YorkUnited States
- Center for Neural Science, New York UniversityNew YorkUnited States
| |
Collapse
|
10
|
Nicholas J, Daw ND, Shohamy D. Uncertainty alters the balance between incremental learning and episodic memory. eLife 2022; 11:81679. [PMID: 36458809 PMCID: PMC9810331 DOI: 10.7554/elife.81679] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 12/01/2022] [Indexed: 12/04/2022] Open
Abstract
A key question in decision-making is how humans arbitrate between competing learning and memory systems to maximize reward. We address this question by probing the balance between the effects, on choice, of incremental trial-and-error learning versus episodic memories of individual events. Although a rich literature has studied incremental learning in isolation, the role of episodic memory in decision-making has only recently drawn focus, and little research disentangles their separate contributions. We hypothesized that the brain arbitrates rationally between these two systems, relying on each in circumstances to which it is most suited, as indicated by uncertainty. We tested this hypothesis by directly contrasting contributions of episodic and incremental influence to decisions, while manipulating the relative uncertainty of incremental learning using a well-established manipulation of reward volatility. Across two large, independent samples of young adults, participants traded these influences off rationally, depending more on episodic information when incremental summaries were more uncertain. These results support the proposal that the brain optimizes the balance between different forms of learning and memory according to their relative uncertainties and elucidate the circumstances under which episodic memory informs decisions.
Collapse
Affiliation(s)
- Jonathan Nicholas
- Department of Psychology, Columbia UniversityNew YorkUnited States,Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia UniversityNew YorkUnited States
| | - Nathaniel D Daw
- Department of Psychology, Princeton UniversityPrincetonUnited States,Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| | - Daphna Shohamy
- Department of Psychology, Columbia UniversityNew YorkUnited States,Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia UniversityNew YorkUnited States,The Kavli Institute for Brain Science, Columbia UniversityNew YorkUnited States
| |
Collapse
|
11
|
Botvinik-Nezer R, Bakkour A, Salomon T, Shohamy D, Schonberg T. Memory for individual items is related to nonreinforced preference change. ACTA ACUST UNITED AC 2021; 28:348-360. [PMID: 34526380 DOI: 10.1101/lm.053411.121] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2021] [Accepted: 07/14/2021] [Indexed: 01/11/2023]
Abstract
It is commonly assumed that memories contribute to value-based decisions. Nevertheless, most theories of value-based decision-making do not account for memory influences on choice. Recently, new interest has emerged in the interactions between these two fundamental processes, mainly using reinforcement-based paradigms. Here, we aimed to study the role memory processes play in preference change following the nonreinforced cue-approach training (CAT) paradigm. In CAT, the mere association of cued items with a speeded motor response influences choices. Previous studies with this paradigm showed that a single training session induces a long-lasting effect of enhanced preferences for high-value trained stimuli, that is maintained for several months. We hypothesized that CAT increases memory of trained items, leading to enhanced accessibility of their positive associative memories and in turn to preference changes. In two preregistered experiments, we found evidence that memory is enhanced for trained items and that better memory is correlated with enhanced preferences at the individual item level, both immediately and 1 mo following CAT. Our findings suggest that memory plays a central role in value-based decision-making following CAT, even in the absence of external reinforcements. These findings contribute to new theories relating memory and value-based decision-making and set the groundwork for the implementation of novel nonreinforced behavioral interventions that lead to long-lasting behavioral change.
Collapse
Affiliation(s)
- Rotem Botvinik-Nezer
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv 6997801, Israel.,School of Neurobiology, Biochemistry, and Biophysics, Faculty of Life Science, Tel Aviv University, Tel Aviv 6997801, Israel.,Department of Psychological and Brain Sciences, Dartmouth College, Hanover, New Hampshire 03755, USA
| | - Akram Bakkour
- Department of Psychology, Columbia University, New York, New York 10027, USA.,Department of Psychology, the University of Chicago, Chicago, Illinois 60637, USA
| | - Tom Salomon
- School of Neurobiology, Biochemistry, and Biophysics, Faculty of Life Science, Tel Aviv University, Tel Aviv 6997801, Israel
| | - Daphna Shohamy
- Department of Psychology, Columbia University, New York, New York 10027, USA.,Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York 10027, USA.,the Kavli Institute for Brain Science, Columbia University, New York, New York 10027, USA
| | - Tom Schonberg
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv 6997801, Israel.,School of Neurobiology, Biochemistry, and Biophysics, Faculty of Life Science, Tel Aviv University, Tel Aviv 6997801, Israel
| |
Collapse
|
12
|
Biderman N, Shohamy D. Memory and decision making interact to shape the value of unchosen options. Nat Commun 2021; 12:4648. [PMID: 34330909 PMCID: PMC8324852 DOI: 10.1038/s41467-021-24907-x] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2021] [Accepted: 07/08/2021] [Indexed: 02/07/2023] Open
Abstract
The goal of deliberation is to separate between options so that we can commit to one and leave the other behind. However, deliberation can, paradoxically, also form an association in memory between the chosen and unchosen options. Here, we consider this possibility and examine its consequences for how outcomes affect not only the value of the options we chose, but also, by association, the value of options we did not choose. In five experiments (total n = 612), including a preregistered experiment (n = 235), we found that the value assigned to unchosen options is inversely related to their chosen counterparts. Moreover, this inverse relationship was associated with participants' memory of the pairs they chose between. Our findings suggest that deciding between options does not end the competition between them. Deliberation binds choice options together in memory such that the learned value of one can affect the inferred value of the other.
Collapse
Affiliation(s)
- Natalie Biderman
- Department of Psychology and Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia University, New York, NY, USA.
| | - Daphna Shohamy
- Department of Psychology and Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia University, New York, NY, USA.
- The Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| |
Collapse
|
13
|
Recanatesi S, Farrell M, Lajoie G, Deneve S, Rigotti M, Shea-Brown E. Predictive learning as a network mechanism for extracting low-dimensional latent space representations. Nat Commun 2021; 12:1417. [PMID: 33658520 PMCID: PMC7930246 DOI: 10.1038/s41467-021-21696-1] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2019] [Accepted: 01/22/2021] [Indexed: 01/02/2023] Open
Abstract
Artificial neural networks have recently achieved many successes in solving sequential processing and planning tasks. Their success is often ascribed to the emergence of the task’s low-dimensional latent structure in the network activity – i.e., in the learned neural representations. Here, we investigate the hypothesis that a means for generating representations with easily accessed low-dimensional latent structure, possibly reflecting an underlying semantic organization, is through learning to predict observations about the world. Specifically, we ask whether and when network mechanisms for sensory prediction coincide with those for extracting the underlying latent variables. Using a recurrent neural network model trained to predict a sequence of observations we show that network dynamics exhibit low-dimensional but nonlinearly transformed representations of sensory inputs that map the latent structure of the sensory environment. We quantify these results using nonlinear measures of intrinsic dimensionality and linear decodability of latent variables, and provide mathematical arguments for why such useful predictive representations emerge. We focus throughout on how our results can aid the analysis and interpretation of experimental data. Neural networks trained using predictive models generate representations that recover the underlying low-dimensional latent structure in the data. Here, the authors demonstrate that a network trained on a spatial navigation task generates place-related neural activations similar to those observed in the hippocampus and show that these are related to the latent structure.
Collapse
Affiliation(s)
- Stefano Recanatesi
- University of Washington Center for Computational Neuroscience and Swartz Center for Theoretical Neuroscience, Seattle, WA, USA.
| | - Matthew Farrell
- Department of Applied Mathematics, University of Washington, Seattle, WA, USA
| | - Guillaume Lajoie
- Department of Mathematics and Statistics, Université de Montréal, Montreal, QC, Canada.,Mila-Quebec Artificial Intelligence Institute, Montreal, QC, Canada
| | - Sophie Deneve
- Group for Neural Theory, Ecole Normal Superieur, Paris, France
| | | | - Eric Shea-Brown
- University of Washington Center for Computational Neuroscience and Swartz Center for Theoretical Neuroscience, Seattle, WA, USA.,Department of Applied Mathematics, University of Washington, Seattle, WA, USA.,Allen Institute for Brain Science, Seattle, WA, USA
| |
Collapse
|
14
|
Shin YS, DuBrow S. Structuring Memory Through Inference‐Based Event Segmentation. Top Cogn Sci 2020; 13:106-127. [DOI: 10.1111/tops.12505] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2020] [Revised: 03/29/2019] [Accepted: 04/14/2020] [Indexed: 11/28/2022]
|
15
|
Decker AL, Duncan K. Acetylcholine and the complex interdependence of memory and attention. Curr Opin Behav Sci 2020. [DOI: 10.1016/j.cobeha.2020.01.013] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/17/2023]
|