1
|
Rosická AM, Teckentrup V, Fittipaldi S, Ibanez A, Pringle A, Gallagher E, Hanlon AK, Claus N, McCrory C, Lawlor B, Naci L, Gillan CM. Modifiable dementia risk factors associated with objective and subjective cognition. Alzheimers Dement 2024. [PMID: 39382098 DOI: 10.1002/alz.13885] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Revised: 03/28/2024] [Accepted: 04/19/2024] [Indexed: 10/10/2024]
Abstract
INTRODUCTION Early detection of both objective and subjective cognitive impairment is important. Subjective complaints in healthy individuals can precede objective deficits. However, the differential associations of objective and subjective cognition with modifiable dementia risk factors are unclear. METHODS We gathered a large cross-sectional sample (N = 3327, age 18 to 84) via a smartphone app and quantified the associations of 13 risk factors with subjective memory problems and three objective measures of executive function (visual working memory, cognitive flexibility, model-based planning). RESULTS Depression, socioeconomic status, hearing handicap, loneliness, education, smoking, tinnitus, little exercise, small social network, stroke, diabetes, and hypertension were all associated with impairments in at least one cognitive measure. Subjective memory had the strongest link to most factors; these associations persisted after controlling for depression. Age mostly did not moderate these associations. DISCUSSION Subjective cognition was more sensitive to self-report risk factors than objective cognition. Smartphones could facilitate detecting the earliest cognitive impairments. HIGHLIGHTS Smartphone assessments of cognition were sensitive to dementia risk factors. Subjective cognition had stronger links to most factors than did objective cognition. These associations were not fully explained by depression. These associations were largely consistent across the lifespan.
Collapse
Affiliation(s)
| | | | - Sol Fittipaldi
- School of Psychology, Trinity College Dublin, Dublin, Ireland
- Global Brain Health Institute, Trinity College Dublin, Dublin, Ireland
- Latin American Brain Health Institute (BrainLat), Santiago, Chile
| | - Agustin Ibanez
- Global Brain Health Institute, Trinity College Dublin, Dublin, Ireland
- Latin American Brain Health Institute (BrainLat), Santiago, Chile
| | - Andrew Pringle
- School of Psychology, Trinity College Dublin, Dublin, Ireland
| | | | | | - Nathalie Claus
- School of Psychology, Trinity College Dublin, Dublin, Ireland
- Department of Psychology, Chair of Clinical Psychology & Psychological Treatment, LMU Munich, Munich, Germany
| | - Cathal McCrory
- Department of Medical Gerontology, The Irish Longitudinal Study on Ageing, School of Medicine, Trinity College Dublin, Dublin, Ireland
| | - Brian Lawlor
- School of Psychology, Trinity College Dublin, Dublin, Ireland
- Global Brain Health Institute, Trinity College Dublin, Dublin, Ireland
| | - Lorina Naci
- School of Psychology, Trinity College Dublin, Dublin, Ireland
- Global Brain Health Institute, Trinity College Dublin, Dublin, Ireland
| | - Claire M Gillan
- School of Psychology, Trinity College Dublin, Dublin, Ireland
- Global Brain Health Institute, Trinity College Dublin, Dublin, Ireland
| |
Collapse
|
2
|
Mahmoodi A, Luo S, Harbison C, Piray P, Rushworth MFS. Human hippocampus and dorsomedial prefrontal cortex infer and update latent causes during social interaction. Neuron 2024:S0896-6273(24)00649-4. [PMID: 39353432 DOI: 10.1016/j.neuron.2024.09.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Revised: 06/04/2024] [Accepted: 09/03/2024] [Indexed: 10/04/2024]
Abstract
Latent-cause inference is the process of identifying features of the environment that have caused an outcome. This problem is especially important in social settings where individuals may not make equal contributions to the outcomes they achieve together. Here, we designed a novel task in which participants inferred which of two characters was more likely to have been responsible for outcomes achieved by working together. Using computational modeling, univariate and multivariate analysis of human fMRI, and continuous theta-burst stimulation, we identified two brain regions that solved the task. Notably, as each outcome occurred, it was possible to decode the inference of its cause (the responsible character) from hippocampal activity. Activity in dorsomedial prefrontal cortex (dmPFC) updated estimates of association between cause-responsible character-and the outcome. Disruption of dmPFC activity impaired participants' ability to update their estimate as a function of inferred responsibility but spared their ability to infer responsibility.
Collapse
Affiliation(s)
- Ali Mahmoodi
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Shuyi Luo
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Caroline Harbison
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Payam Piray
- Department of Psychology, University of Southern California, Los Angeles, CA, USA
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| |
Collapse
|
3
|
Loosen AM, Kato A, Gu X. Revisiting the role of computational neuroimaging in the era of integrative neuroscience. Neuropsychopharmacology 2024:10.1038/s41386-024-01946-8. [PMID: 39242921 DOI: 10.1038/s41386-024-01946-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/05/2024] [Revised: 07/12/2024] [Accepted: 07/17/2024] [Indexed: 09/09/2024]
Abstract
Computational models have become integral to human neuroimaging research, providing both mechanistic insights and predictive tools for human cognition and behavior. However, concerns persist regarding the ecological validity of lab-based neuroimaging studies and whether their spatiotemporal resolution is not sufficient for capturing neural dynamics. This review aims to re-examine the utility of computational neuroimaging, particularly in light of the growing prominence of alternative neuroscientific methods and the growing emphasis on more naturalistic behaviors and paradigms. Specifically, we will explore how computational modeling can both enhance the analysis of high-dimensional imaging datasets and, conversely, how neuroimaging, in conjunction with other data modalities, can inform computational models through the lens of neurobiological plausibility. Collectively, this evidence suggests that neuroimaging remains critical for human neuroscience research, and when enhanced by computational models, imaging can serve an important role in bridging levels of analysis and understanding. We conclude by proposing key directions for future research, emphasizing the development of standardized paradigms and the integrative use of computational modeling across neuroimaging techniques.
Collapse
Affiliation(s)
- Alisa M Loosen
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
| | - Ayaka Kato
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
| | - Xiaosi Gu
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| |
Collapse
|
4
|
Nitsch A, Garvert MM, Bellmund JLS, Schuck NW, Doeller CF. Grid-like entorhinal representation of an abstract value space during prospective decision making. Nat Commun 2024; 15:1198. [PMID: 38336756 PMCID: PMC10858181 DOI: 10.1038/s41467-024-45127-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Accepted: 01/16/2024] [Indexed: 02/12/2024] Open
Abstract
How valuable a choice option is often changes over time, making the prediction of value changes an important challenge for decision making. Prior studies identified a cognitive map in the hippocampal-entorhinal system that encodes relationships between states and enables prediction of future states, but does not inherently convey value during prospective decision making. In this fMRI study, participants predicted changing values of choice options in a sequence, forming a trajectory through an abstract two-dimensional value space. During this task, the entorhinal cortex exhibited a grid-like representation with an orientation aligned to the axis through the value space most informative for choices. A network of brain regions, including ventromedial prefrontal cortex, tracked the prospective value difference between options. These findings suggest that the entorhinal grid system supports the prediction of future values by representing a cognitive map, which might be used to generate lower-dimensional value signals to guide prospective decision making.
Collapse
Affiliation(s)
- Alexander Nitsch
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.
| | - Mona M Garvert
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Berlin, Germany
- Max Planck UCL Centre for Computational Psychiatry and Aging Research, Berlin, Germany
- Faculty of Human Sciences, Julius-Maximilians-Universität Würzburg, Würzburg, Germany
| | - Jacob L S Bellmund
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Nicolas W Schuck
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Berlin, Germany
- Max Planck UCL Centre for Computational Psychiatry and Aging Research, Berlin, Germany
- Institute of Psychology, Universität Hamburg, Hamburg, Germany
| | - Christian F Doeller
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Jebsen Centre for Alzheimer's Disease, Norwegian University of Science and Technology, Trondheim, Norway.
- Wilhelm Wundt Institute for Psychology, Leipzig University, Leipzig, Germany.
- Department of Psychology, Technical University Dresden, Dresden, Germany.
| |
Collapse
|
5
|
Yang L, Jin F, Yang L, Li J, Li Z, Li M, Shang Z. The Hippocampus in Pigeons Contributes to the Model-Based Valuation and the Relationship between Temporal Context States. Animals (Basel) 2024; 14:431. [PMID: 38338074 PMCID: PMC10854895 DOI: 10.3390/ani14030431] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2023] [Revised: 01/25/2024] [Accepted: 01/25/2024] [Indexed: 02/12/2024] Open
Abstract
Model-based decision-making guides organism behavior by the representation of the relationships between different states. Previous studies have shown that the mammalian hippocampus (Hp) plays a key role in learning the structure of relationships among experiences. However, the hippocampal neural mechanisms of birds for model-based learning have rarely been reported. Here, we trained six pigeons to perform a two-step task and explore whether their Hp contributes to model-based learning. Behavioral performance and hippocampal multi-channel local field potentials (LFPs) were recorded during the task. We estimated the subjective values using a reinforcement learning model dynamically fitted to the pigeon's choice of behavior. The results show that the model-based learner can capture the behavioral choices of pigeons well throughout the learning process. Neural analysis indicated that high-frequency (12-100 Hz) power in Hp represented the temporal context states. Moreover, dynamic correlation and decoding results provided further support for the high-frequency dependence of model-based valuations. In addition, we observed a significant increase in hippocampal neural similarity at the low-frequency band (1-12 Hz) for common temporal context states after learning. Overall, our findings suggest that pigeons use model-based inferences to learn multi-step tasks, and multiple LFP frequency bands collaboratively contribute to model-based learning. Specifically, the high-frequency (12-100 Hz) oscillations represent model-based valuations, while the low-frequency (1-12 Hz) neural similarity is influenced by the relationship between temporal context states. These results contribute to our understanding of the neural mechanisms underlying model-based learning and broaden the scope of hippocampal contributions to avian behavior.
Collapse
Affiliation(s)
- Lifang Yang
- School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001, China; (L.Y.); (F.J.); (L.Y.); (J.L.); (Z.L.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
| | - Fuli Jin
- School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001, China; (L.Y.); (F.J.); (L.Y.); (J.L.); (Z.L.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
| | - Long Yang
- School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001, China; (L.Y.); (F.J.); (L.Y.); (J.L.); (Z.L.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
| | - Jiajia Li
- School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001, China; (L.Y.); (F.J.); (L.Y.); (J.L.); (Z.L.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
| | - Zhihui Li
- School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001, China; (L.Y.); (F.J.); (L.Y.); (J.L.); (Z.L.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
- Institute of Medical Engineering Technology and Data Mining, Zhengzhou University, Zhengzhou 450001, China
| | - Mengmeng Li
- School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001, China; (L.Y.); (F.J.); (L.Y.); (J.L.); (Z.L.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
| | - Zhigang Shang
- School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001, China; (L.Y.); (F.J.); (L.Y.); (J.L.); (Z.L.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
- Institute of Medical Engineering Technology and Data Mining, Zhengzhou University, Zhengzhou 450001, China
| |
Collapse
|
6
|
Sherman BE, Turk-Browne NB, Goldfarb EV. Multiple Memory Subsystems: Reconsidering Memory in the Mind and Brain. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2024; 19:103-125. [PMID: 37390333 PMCID: PMC10756937 DOI: 10.1177/17456916231179146] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/02/2023]
Abstract
The multiple-memory-systems framework-that distinct types of memory are supported by distinct brain systems-has guided learning and memory research for decades. However, recent work challenges the one-to-one mapping between brain structures and memory types central to this taxonomy, with key memory-related structures supporting multiple functions across substructures. Here we integrate cross-species findings in the hippocampus, striatum, and amygdala to propose an updated framework of multiple memory subsystems (MMSS). We provide evidence for two organizational principles of the MMSS theory: First, opposing memory representations are colocated in the same brain structures; second, parallel memory representations are supported by distinct structures. We discuss why this burgeoning framework has the potential to provide a useful revision of classic theories of long-term memory, what evidence is needed to further validate the framework, and how this novel perspective on memory organization may guide future research.
Collapse
Affiliation(s)
| | | | - Elizabeth V Goldfarb
- Department of Psychology, Yale University
- Wu Tsai Institute, Yale University
- Department of Psychiatry, Yale University
- National Center for PTSD, West Haven, USA
| |
Collapse
|
7
|
Wang Y, Guo L, Wang R, Wang Y, Duan F, Zhan Y, Cheng J, Sun X, Tang Z. Abnormal Topological Organization of White Matter Structural Networks in Normal Tension Glaucoma Revealed via Diffusion Tensor Tractography. Brain Sci 2023; 13:1597. [PMID: 38002558 PMCID: PMC10669977 DOI: 10.3390/brainsci13111597] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Revised: 11/10/2023] [Accepted: 11/15/2023] [Indexed: 11/26/2023] Open
Abstract
BACKGROUND Normal tension glaucoma (NTG) is considered a neurodegenerative disease with glaucomatous damage extending to diffuse brain areas. Therefore, this study aims to explore the abnormalities in the NTG structural network to help in the early diagnosis and course evaluation of NTG. METHODS The structural networks of 46 NTG patients and 19 age- and sex-matched healthy controls were constructed using diffusion tensor imaging, followed by graph theory analysis and correlation analysis of small-world properties with glaucoma clinical indicators. In addition, the network-based statistical analysis (NBS) method was used to compare structural network connectivity differences between NTG patients and healthy controls. RESULTS Structural brain networks in both NTG and NC groups exhibited small-world properties. However, the small-world index in the severe NTG group was reduced and correlated with a mean deviation of the visual field (MDVF) and retinal nerve fiber layer (RNFL) thickness. When compared to healthy controls, degree centrality and nodal efficiency in visual brain areas were significantly decreased, and betweenness centrality and nodal local efficiency in both visual and nonvisual brain areas were also significantly altered in NTG patients (all p < 0.05, FDR corrected). Furthermore, NTG patients exhibited increased structural connectivity in the occipitotemporal area, with the left fusiform gyrus (FFG.L) as the hub (p < 0.05). CONCLUSIONS NTG exhibited altered global properties and local properties of visual and cognitive-emotional brain areas, with enhanced structural connections within the occipitotemporal area. Moreover, the disrupted small-world properties of white matter might be imaging biomarkers for assessing NTG progression.
Collapse
Affiliation(s)
- Yin Wang
- Department of Radiology, Eye & ENT Hospital of Fudan University, Fudan University, Shanghai 200031, China (F.D.)
| | - Linying Guo
- Department of Radiology, Eye & ENT Hospital of Fudan University, Fudan University, Shanghai 200031, China (F.D.)
| | - Rong Wang
- Department of Radiology, Huashan Hospital of Fudan University, Fudan University, Shanghai 200040, China
| | - Yuzhe Wang
- Department of Radiology, Zhongshan Hospital of Fudan University, Fudan University, Shanghai 200032, China; (Y.W.)
| | - Fei Duan
- Department of Radiology, Eye & ENT Hospital of Fudan University, Fudan University, Shanghai 200031, China (F.D.)
| | - Yang Zhan
- Department of Radiology, Zhongshan Hospital of Fudan University, Fudan University, Shanghai 200032, China; (Y.W.)
| | - Jingfeng Cheng
- Department of Radiology, Eye & ENT Hospital of Fudan University, Fudan University, Shanghai 200031, China (F.D.)
| | - Xinghuai Sun
- Department of Ophthalmology & Visual Science, Eye & ENT Hospital of Fudan University, Fudan University, Shanghai 200031, China;
| | - Zuohua Tang
- Department of Radiology, Eye & ENT Hospital of Fudan University, Fudan University, Shanghai 200031, China (F.D.)
| |
Collapse
|
8
|
Eppinger B, Ruel A, Bolenz F. Diminished State Space Theory of Human Aging. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2023:17456916231204811. [PMID: 37931229 DOI: 10.1177/17456916231204811] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2023]
Abstract
Many new technologies, such as smartphones, computers, or public-access systems (like ticket-vending machines), are a challenge for older adults. One feature that these technologies have in common is that they involve underlying, partially observable, structures (state spaces) that determine the actions that are necessary to reach a certain goal (e.g., to move from one menu to another, to change a function, or to activate a new service). In this work we provide a theoretical, neurocomputational account to explain these behavioral difficulties in older adults. Based on recent findings from age-comparative computational- and cognitive-neuroscience studies, we propose that age-related impairments in complex goal-directed behavior result from an underlying deficit in the representation of state spaces of cognitive tasks. Furthermore, we suggest that these age-related deficits in adaptive decision-making are due to impoverished neural representations in the orbitofrontal cortex and hippocampus.
Collapse
Affiliation(s)
- Ben Eppinger
- Institute of Psychology, University of Greifswald
- Department of Psychology, Concordia University
- PERFORM Centre, Concordia University
- Faculty of Psychology, Technische Universität Dresden
| | - Alexa Ruel
- Department of Psychology, Concordia University
- PERFORM Centre, Concordia University
- Institute of Psychology, University of Hamburg
| | - Florian Bolenz
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany
- Science of Intelligence/Cluster of Excellence, Technical University of Berlin
| |
Collapse
|
9
|
Schwartenbeck P, Baram A, Liu Y, Mark S, Muller T, Dolan R, Botvinick M, Kurth-Nelson Z, Behrens T. Generative replay underlies compositional inference in the hippocampal-prefrontal circuit. Cell 2023; 186:4885-4897.e14. [PMID: 37804832 PMCID: PMC10914680 DOI: 10.1016/j.cell.2023.09.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 01/23/2023] [Accepted: 09/06/2023] [Indexed: 10/09/2023]
Abstract
Human reasoning depends on reusing pieces of information by putting them together in new ways. However, very little is known about how compositional computation is implemented in the brain. Here, we ask participants to solve a series of problems that each require constructing a whole from a set of elements. With fMRI, we find that representations of novel constructed objects in the frontal cortex and hippocampus are relational and compositional. With MEG, we find that replay assembles elements into compounds, with each replay sequence constituting a hypothesis about a possible configuration of elements. The content of sequences evolves as participants solve each puzzle, progressing from predictable to uncertain elements and gradually converging on the correct configuration. Together, these results suggest a computational bridge between apparently distinct functions of hippocampal-prefrontal circuitry and a role for generative replay in compositional inference and hypothesis testing.
Collapse
Affiliation(s)
- Philipp Schwartenbeck
- University of Tübingen, Tübingen, Germany; Max Planck Institute for Biological Cybernetics, Tübingen, Baden-Württemberg, Germany; Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK.
| | - Alon Baram
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Shirley Mark
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK
| | - Timothy Muller
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK; Institute of Neurology, University College London, London WC1N 3BG, UK
| | - Raymond Dolan
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Department of Psychiatry, Universitätsmedizin Berlin (Campus Charité Mitte), Berlin, Germany
| | - Matthew Botvinick
- Google DeepMind, London, UK; Gatsby Computational Neuroscience Unit, University College London, London, UK
| | - Zeb Kurth-Nelson
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Google DeepMind, London, UK
| | - Timothy Behrens
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK; Sainsbury Wellcome Centre for Neural Circuits and Behaviour, UCL, London W1T 4JG, UK
| |
Collapse
|
10
|
Noh SM, Singla UK, Bennett IJ, Bornstein AM. Memory precision and age differentially predict the use of decision-making strategies across the lifespan. Sci Rep 2023; 13:17014. [PMID: 37813942 PMCID: PMC10562379 DOI: 10.1038/s41598-023-44107-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Accepted: 10/03/2023] [Indexed: 10/11/2023] Open
Abstract
Memory function declines in normal aging, in a relatively continuous fashion following middle-age. The effect of aging on decision-making is less well-understood, with seemingly conflicting results on both the nature and direction of these age effects. One route for clarifying these mixed findings is to understand how age-related differences in memory affect decisions. Recent work has proposed memory sampling as a specific computational role for memory in decision-making, alongside well-studied mechanisms of reinforcement learning (RL). Here, we tested the hypothesis that age-related declines in episodic memory alter memory sampling. Participants (total N = 361; ages 18-77) performed one of two variants of a standard reward-guided decision experiment with additional trial-unique mnemonic content and a separately-administered task for assessing memory precision. When we fit participants' choices with a hybrid computational model implementing both memory-based and RL-driven valuation side-by-side, we found that memory precision tracked the contribution of memory sampling to choice. At the same time, age corresponded to decreasing influence of RL and increasing perseveration. A second experiment confirmed these results and further revealed that memory precision tracked the specificity of memories selected for sampling. Together, these findings suggest that differences in decision-making across the lifespan may be related to memory function, and that interventions which aim to improve the former may benefit from targeting the latter.
Collapse
Affiliation(s)
- Sharon M Noh
- Department of Cognitive Sciences, The University of California, Irvine, Irvine, CA, USA.
| | - Umesh K Singla
- Princeton Neuroscience Institute, Princeton, NJ, USA
- Department of Computer Science and Engineering, University of California, San Diego, La Jolla, CA, USA
| | - Ilana J Bennett
- Department of Psychology, The University of California, Riverside, Riverside, CA, USA
| | - Aaron M Bornstein
- Department of Cognitive Sciences, The University of California, Irvine, Irvine, CA, USA.
- Center for the Neurobiology of Learning and Memory, The University of California, Irvine, Irvine, CA, USA.
| |
Collapse
|
11
|
Mehrotra D, Dubé L. Accounting for multiscale processing in adaptive real-world decision-making via the hippocampus. Front Neurosci 2023; 17:1200842. [PMID: 37732307 PMCID: PMC10508350 DOI: 10.3389/fnins.2023.1200842] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Accepted: 08/25/2023] [Indexed: 09/22/2023] Open
Abstract
For adaptive real-time behavior in real-world contexts, the brain needs to allow past information over multiple timescales to influence current processing for making choices that create the best outcome as a person goes about making choices in their everyday life. The neuroeconomics literature on value-based decision-making has formalized such choice through reinforcement learning models for two extreme strategies. These strategies are model-free (MF), which is an automatic, stimulus-response type of action, and model-based (MB), which bases choice on cognitive representations of the world and causal inference on environment-behavior structure. The emphasis of examining the neural substrates of value-based decision making has been on the striatum and prefrontal regions, especially with regards to the "here and now" decision-making. Yet, such a dichotomy does not embrace all the dynamic complexity involved. In addition, despite robust research on the role of the hippocampus in memory and spatial learning, its contribution to value-based decision making is just starting to be explored. This paper aims to better appreciate the role of the hippocampus in decision-making and advance the successor representation (SR) as a candidate mechanism for encoding state representations in the hippocampus, separate from reward representations. To this end, we review research that relates hippocampal sequences to SR models showing that the implementation of such sequences in reinforcement learning agents improves their performance. This also enables the agents to perform multiscale temporal processing in a biologically plausible manner. Altogether, we articulate a framework to advance current striatal and prefrontal-focused decision making to better account for multiscale mechanisms underlying various real-world time-related concepts such as the self that cumulates over a person's life course.
Collapse
Affiliation(s)
- Dhruv Mehrotra
- Integrated Program in Neuroscience, McGill University, Montréal, QC, Canada
- Montréal Neurological Institute, McGill University, Montréal, QC, Canada
| | - Laurette Dubé
- Desautels Faculty of Management, McGill University, Montréal, QC, Canada
- McGill Center for the Convergence of Health and Economics, McGill University, Montréal, QC, Canada
| |
Collapse
|
12
|
Wise T, Charpentier CJ, Dayan P, Mobbs D. Interactive cognitive maps support flexible behavior under threat. Cell Rep 2023; 42:113008. [PMID: 37610871 PMCID: PMC10658881 DOI: 10.1016/j.celrep.2023.113008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 07/11/2023] [Accepted: 08/03/2023] [Indexed: 08/25/2023] Open
Abstract
In social environments, survival can depend upon inferring and adapting to other agents' goal-directed behavior. However, it remains unclear how humans achieve this, despite the fact that many decisions must account for complex, dynamic agents acting according to their own goals. Here, we use a predator-prey task (total n = 510) to demonstrate that humans exploit an interactive cognitive map of the social environment to infer other agents' preferences and simulate their future behavior, providing for flexible, generalizable responses. A model-based inverse reinforcement learning model explained participants' inferences about threatening agents' preferences, with participants using this inferred knowledge to enact generalizable, model-based behavioral responses. Using tree-search planning models, we then found that behavior was best explained by a planning algorithm that incorporated simulations of the threat's goal-directed behavior. Our results indicate that humans use a cognitive map to determine other agents' preferences, facilitating generalized predictions of their behavior and effective responses.
Collapse
Affiliation(s)
- Toby Wise
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK; Department of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA.
| | - Caroline J Charpentier
- Department of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA; Department of Psychology, University of Maryland, College Park, MD, USA; Brain and Behavior Institute, University of Maryland, College Park, MD, USA
| | - Peter Dayan
- Max Planck Institute for Biological Cybernetics, Tübingen, Germany; University of Tübingen, Tübingen, Germany
| | - Dean Mobbs
- Department of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA; Computation and Neural Systems Program, California Institute of Technology, Pasadena, CA, USA
| |
Collapse
|
13
|
Miller AMP, Jacob AD, Ramsaran AI, De Snoo ML, Josselyn SA, Frankland PW. Emergence of a predictive model in the hippocampus. Neuron 2023; 111:1952-1965.e5. [PMID: 37015224 PMCID: PMC10293047 DOI: 10.1016/j.neuron.2023.03.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Revised: 01/23/2023] [Accepted: 03/08/2023] [Indexed: 04/05/2023]
Abstract
The brain organizes experiences into memories that guide future behavior. Hippocampal CA1 population activity is hypothesized to reflect predictive models that contain information about future events, but little is known about how they develop. We trained mice on a series of problems with or without a common statistical structure to observe how memories are formed and updated. Mice that learned structured problems integrated their experiences into a predictive model that contained the solutions to upcoming novel problems. Retrieving the model during learning improved discrimination accuracy and facilitated learning. Using calcium imaging to track CA1 activity during learning, we found that hippocampal ensemble activity became more stable as mice formed a predictive model. The hippocampal ensemble was reactivated during training and incorporated new activity patterns from each training problem. These results show how hippocampal activity supports building predictive models by organizing new information with respect to existing memories.
Collapse
Affiliation(s)
- Adam M P Miller
- Program in Neurosciences and Mental Health, The Hospital for Sick Children, Toronto, ON, Canada
| | - Alex D Jacob
- Program in Neurosciences and Mental Health, The Hospital for Sick Children, Toronto, ON, Canada; Department of Psychology, University of Toronto, Toronto, ON, Canada
| | - Adam I Ramsaran
- Program in Neurosciences and Mental Health, The Hospital for Sick Children, Toronto, ON, Canada; Department of Psychology, University of Toronto, Toronto, ON, Canada
| | - Mitchell L De Snoo
- Program in Neurosciences and Mental Health, The Hospital for Sick Children, Toronto, ON, Canada; Institute of Medical Sciences, University of Toronto, Toronto, ON, Canada
| | - Sheena A Josselyn
- Program in Neurosciences and Mental Health, The Hospital for Sick Children, Toronto, ON, Canada; Department of Psychology, University of Toronto, Toronto, ON, Canada; Department of Physiology, University of Toronto, Toronto, ON, Canada; Institute of Medical Sciences, University of Toronto, Toronto, ON, Canada; Brain, Mind, & Consciousness Program, Canadian Institute for Advanced Research, Toronto, ON, Canada
| | - Paul W Frankland
- Program in Neurosciences and Mental Health, The Hospital for Sick Children, Toronto, ON, Canada; Department of Psychology, University of Toronto, Toronto, ON, Canada; Department of Physiology, University of Toronto, Toronto, ON, Canada; Institute of Medical Sciences, University of Toronto, Toronto, ON, Canada; Child & Brain Development Program, Canadian Institute for Advanced Research, Toronto, ON, Canada.
| |
Collapse
|
14
|
Zhu SL, Lakshminarasimhan KJ, Angelaki DE. Computational cross-species views of the hippocampal formation. Hippocampus 2023; 33:586-599. [PMID: 37038890 PMCID: PMC10947336 DOI: 10.1002/hipo.23535] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 03/17/2023] [Accepted: 03/21/2023] [Indexed: 04/12/2023]
Abstract
The discovery of place cells and head direction cells in the hippocampal formation of freely foraging rodents has led to an emphasis of its role in encoding allocentric spatial relationships. In contrast, studies in head-fixed primates have additionally found representations of spatial views. We review recent experiments in freely moving monkeys that expand upon these findings and show that postural variables such as eye/head movements strongly influence neural activity in the hippocampal formation, suggesting that the function of the hippocampus depends on where the animal looks. We interpret these results in the light of recent studies in humans performing challenging navigation tasks which suggest that depending on the context, eye/head movements serve one of two roles-gathering information about the structure of the environment (active sensing) or externalizing the contents of internal beliefs/deliberation (embodied cognition). These findings prompt future experimental investigations into the information carried by signals flowing between the hippocampal formation and the brain regions controlling postural variables, and constitute a basis for updating computational theories of the hippocampal system to accommodate the influence of eye/head movements.
Collapse
Affiliation(s)
- Seren L Zhu
- Center for Neural Science, New York University, New York, New York, USA
| | - Kaushik J Lakshminarasimhan
- Center for Theoretical Neuroscience, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York, USA
| | - Dora E Angelaki
- Center for Neural Science, New York University, New York, New York, USA
- Mechanical and Aerospace Engineering, Tandon School of Engineering, New York University, New York, New York, USA
| |
Collapse
|
15
|
Wise T, Robinson OJ, Gillan CM. Identifying Transdiagnostic Mechanisms in Mental Health Using Computational Factor Modeling. Biol Psychiatry 2023; 93:690-703. [PMID: 36725393 PMCID: PMC10017264 DOI: 10.1016/j.biopsych.2022.09.034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Revised: 09/09/2022] [Accepted: 09/27/2022] [Indexed: 02/03/2023]
Abstract
Most psychiatric disorders do not occur in isolation, and most psychiatric symptom dimensions are not uniquely expressed within a single diagnostic category. Current treatments fail to work for around 25% to 40% of individuals, perhaps due at least in part to an overreliance on diagnostic categories in treatment development and allocation. In this review, we describe ongoing efforts in the field to surmount these challenges and precisely characterize psychiatric symptom dimensions using large-scale studies of unselected samples via remote, online, and "citizen science" efforts that take a dimensional, mechanistic approach. We discuss the importance that efforts to identify meaningful psychiatric dimensions be coupled with careful computational modeling to formally specify, test, and potentially falsify candidate mechanisms that underlie transdiagnostic symptom dimensions. We refer to this approach, i.e., where symptom dimensions are identified and validated against computationally well-defined neurocognitive processes, as computational factor modeling. We describe in detail some recent applications of this method to understand transdiagnostic cognitive processes that include model-based planning, metacognition, appetitive processing, and uncertainty estimation. In this context, we highlight how computational factor modeling has been used to identify specific associations between cognition and symptom dimensions and reveal previously obscured relationships, how findings generalize to smaller in-person clinical and nonclinical samples, and how the method is being adapted and optimized beyond its original instantiation. Crucially, we discuss next steps for this area of research, highlighting the value of more direct investigations of treatment response that bridge the gap between basic research and the clinic.
Collapse
Affiliation(s)
- Toby Wise
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom
| | - Oliver J Robinson
- Neuroscience and Mental Health Group, Institute of Cognitive Neuroscience, University College London, London, United Kingdom; Research Department of Clinical Education and Health Psychology, University College London, London, United Kingdom
| | - Claire M Gillan
- School of Psychology, Trinity College Dublin, Dublin 2, Ireland; Global Brain Health Institute, Trinity College Dublin, Dublin 2, Ireland; Trinity College Institute of Neuroscience, Trinity College Dublin, Dublin 2, Ireland.
| |
Collapse
|
16
|
Harhen NC, Bornstein AM. Overharvesting in human patch foraging reflects rational structure learning and adaptive planning. Proc Natl Acad Sci U S A 2023; 120:e2216524120. [PMID: 36961923 PMCID: PMC10068834 DOI: 10.1073/pnas.2216524120] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Accepted: 02/11/2023] [Indexed: 03/26/2023] Open
Abstract
Patch foraging presents a sequential decision-making problem widely studied across organisms-stay with a current option or leave it in search of a better alternative? Behavioral ecology has identified an optimal strategy for these decisions, but, across species, foragers systematically deviate from it, staying too long with an option or "overharvesting" relative to this optimum. Despite the ubiquity of this behavior, the mechanism underlying it remains unclear and an object of extensive investigation. Here, we address this gap by approaching foraging as both a decision-making and learning problem. Specifically, we propose a model in which foragers 1) rationally infer the structure of their environment and 2) use their uncertainty over the inferred structure representation to adaptively discount future rewards. We find that overharvesting can emerge from this rational statistical inference and uncertainty adaptation process. In a patch-leaving task, we show that human participants adapt their foraging to the richness and dynamics of the environment in ways consistent with our model. These findings suggest that definitions of optimal foraging could be extended by considering how foragers reduce and adapt to uncertainty over representations of their environment.
Collapse
Affiliation(s)
- Nora C. Harhen
- Department of Cognitive Sciences, University of California, Irvine, CA92697
| | - Aaron M. Bornstein
- Department of Cognitive Sciences, University of California, Irvine, CA92697
- Center for the Neurobiology of Learning and Memory, University of California, Irvine, CA92697
| |
Collapse
|
17
|
Nissan N, Hertz U, Shahar N, Gabay Y. Distinct reinforcement learning profiles distinguish between language and attentional neurodevelopmental disorders. BEHAVIORAL AND BRAIN FUNCTIONS : BBF 2023; 19:6. [PMID: 36941632 PMCID: PMC10029183 DOI: 10.1186/s12993-023-00207-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Received: 07/31/2022] [Accepted: 01/26/2023] [Indexed: 03/23/2023]
Abstract
BACKGROUND Theoretical models posit abnormalities in cortico-striatal pathways in two of the most common neurodevelopmental disorders (Developmental dyslexia, DD, and Attention deficit hyperactive disorder, ADHD), but it is still unclear what distinct cortico-striatal dysfunction might distinguish language disorders from others that exhibit very different symptomatology. Although impairments in tasks that depend on the cortico-striatal network, including reinforcement learning (RL), have been implicated in both disorders, there has been little attempt to dissociate between different types of RL or to compare learning processes in these two types of disorders. The present study builds upon prior research indicating the existence of two learning manifestations of RL and evaluates whether these processes can be differentiated in language and attention deficit disorders. We used a two-step RL task shown to dissociate model-based from model-free learning in human learners. RESULTS Our results show that, relative to neurotypicals, DD individuals showed an impairment in model-free but not in model-based learning, whereas in ADHD the ability to use both model-free and model-based learning strategies was significantly compromised. CONCLUSIONS Thus, learning impairments in DD may be linked to a selective deficit in the ability to form action-outcome associations based on previous history, whereas in ADHD some learning deficits may be related to an incapacity to pursue rewards based on the tasks' structure. Our results indicate how different patterns of learning deficits may underlie different disorders, and how computation-minded experimental approaches can differentiate between them.
Collapse
Affiliation(s)
- Noyli Nissan
- Department of Special Education, University of Haifa, Haifa, Israel
- Edmond J. Safra Brain Research Center for the Study of Learning Disabilities, University of Haifa, 199 Abba Khoushy Ave, Haifa, Israel
| | - Uri Hertz
- Department of Cognitive Sciences, University of Haifa, Haifa, Israel
| | - Nitzan Shahar
- The School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
| | - Yafit Gabay
- Department of Special Education, University of Haifa, Haifa, Israel.
- Edmond J. Safra Brain Research Center for the Study of Learning Disabilities, University of Haifa, 199 Abba Khoushy Ave, Haifa, Israel.
| |
Collapse
|
18
|
Ruel A, Bolenz F, Li SC, Fischer A, Eppinger B. Neural evidence for age-related deficits in the representation of state spaces. Cereb Cortex 2023; 33:1768-1781. [PMID: 35510942 DOI: 10.1093/cercor/bhac171] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Revised: 04/08/2022] [Accepted: 04/09/2022] [Indexed: 11/12/2022] Open
Abstract
Under high cognitive demands, older adults tend to resort to simpler, habitual, or model-free decision strategies. This age-related shift in decision behavior has been attributed to deficits in the representation of the cognitive maps, or state spaces, necessary for more complex model-based decision-making. Yet, the neural mechanisms behind this shift remain unclear. In this study, we used a modified 2-stage Markov task in combination with computational modeling and single-trial EEG analyses to establish neural markers of age-related changes in goal-directed decision-making under different demands on the representation of state spaces. Our results reveal that the shift to simpler decision strategies in older adults is due to (i) impairments in the representation of the transition structure of the task and (ii) a diminished signaling of the reward value associated with decision options. In line with the diminished state space hypothesis of human aging, our findings suggest that deficits in goal-directed, model-based behavior in older adults result from impairments in the representation of state spaces of cognitive tasks.
Collapse
Affiliation(s)
- Alexa Ruel
- Department of Psychology, Concordia University, 7141 Sherbrooke Street W. Montreal, Quebec H4B 1R6, Canada
| | - Florian Bolenz
- Chair of Lifespan Developmental Neuroscience, Faculty of Psychology, Technische Universität Dresden, Bürogebäude, Zi. 244 Strehlener Straße 22/24, Dresden 01069, Germany.,Center for Adaptive Rationality, Max Planck Institute for Human Development, Lentzeallee 94, Berlin 14195, Germany.,Cluster of Excellence "Science of Intelligence", Technische Universität Berlin, Straße des 17. Juni 135, Berlin 10623, Germany
| | - Shu-Chen Li
- Chair of Lifespan Developmental Neuroscience, Faculty of Psychology, Technische Universität Dresden, Bürogebäude, Zi. 244 Strehlener Straße 22/24, Dresden 01069, Germany.,Cluster of Excellence "Centre for Tactile Internet with Human-in-the-Loop", Technische Universität Dresden, Bürogebäude, Zi. 244 Strehlener Straße 22/24, Dresden 01069, Germany
| | - Adrian Fischer
- Department of Education and Psychology, Freie Universität Berlin, Habelschwerdter Allee 45, Berlin 14195, Germany
| | - Ben Eppinger
- Department of Psychology, Concordia University, 7141 Sherbrooke Street W. Montreal, Quebec H4B 1R6, Canada.,Chair of Lifespan Developmental Neuroscience, Faculty of Psychology, Technische Universität Dresden, Bürogebäude, Zi. 244 Strehlener Straße 22/24, Dresden 01069, Germany.,PERFORM Center, Concordia University, 7200 Sherbrooke St. W. Montreal, Quebec H4B 1R6, Canada
| |
Collapse
|
19
|
History of suicide attempt associated with amygdala and hippocampus changes among individuals with schizophrenia. Eur Arch Psychiatry Clin Neurosci 2023:10.1007/s00406-023-01554-5. [PMID: 36788147 DOI: 10.1007/s00406-023-01554-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Accepted: 01/09/2023] [Indexed: 02/16/2023]
Abstract
Abnormalities in subcortical brain structures may reflect higher suicide risk in mood disorders, but less is known about its associations for schizophrenia. This cross-sectional imaging study aimed to explore whether the history of suicide attempts was associated with subcortical changes among individuals with schizophrenia. We recruited 44 individuals with schizophrenia and a history of suicide attempts (SZ-SA) and 44 individuals with schizophrenia but without a history of suicide attempts (SZ-NSA) and 44 healthy controls. Linear regression showed that SZ-SA had smaller volumes of the hippocampus (Cohen's d = -0.72), the amygdala (Cohen's d = -0.69), and some nuclei of the amygdala (Cohen's d, -0.57 to -0.72) than SZ-NSA after adjusting for age, sex, illness phase, and intracranial volume. There was no difference in the volume of the subfields of the hippocampus. It suggests the history of suicide attempts is associated with subcortical volume alterations in schizophrenia.
Collapse
|
20
|
Fan CL, Sokolowski HM, Rosenbaum RS, Levine B. What about "space" is important for episodic memory? WILEY INTERDISCIPLINARY REVIEWS. COGNITIVE SCIENCE 2023; 14:e1645. [PMID: 36772875 DOI: 10.1002/wcs.1645] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Revised: 01/16/2023] [Accepted: 01/18/2023] [Indexed: 02/12/2023]
Abstract
Early cognitive neuroscientific research revealed that the hippocampus is crucial for spatial navigation in rodents, and for autobiographical episodic memory in humans. Researchers quickly linked these streams to propose that the human hippocampus supports memory through its role in representing space, and research on the link between spatial cognition and episodic memory in humans has proliferated over the past several decades. Different researchers apply the term "spatial" in a variety of contexts, however, and it remains unclear what aspect of space may be critical to memory. Similarly, "episodic" has been defined and tested in different ways. Naturalistic assessment of spatial memory and episodic memory (i.e., episodic autobiographical memory) is required to unify the scale and biological relevance in comparisons of spatial and mnemonic processing. Limitations regarding the translation of rodent to human research, human ontogeny, and inter-individual variability require greater consideration in the interpretation of this literature. In this review, we outline the aspects of space that are (and are not) commonly linked to episodic memory, and then we discuss these dimensions through the lens of individual differences in naturalistic autobiographical memory. Future studies should carefully consider which aspect(s) of space are being linked to memory within the context of naturalistic human cognition. This article is categorized under: Psychology > Memory.
Collapse
Affiliation(s)
- Carina L Fan
- Department of Psychology, University of Toronto, Toronto, Ontario, Canada.,Rotman Research Institute, Baycrest, Toronto, Ontario, Canada
| | | | - R Shayna Rosenbaum
- Rotman Research Institute, Baycrest, Toronto, Ontario, Canada.,Department of Psychology, York University, Toronto, Ontario, Canada
| | - Brian Levine
- Department of Psychology, University of Toronto, Toronto, Ontario, Canada.,Rotman Research Institute, Baycrest, Toronto, Ontario, Canada.,Department of Medicine, Neurology, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
21
|
Wimmer GE, Liu Y, McNamee DC, Dolan RJ. Distinct replay signatures for prospective decision-making and memory preservation. Proc Natl Acad Sci U S A 2023; 120:e2205211120. [PMID: 36719914 PMCID: PMC9963918 DOI: 10.1073/pnas.2205211120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 12/05/2022] [Indexed: 02/01/2023] Open
Abstract
Theories of neural replay propose that it supports a range of functions, most prominently planning and memory consolidation. Here, we test the hypothesis that distinct signatures of replay in the same task are related to model-based decision-making ("planning") and memory preservation. We designed a reward learning task wherein participants utilized structure knowledge for model-based evaluation, while at the same time had to maintain knowledge of two independent and randomly alternating task environments. Using magnetoencephalography and multivariate analysis, we first identified temporally compressed sequential reactivation, or replay, both prior to choice and following reward feedback. Before choice, prospective replay strength was enhanced for the current task-relevant environment when a model-based planning strategy was beneficial. Following reward receipt, and consistent with a memory preservation role, replay for the alternative distal task environment was enhanced as a function of decreasing recency of experience with that environment. Critically, these planning and memory preservation relationships were selective to pre-choice and post-feedback periods, respectively. Our results provide support for key theoretical proposals regarding the functional role of replay and demonstrate that the relative strength of planning and memory-related signals are modulated by ongoing computational and task demands.
Collapse
Affiliation(s)
- G. Elliott Wimmer
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, LondonWC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3BG, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
- Chinese Institute for Brain Research, Beijing100875, China
| | - Daniel C. McNamee
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, LondonWC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3BG, UK
- Neuroscience Programme, Champalimaud Research, Lisbon1400-038, Portugal
| | - Raymond J. Dolan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, LondonWC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3BG, UK
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| |
Collapse
|
22
|
Foundations of human spatial problem solving. Sci Rep 2023; 13:1485. [PMID: 36707649 PMCID: PMC9883268 DOI: 10.1038/s41598-023-28834-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Accepted: 01/25/2023] [Indexed: 01/28/2023] Open
Abstract
Despite great strides in both machine learning and neuroscience, we do not know how the human brain solves problems in the general sense. We approach this question by drawing on the framework of engineering control theory. We demonstrate a computational neural model with only localist learning laws that is able to find solutions to arbitrary problems. The model and humans perform a multi-step task with arbitrary and changing starting and desired ending states. Using a combination of computational neural modeling, human fMRI, and representational similarity analysis, we show here that the roles of a number of brain regions can be reinterpreted as interacting mechanisms of a control theoretic system. The results suggest a new set of functional perspectives on the orbitofrontal cortex, hippocampus, basal ganglia, anterior temporal lobe, lateral prefrontal cortex, and visual cortex, as well as a new path toward artificial general intelligence.
Collapse
|
23
|
Mathar D, Erfanian Abdoust M, Marrenbach T, Tuzsus D, Peters J. The catecholamine precursor Tyrosine reduces autonomic arousal and decreases decision thresholds in reinforcement learning and temporal discounting. PLoS Comput Biol 2022; 18:e1010785. [PMID: 36548401 PMCID: PMC9822114 DOI: 10.1371/journal.pcbi.1010785] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Revised: 01/06/2023] [Accepted: 12/01/2022] [Indexed: 12/24/2022] Open
Abstract
Supplementation with the catecholamine precursor L-Tyrosine might enhance cognitive performance, but overall findings are mixed. Here, we investigate the effect of a single dose of tyrosine (2g) vs. placebo on two catecholamine-dependent trans-diagnostic traits: model-based control during reinforcement learning (2-step task) and temporal discounting, using a double-blind, placebo-controlled, within-subject design (n = 28 healthy male participants). We leveraged drift diffusion models in a hierarchical Bayesian framework to jointly model participants' choices and response times (RTS) in both tasks. Furthermore, comprehensive autonomic monitoring (heart rate, heart rate variability, pupillometry, spontaneous eye blink rate) was performed both pre- and post-supplementation, to explore potential physiological effects of supplementation. Across tasks, tyrosine consistently reduced participants' RTs without deteriorating task-performance. Diffusion modeling linked this effect to attenuated decision-thresholds in both tasks and further revealed increased model-based control (2-step task) and (if anything) attenuated temporal discounting. On the physiological level, participants' pupil dilation was predictive of the individual degree of temporal discounting. Tyrosine supplementation reduced physiological arousal as revealed by increases in pupil dilation variability and reductions in heart rate. Supplementation-related changes in physiological arousal predicted individual changes in temporal discounting. Our findings provide first evidence that tyrosine supplementation might impact psychophysiological parameters, and suggest that modeling approaches based on sequential sampling models can yield novel insights into latent cognitive processes modulated by amino-acid supplementation.
Collapse
Affiliation(s)
- David Mathar
- Department of Psychology, Biological Psychology, University of Cologne, Cologne, Germany
| | - Mani Erfanian Abdoust
- Biological Psychology of Decision Making, Institute of Experimental Psychology, Heinrich Heine University Duesseldorf, Duesseldorf, Germany
| | - Tobias Marrenbach
- Biological Psychology of Decision Making, Institute of Experimental Psychology, Heinrich Heine University Duesseldorf, Duesseldorf, Germany
| | - Deniz Tuzsus
- Department of Psychology, Biological Psychology, University of Cologne, Cologne, Germany
| | - Jan Peters
- Department of Psychology, Biological Psychology, University of Cologne, Cologne, Germany
| |
Collapse
|
24
|
Abstract
In reinforcement learning (RL) experiments, participants learn to make rewarding choices in response to different stimuli; RL models use outcomes to estimate stimulus-response values that change incrementally. RL models consider any response type indiscriminately, ranging from more concretely defined motor choices (pressing a key with the index finger), to more general choices that can be executed in a number of ways (selecting dinner at the restaurant). However, does the learning process vary as a function of the choice type? In Experiment 1, we show that it does: Participants were slower and less accurate in learning correct choices of a general format compared with learning more concrete motor actions. Using computational modeling, we show that two mechanisms contribute to this. First, there was evidence of irrelevant credit assignment: The values of motor actions interfered with the values of other choice dimensions, resulting in more incorrect choices when the correct response was not defined by a single motor action; second, information integration for relevant general choices was slower. In Experiment 2, we replicated and further extended the findings from Experiment 1 by showing that slowed learning was attributable to weaker working memory use, rather than slowed RL. In both experiments, we ruled out the explanation that the difference in performance between two condition types was driven by difficulty/different levels of complexity. We conclude that defining a more abstract choice space used by multiple learning systems for credit assignment recruits executive resources, limiting how much such processes then contribute to fast learning.
Collapse
Affiliation(s)
| | - Amy Zou
- University of California, Berkeley
| | - Anne G E Collins
- University of California, Berkeley
- Helen Wills Neuroscience Institute Berkeley, CA
| |
Collapse
|
25
|
Eckstein MK, Master SL, Xia L, Dahl RE, Wilbrecht L, Collins AGE. The interpretation of computational model parameters depends on the context. eLife 2022; 11:e75474. [PMID: 36331872 PMCID: PMC9635876 DOI: 10.7554/elife.75474] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Accepted: 09/09/2022] [Indexed: 11/06/2022] Open
Abstract
Reinforcement Learning (RL) models have revolutionized the cognitive and brain sciences, promising to explain behavior from simple conditioning to complex problem solving, to shed light on developmental and individual differences, and to anchor cognitive processes in specific brain mechanisms. However, the RL literature increasingly reveals contradictory results, which might cast doubt on these claims. We hypothesized that many contradictions arise from two commonly-held assumptions about computational model parameters that are actually often invalid: That parameters generalize between contexts (e.g. tasks, models) and that they capture interpretable (i.e. unique, distinctive) neurocognitive processes. To test this, we asked 291 participants aged 8-30 years to complete three learning tasks in one experimental session, and fitted RL models to each. We found that some parameters (exploration / decision noise) showed significant generalization: they followed similar developmental trajectories, and were reciprocally predictive between tasks. Still, generalization was significantly below the methodological ceiling. Furthermore, other parameters (learning rates, forgetting) did not show evidence of generalization, and sometimes even opposite developmental trajectories. Interpretability was low for all parameters. We conclude that the systematic study of context factors (e.g. reward stochasticity; task volatility) will be necessary to enhance the generalizability and interpretability of computational cognitive models.
Collapse
Affiliation(s)
| | - Sarah L Master
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States
- Department of Psychology, New York UniversityNew YorkUnited States
| | - Liyu Xia
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States
- Department of Mathematics, University of California, BerkeleyBerkeleyUnited States
| | - Ronald E Dahl
- Institute of Human Development, University of California, BerkeleyBerkeleyUnited States
| | - Linda Wilbrecht
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States
- Helen Wills Neuroscience Institute, University of California, BerkeleyBerkeleyUnited States
| | - Anne GE Collins
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States
- Helen Wills Neuroscience Institute, University of California, BerkeleyBerkeleyUnited States
| |
Collapse
|
26
|
Polti I, Nau M, Kaplan R, van Wassenhove V, Doeller CF. Rapid encoding of task regularities in the human hippocampus guides sensorimotor timing. eLife 2022; 11:e79027. [PMID: 36317500 PMCID: PMC9625083 DOI: 10.7554/elife.79027] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Accepted: 10/02/2022] [Indexed: 11/17/2022] Open
Abstract
The brain encodes the statistical regularities of the environment in a task-specific yet flexible and generalizable format. Here, we seek to understand this process by bridging two parallel lines of research, one centered on sensorimotor timing, and the other on cognitive mapping in the hippocampal system. By combining functional magnetic resonance imaging (fMRI) with a fast-paced time-to-contact (TTC) estimation task, we found that the hippocampus signaled behavioral feedback received in each trial as well as performance improvements across trials along with reward-processing regions. Critically, it signaled performance improvements independent from the tested intervals, and its activity accounted for the trial-wise regression-to-the-mean biases in TTC estimation. This is in line with the idea that the hippocampus supports the rapid encoding of temporal context even on short time scales in a behavior-dependent manner. Our results emphasize the central role of the hippocampus in statistical learning and position it at the core of a brain-wide network updating sensorimotor representations in real time for flexible behavior.
Collapse
Affiliation(s)
- Ignacio Polti
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Jebsen Centre for Alzheimer’s Disease, Norwegian University of Science and TechnologyTrondheimNorway
- Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
| | - Matthias Nau
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Jebsen Centre for Alzheimer’s Disease, Norwegian University of Science and TechnologyTrondheimNorway
- Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
| | - Raphael Kaplan
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Jebsen Centre for Alzheimer’s Disease, Norwegian University of Science and TechnologyTrondheimNorway
- Department of Basic Psychology, Clinical Psychology, and Psychobiology, Universitat Jaume ICastellón de la PlanaSpain
| | - Virginie van Wassenhove
- CEA DRF/Joliot, NeuroSpin; INSERM, Cognitive Neuroimaging Unit; CNRS, Université Paris-SaclayGif-Sur-YvetteFrance
| | - Christian F Doeller
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Jebsen Centre for Alzheimer’s Disease, Norwegian University of Science and TechnologyTrondheimNorway
- Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
- Wilhelm Wundt Institute of Psychology, Leipzig UniversityLeipzigGermany
| |
Collapse
|
27
|
Vijayabaskaran S, Cheng S. Navigation task and action space drive the emergence of egocentric and allocentric spatial representations. PLoS Comput Biol 2022; 18:e1010320. [PMID: 36315587 PMCID: PMC9648855 DOI: 10.1371/journal.pcbi.1010320] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Revised: 11/10/2022] [Accepted: 10/18/2022] [Indexed: 11/12/2022] Open
Abstract
In general, strategies for spatial navigation could employ one of two spatial reference frames: egocentric or allocentric. Notwithstanding intuitive explanations, it remains unclear however under what circumstances one strategy is chosen over another, and how neural representations should be related to the chosen strategy. Here, we first use a deep reinforcement learning model to investigate whether a particular type of navigation strategy arises spontaneously during spatial learning without imposing a bias onto the model. We then examine the spatial representations that emerge in the network to support navigation. To this end, we study two tasks that are ethologically valid for mammals—guidance, where the agent has to navigate to a goal location fixed in allocentric space, and aiming, where the agent navigates to a visible cue. We find that when both navigation strategies are available to the agent, the solutions it develops for guidance and aiming are heavily biased towards the allocentric or the egocentric strategy, respectively, as one might expect. Nevertheless, the agent can learn both tasks using either type of strategy. Furthermore, we find that place-cell-like allocentric representations emerge preferentially in guidance when using an allocentric strategy, whereas egocentric vector representations emerge when using an egocentric strategy in aiming. We thus find that alongside the type of navigational strategy, the nature of the task plays a pivotal role in the type of spatial representations that emerge. Most species rely on navigation in space to find water, food, and mates, as well as to return home. When navigating, humans and animals can use one of two reference frames: one based on stable landmarks in the external environment, such as moving due north and then east, or one centered on oneself, such as moving forward and turning left. However, it remains unclear how these reference frames are chosen and interact in navigation tasks, as well as how they are supported by representations in the brain. We therefore modeled two navigation tasks that would each benefit from using one of these reference frames, and trained an artificial agent to learn to solve them through trial and error. Our results show that when given the choice, the agent leveraged the appropriate reference frame to solve the task, but surprisingly could also use the other reference frame when constrained to do so. We also show that the representations that emerge to enable the agent to solve the tasks exist on a spectrum, and are more complex than commonly thought. These representations reflect both the task and reference frame being used, and provide useful insights for the design of experimental tasks to study the use of navigational strategies.
Collapse
Affiliation(s)
| | - Sen Cheng
- Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany
| |
Collapse
|
28
|
de Cothi W, Nyberg N, Griesbauer EM, Ghanamé C, Zisch F, Lefort JM, Fletcher L, Newton C, Renaudineau S, Bendor D, Grieves R, Duvelle É, Barry C, Spiers HJ. Predictive maps in rats and humans for spatial navigation. Curr Biol 2022; 32:3676-3689.e5. [PMID: 35863351 PMCID: PMC9616735 DOI: 10.1016/j.cub.2022.06.090] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Revised: 05/19/2022] [Accepted: 06/29/2022] [Indexed: 11/25/2022]
Abstract
Much of our understanding of navigation comes from the study of individual species, often with specific tasks tailored to those species. Here, we provide a novel experimental and analytic framework integrating across humans, rats, and simulated reinforcement learning (RL) agents to interrogate the dynamics of behavior during spatial navigation. We developed a novel open-field navigation task ("Tartarus maze") requiring dynamic adaptation (shortcuts and detours) to frequently changing obstructions on the path to a hidden goal. Humans and rats were remarkably similar in their trajectories. Both species showed the greatest similarity to RL agents utilizing a "successor representation," which creates a predictive map. Humans also displayed trajectory features similar to model-based RL agents, which implemented an optimal tree-search planning procedure. Our results help refine models seeking to explain mammalian navigation in dynamic environments and highlight the utility of modeling the behavior of different species to uncover the shared mechanisms that support behavior.
Collapse
Affiliation(s)
- William de Cothi
- Department of Cell and Developmental Biology, University College London, London, UK; Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK.
| | - Nils Nyberg
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Eva-Maria Griesbauer
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Carole Ghanamé
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Fiona Zisch
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK; The Bartlett School of Architecture, University College London, London, UK
| | - Julie M Lefort
- Department of Cell and Developmental Biology, University College London, London, UK
| | - Lydia Fletcher
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Coco Newton
- Department of Clinical Neurosciences, University of Cambridge, Cambridge, UK
| | - Sophie Renaudineau
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Daniel Bendor
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Roddy Grieves
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Éléonore Duvelle
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Caswell Barry
- Department of Cell and Developmental Biology, University College London, London, UK
| | - Hugo J Spiers
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK.
| |
Collapse
|
29
|
Bo O'Connor B, Fowler Z. How Imagination and Memory Shape the Moral Mind. PERSONALITY AND SOCIAL PSYCHOLOGY REVIEW 2022; 27:226-249. [PMID: 36062349 DOI: 10.1177/10888683221114215] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Interdisciplinary research has proposed a multifaceted view of human cognition and morality, establishing that inputs from multiple cognitive and affective processes guide moral decisions. However, extant work on moral cognition has largely overlooked the contributions of episodic representation. The ability to remember or imagine a specific moment in time plays a broadly influential role in cognition and behavior. Yet, existing research has only begun exploring the influence of episodic representation on moral cognition. Here, we evaluate the theoretical connections between episodic representation and moral cognition, review emerging empirical work revealing how episodic representation affects moral decision-making, and conclude by highlighting gaps in the literature and open questions. We argue that a comprehensive model of moral cognition will require including the episodic memory system, further delineating its direct influence on moral thought, and better understanding its interactions with other mental processes to fundamentally shape our sense of right and wrong.
Collapse
Affiliation(s)
| | - Zoë Fowler
- University at Albany, State University of New York, USA
| |
Collapse
|
30
|
Effects of Different Lipopolysaccharide Doses on Short- and Long-Term Spatial Memory and Hippocampus Morphology in an Experimental Alzheimer’s Disease Model. CLINICAL AND TRANSLATIONAL NEUROSCIENCE 2022. [DOI: 10.3390/ctn6030020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Background: Alzheimer’s disease (AD) is a progressive neurodegenerative disease and the most common cause of dementia. Various animal models are widely used to investigate its underlying mechanisms, including lipopolysaccharide (LPS)-induced neuroinflammation models. Aim: In this study, we aimed to investigate the effect of different doses (0.25, 0.5, and 0.75 mg/kg) of LPS on short- and long-term spatial memory and hippocampal morphology in an experimental AD mouse model. Materials and methods: Twenty-four adult male Swiss mice (SWR/J) weighing 18–25 g were divided into four groups: control, 0.25 mg/kg LPS, 0.50 mg/kg LPS, and 0.75 mg/kg LPS. All groups were treated with LPS or vehicle for 7 days. Behavioral tests were started (Morris water maze for 6 days and Y maze for 1 day) on the last 2 days of injections. After the behavioral procedures, tissues were collected for further histological investigations. Result: All LPS doses induced significant short- and long-term spatial memory impairment in both the Y maze and Morris water maze compared with the control group. Furthermore, histological examination of the hippocampus indicated degenerating neurons in both the 0.50 mg/kg and 0.75 mg/kg LPS groups, while the 0.25 mg/kg LPS group showed less degeneration. Conclusion: our results showed that 0.75 mg/kg LPS had a greater impact on early-stage spatial learning memory and short-term memory than other doses. Our behavioral and histological findings suggest 0.75 mg/kg LPS as a promising dose for LPS-induced AD models.
Collapse
|
31
|
A comparison of reinforcement learning models of human spatial navigation. Sci Rep 2022; 12:13923. [PMID: 35978035 PMCID: PMC9385652 DOI: 10.1038/s41598-022-18245-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2022] [Accepted: 08/08/2022] [Indexed: 11/09/2022] Open
Abstract
Reinforcement learning (RL) models have been influential in characterizing human learning and decision making, but few studies apply them to characterizing human spatial navigation and even fewer systematically compare RL models under different navigation requirements. Because RL can characterize one's learning strategies quantitatively and in a continuous manner, and one's consistency of using such strategies, it can provide a novel and important perspective for understanding the marked individual differences in human navigation and disentangle navigation strategies from navigation performance. One-hundred and fourteen participants completed wayfinding tasks in a virtual environment where different phases manipulated navigation requirements. We compared performance of five RL models (3 model-free, 1 model-based and 1 "hybrid") at fitting navigation behaviors in different phases. Supporting implications from prior literature, the hybrid model provided the best fit regardless of navigation requirements, suggesting the majority of participants rely on a blend of model-free (route-following) and model-based (cognitive mapping) learning in such navigation scenarios. Furthermore, consistent with a key prediction, there was a correlation in the hybrid model between the weight on model-based learning (i.e., navigation strategy) and the navigator's exploration vs. exploitation tendency (i.e., consistency of using such navigation strategy), which was modulated by navigation task requirements. Together, we not only show how computational findings from RL align with the spatial navigation literature, but also reveal how the relationship between navigation strategy and a person's consistency using such strategies changes as navigation requirements change.
Collapse
|
32
|
Glitz L, Juechems K, Summerfield C, Garrett N. Model Sharing in the Human Medial Temporal Lobe. J Neurosci 2022; 42:5410-5426. [PMID: 35606146 PMCID: PMC7613027 DOI: 10.1523/jneurosci.1978-21.2022] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Revised: 04/20/2022] [Accepted: 04/23/2022] [Indexed: 11/21/2022] Open
Abstract
Effective planning involves knowing where different actions take us. However, natural environments are rich and complex, leading to an exponential increase in memory demand as a plan grows in depth. One potential solution is to filter out features of the environment irrelevant to the task at hand. This enables a shared model of transition dynamics to be used for planning over a range of different input features. Here, we asked human participants (13 male, 16 female) to perform a sequential decision-making task, designed so that knowledge should be integrated independently of the input features (visual cues) present in one case but not in another. Participants efficiently switched between using a low-dimensional (cue independent) and a high-dimensional (cue specific) representation of state transitions. fMRI data identified the medial temporal lobe as a locus for learning state transitions. Within this region, multivariate patterns of BOLD responses were less correlated between trials with differing input features but similar state associations in the high dimensional than in the low dimensional case, suggesting that these patterns switched between separable (specific to input features) and shared (invariant to input features) transition models. Finally, we show that transition models are updated more strongly following the receipt of positive compared with negative outcomes, a finding that challenges conventional theories of planning. Together, these findings propose a computational and neural account of how information relevant for planning can be shared and segmented in response to the vast array of contextual features we encounter in our world.SIGNIFICANCE STATEMENT Effective planning involves maintaining an accurate model of which actions take us to which locations. But in a world awash with information, mapping actions to states with the right level of complexity is critical. Using a new decision-making "heist task" in conjunction with computational modeling and fMRI, we show that patterns of BOLD responses in the medial temporal lobe-a brain region key for prospective planning-become less sensitive to the presence of visual features when these are irrelevant to the task at hand. By flexibly adapting the complexity of task-state representations in this way, state-action mappings learned under one set of features can be used to plan in the presence of others.
Collapse
Affiliation(s)
- Leonie Glitz
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6HG, United Kingdom
| | - Keno Juechems
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6HG, United Kingdom
| | | | - Neil Garrett
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6HG, United Kingdom
- School of Psychology, University of East Anglia, Norwich NR4 7TJ, United Kingdom
| |
Collapse
|
33
|
Zou X, Yang H, Li Q, Li N, Hou Y, Wang X, Meng X, Yu J, Zhang Y, Tang C, Kuang T. Protective Effect of Brassica rapa Polysaccharide against Acute High-Altitude Hypoxia-Induced Brain Injury and Its Metabolomics. OXIDATIVE MEDICINE AND CELLULAR LONGEVITY 2022; 2022:3063899. [PMID: 39282147 PMCID: PMC11401678 DOI: 10.1155/2022/3063899] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/08/2021] [Revised: 02/24/2022] [Accepted: 03/15/2022] [Indexed: 09/18/2024]
Abstract
Brassica rapa L., a traditional Tibetan medicine, has been wildly used for treating plateau disease. Polysaccharide is an important chemical component in B. rapa. The present study aimed to evaluate the effect of B. rapa polysaccharide (BRP) against acute high-altitude hypoxia (AHH) induced brain injury and its metabolic mechanism. The rats were randomly divided into six groups: control group, AHH group, Hongjingtian oral liquid group, and three BRP groups (38, 75, and 150 mg/kg/d). Serum levels of superoxide dismutase (SOD), malondialdehyde (MDA), glutathione (GSH), oxidized glutathione (GSSG), and lactate dehydrogenase (LDH) were detected by commercial biochemical kits. Hippocampus and cortex histopathological changes were observed by H&E staining and Nissl staining. Neuronal apoptosis was observed by TUNEL staining. The protein and gene expression of Caspase-3, Bax, Bcl-2, p-PI3K, PI3K, p-Akt, Akt, HIF-1α, microRNA 210, ISCU1/2, and COX10 were detected by western blotting and qRT-PCR. Then, a brain metabolomics method based on UPLC-Q-Exactive-MS was performed to discover potential biomarkers and analyze metabolic pathways. It was found that BRP decreased levels of MDA, LDH, and GSSG, increased GSH and SOD, reduced the pathological changes, inhibited apoptosis, and activated the PI3K/Akt/HIF-1α signaling pathway as evidenced by increased phosphorylation of PI3K and Akt, enhanced protein expression of HIF-1α and gene levels of microRNA210, ISCU1/2, and COX10. Furthermore, 15 endogenous potential biomarkers were identified in the brain through metabolomics analysis. BRP can regulate 7 potential biomarkers and the corresponding metabolic pathways were mainly associated with pyruvate metabolism and glycolysis/gluconeogenesis. Collectively, BRP has a clear protective effect on AHH-induced brain injury and its mechanisms may be related to ameliorate oxidative stress injury, inhibit apoptosis by activating PI3K/Akt/HIF-1α signaling pathway, and reverse metabolic pathway disturbances.
Collapse
Affiliation(s)
- Xuemei Zou
- State Key Laboratory of Southwestern Chinese Medicine Resources, School of Ethnic Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu 611137, China
| | - Hailing Yang
- Innovative Institute of Chinese Medicine and Pharmacy, Chengdu University of Traditional Chinese Medicine, Chengdu 611137, China
| | - Qiuyue Li
- Pharmacy Intravenous Admixture Services, The Affiliated Traditional Chinese Medicine Hospital of Southwest Medical University, Luzhou 646600, China
| | - Ning Li
- State Key Laboratory of Southwestern Chinese Medicine Resources, School of Ethnic Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu 611137, China
| | - Ya Hou
- School of Pharmacy, Chengdu University of Traditional Chinese Medicine, Chengdu 611137, China
| | - Xiaobo Wang
- Innovative Institute of Chinese Medicine and Pharmacy, Chengdu University of Traditional Chinese Medicine, Chengdu 611137, China
| | - Xianli Meng
- Innovative Institute of Chinese Medicine and Pharmacy, Chengdu University of Traditional Chinese Medicine, Chengdu 611137, China
| | - Jia Yu
- State Key Laboratory of Southwestern Chinese Medicine Resources, School of Ethnic Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu 611137, China
| | - Yi Zhang
- State Key Laboratory of Southwestern Chinese Medicine Resources, School of Ethnic Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu 611137, China
| | - Ce Tang
- State Key Laboratory of Southwestern Chinese Medicine Resources, School of Ethnic Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu 611137, China
| | - Tingting Kuang
- State Key Laboratory of Southwestern Chinese Medicine Resources, School of Ethnic Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu 611137, China
| |
Collapse
|
34
|
Kinley I, Amlung M, Becker S. Pathologies of precision: A Bayesian account of goals, habits, and episodic foresight in addiction. Brain Cogn 2022; 158:105843. [DOI: 10.1016/j.bandc.2022.105843] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Revised: 01/02/2022] [Accepted: 01/08/2022] [Indexed: 12/20/2022]
|
35
|
Rmus M, Ritz H, Hunter LE, Bornstein AM, Shenhav A. Humans can navigate complex graph structures acquired during latent learning. Cognition 2022; 225:105103. [PMID: 35364400 PMCID: PMC9201735 DOI: 10.1016/j.cognition.2022.105103] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2021] [Revised: 03/09/2022] [Accepted: 03/20/2022] [Indexed: 11/03/2022]
Abstract
Humans appear to represent many forms of knowledge in associative networks whose nodes are multiply connected, including sensory, spatial, and semantic. Recent work has shown that explicitly augmenting artificial agents with such graph-structured representations endows them with more human-like capabilities of compositionality and transfer learning. An open question is how humans acquire these representations. Previously, it has been shown that humans can learn to navigate graph-structured conceptual spaces on the basis of direct experience with trajectories that intentionally draw the network contours (Schapiro, Kustner, & Turk-Browne, 2012; Schapiro, Turk-Browne, Botvinick, & Norman, 2016), or through direct experience with rewards that covary with the underlying associative distance (Wu, Schulz, Speekenbrink, Nelson, & Meder, 2018). Here, we provide initial evidence that this capability is more general, extending to learning to reason about shortest-path distances across a graph structure acquired across disjoint experiences with randomized edges of the graph - a form of latent learning. In other words, we show that humans can infer graph structures, assembling them from disordered experiences. We further show that the degree to which individuals learn to reason correctly and with reference to the structure of the graph corresponds to their propensity, in a separate task, to use model-based reinforcement learning to achieve rewards. This connection suggests that the correct acquisition of graph-structured relationships is a central ability underlying forward planning and reasoning, and may be a core computation across the many domains in which graph-based reasoning is advantageous.
Collapse
Affiliation(s)
- Milena Rmus
- Department of Psychology, University of California, Berkeley, USA.
| | - Harrison Ritz
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, USA
| | | | - Aaron M Bornstein
- Department of Cognitive Sciences, University of California, Irvine, USA; Center for the Neurobiology of Learning and Memory, University of California, Irvine, USA
| | - Amitai Shenhav
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, USA; Carney Institute for Brain Science, Brown University, USA
| |
Collapse
|
36
|
Abstract
In human neuroscience, studies of cognition are rarely grounded in non-task-evoked, 'spontaneous' neural activity. Indeed, studies of spontaneous activity tend to focus predominantly on intrinsic neural patterns (for example, resting-state networks). Taking a 'representation-rich' approach bridges the gap between cognition and resting-state communities: this approach relies on decoding task-related representations from spontaneous neural activity, allowing quantification of the representational content and rich dynamics of such activity. For example, if we know the neural representation of an episodic memory, we can decode its subsequent replay during rest. We argue that such an approach advances cognitive research beyond a focus on immediate task demand and provides insight into the functional relevance of the intrinsic neural pattern (for example, the default mode network). This in turn enables a greater integration between human and animal neuroscience, facilitating experimental testing of theoretical accounts of intrinsic activity, and opening new avenues of research in psychiatry.
Collapse
|
37
|
Abstract
Recent breakthroughs in artificial intelligence (AI) have enabled machines to plan in tasks previously thought to be uniquely human. Meanwhile, the planning algorithms implemented by the brain itself remain largely unknown. Here, we review neural and behavioral data in sequential decision-making tasks that elucidate the ways in which the brain does-and does not-plan. To systematically review available biological data, we create a taxonomy of planning algorithms by summarizing the relevant design choices for such algorithms in AI. Across species, recording techniques, and task paradigms, we find converging evidence that the brain represents future states consistent with a class of planning algorithms within our taxonomy-focused, depth-limited, and serial. However, we argue that current data are insufficient for addressing more detailed algorithmic questions. We propose a new approach leveraging AI advances to drive experiments that can adjudicate between competing candidate algorithms.
Collapse
|
38
|
Brunec IK, Momennejad I. Predictive Representations in Hippocampal and Prefrontal Hierarchies. J Neurosci 2022; 42:299-312. [PMID: 34799416 PMCID: PMC8802932 DOI: 10.1523/jneurosci.1327-21.2021] [Citation(s) in RCA: 29] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Revised: 10/19/2021] [Accepted: 10/22/2021] [Indexed: 11/21/2022] Open
Abstract
As we navigate the world, we use learned representations of relational structures to explore and to reach goals. Studies of how relational knowledge enables inference and planning are typically conducted in controlled small-scale settings. It remains unclear, however, how people use stored knowledge in continuously unfolding navigation (e.g., walking long distances in a city). We hypothesized that multiscale predictive representations guide naturalistic navigation in humans, and these scales are organized along posterior-anterior prefrontal and hippocampal hierarchies. We conducted model-based representational similarity analyses of neuroimaging data collected while male and female participants navigated realistically long paths in virtual reality. We tested the pattern similarity of each point, along each path, to a weighted sum of its successor points within predictive horizons of different scales. We found that anterior PFC showed the largest predictive horizons, posterior hippocampus the smallest, with the anterior hippocampus and orbitofrontal regions in between. Our findings offer novel insights into how cognitive maps support hierarchical planning at multiple scales.SIGNIFICANCE STATEMENT Whenever we navigate the world, we represent our journey at multiple horizons: from our immediate surroundings to our distal goal. How are such cognitive maps at different horizons simultaneously represented in the brain? Here, we applied a reinforcement learning-based analysis to neuroimaging data acquired while participants virtually navigated their hometown. We investigated neural patterns in the hippocampus and PFC, key cognitive map regions. We uncovered predictive representations with multiscale horizons in prefrontal and hippocampal gradients, with the longest predictive horizons in anterior PFC and the shortest in posterior hippocampus. These findings provide empirical support for the computational hypothesis that multiscale neural representations guide goal-directed navigation. This advances our understanding of hierarchical planning in everyday navigation of realistic distances.
Collapse
Affiliation(s)
- Iva K Brunec
- Department of Psychology, University of Pennsylvania, Philadelphia, Pennsylvania 19104
| | | |
Collapse
|
39
|
Dombrovski AY, Hallquist MN. Search for solutions, learning, simulation, and choice processes in suicidal behavior. WILEY INTERDISCIPLINARY REVIEWS. COGNITIVE SCIENCE 2022; 13:e1561. [PMID: 34008338 PMCID: PMC9285563 DOI: 10.1002/wcs.1561] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/12/2020] [Revised: 03/06/2021] [Accepted: 04/07/2021] [Indexed: 12/25/2022]
Abstract
Suicide may be viewed as an unfortunate outcome of failures in decision processes. Such failures occur when the demands of a crisis exceed a person's capacity to (i) search for options, (ii) learn and simulate possible futures, and (iii) make advantageous value-based choices. Can individual-level decision deficits and biases drive the progression of the suicidal crisis? Our overview of the evidence on this question is informed by clinical theory and grounded in reinforcement learning and behavioral economics. Cohort and case-control studies provide strong evidence that limited cognitive capacity and particularly impaired cognitive control are associated with suicidal behavior, imposing cognitive constraints on decision-making. We conceptualize suicidal ideation as an element of impoverished consideration sets resulting from a search for solutions under cognitive constraints and mood-congruent Pavlovian influences, a view supported by mostly indirect evidence. More compelling is the evidence of impaired learning in people with a history of suicidal behavior. We speculate that an inability to simulate alternative futures using one's model of the world may undermine alternative solutions in a suicidal crisis. The hypothesis supported by the strongest evidence is that the selection of suicide over alternatives is facilitated by a choice process undermined by randomness. Case-control studies using gambling tasks, armed bandits, and delay discounting support this claim. Future experimental studies will need to uncover real-time dynamics of choice processes in suicidal people. In summary, the decision process framework sheds light on neurocognitive mechanisms that facilitate the progression of the suicidal crisis. This article is categorized under: Economics > Individual Decision-Making Psychology > Emotion and Motivation Psychology > Learning Neuroscience > Behavior.
Collapse
Affiliation(s)
| | - Michael N. Hallquist
- Department of Psychology and NeuroscienceUniversity of North CarolinaChapel HillNorth CarolinaUSA
| |
Collapse
|
40
|
Collins AGE, Shenhav A. Advances in modeling learning and decision-making in neuroscience. Neuropsychopharmacology 2022; 47:104-118. [PMID: 34453117 PMCID: PMC8617262 DOI: 10.1038/s41386-021-01126-y] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/03/2021] [Revised: 07/14/2021] [Accepted: 07/22/2021] [Indexed: 02/07/2023]
Abstract
An organism's survival depends on its ability to learn about its environment and to make adaptive decisions in the service of achieving the best possible outcomes in that environment. To study the neural circuits that support these functions, researchers have increasingly relied on models that formalize the computations required to carry them out. Here, we review the recent history of computational modeling of learning and decision-making, and how these models have been used to advance understanding of prefrontal cortex function. We discuss how such models have advanced from their origins in basic algorithms of updating and action selection to increasingly account for complexities in the cognitive processes required for learning and decision-making, and the representations over which they operate. We further discuss how a deeper understanding of the real-world complexities in these computations has shed light on the fundamental constraints on optimal behavior, and on the complex interactions between corticostriatal pathways to determine such behavior. The continuing and rapid development of these models holds great promise for understanding the mechanisms by which animals adapt to their environments, and what leads to maladaptive forms of learning and decision-making within clinical populations.
Collapse
Affiliation(s)
- Anne G E Collins
- Department of Psychology and Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA.
| | - Amitai Shenhav
- Department of Cognitive, Linguistic, & Psychological Sciences and Carney Institute for Brain Science, Brown University, Providence, RI, USA.
| |
Collapse
|
41
|
Averbeck B, O'Doherty JP. Reinforcement-learning in fronto-striatal circuits. Neuropsychopharmacology 2022; 47:147-162. [PMID: 34354249 PMCID: PMC8616931 DOI: 10.1038/s41386-021-01108-0] [Citation(s) in RCA: 43] [Impact Index Per Article: 21.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Revised: 07/06/2021] [Accepted: 07/09/2021] [Indexed: 01/03/2023]
Abstract
We review the current state of knowledge on the computational and neural mechanisms of reinforcement-learning with a particular focus on fronto-striatal circuits. We divide the literature in this area into five broad research themes: the target of the learning-whether it be learning about the value of stimuli or about the value of actions; the nature and complexity of the algorithm used to drive the learning and inference process; how learned values get converted into choices and associated actions; the nature of state representations, and of other cognitive machinery that support the implementation of various reinforcement-learning operations. An emerging fifth area focuses on how the brain allocates or arbitrates control over different reinforcement-learning sub-systems or "experts". We will outline what is known about the role of the prefrontal cortex and striatum in implementing each of these functions. We then conclude by arguing that it will be necessary to build bridges from algorithmic level descriptions of computational reinforcement-learning to implementational level models to better understand how reinforcement-learning emerges from multiple distributed neural networks in the brain.
Collapse
Affiliation(s)
| | - John P O'Doherty
- Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA.
| |
Collapse
|
42
|
Hunter LE, Meer EA, Gillan CM, Hsu M, Daw ND. Increased and biased deliberation in social anxiety. Nat Hum Behav 2022; 6:146-154. [PMID: 34400815 PMCID: PMC9849449 DOI: 10.1038/s41562-021-01180-y] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2019] [Accepted: 07/08/2021] [Indexed: 02/07/2023]
Abstract
A goal of computational psychiatry is to ground symptoms in basic mechanisms. Theory suggests that avoidance in anxiety disorders may reflect dysregulated mental simulation, a process for evaluating candidate actions. If so, these covert processes should have observable consequences: choices reflecting increased and biased deliberation. In two online general population samples, we examined how self-report symptoms of social anxiety disorder predict choices in a socially framed reinforcement learning task, the patent race, in which the pattern of choices reflects the content of deliberation. Using a computational model to assess learning strategy, we found that self-report social anxiety was indeed associated with increased deliberative evaluation. This effect was stronger for a particular subset of feedback ('upward counterfactual') in one of the experiments, broadly matching the biased content of rumination in social anxiety disorder, and robust to controlling for other psychiatric symptoms. These results suggest a grounding of symptoms of social anxiety disorder in more basic neuro-computational mechanisms.
Collapse
Affiliation(s)
- Lindsay E Hunter
- Department of Psychology, Princeton University, Princeton, NJ, USA.
| | - Elana A Meer
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Claire M Gillan
- School of Psychology, Trinity College Dublin, Dublin, Ireland
- Trinity College Institute of Neuroscience, Trinity College Dublin, Dublin, Ireland
- Global Brain Health Institute, Trinity College Dublin, Dublin, Ireland
| | - Ming Hsu
- Haas School of Business, University of California, Berkeley, Berkeley, CA, USA
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
| | - Nathaniel D Daw
- Department of Psychology, Princeton University, Princeton, NJ, USA.
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
| |
Collapse
|
43
|
Fernández RS, Crivelli L, Pedreira ME, Allegri RF, Correale J. Computational basis of decision-making impairment in multiple sclerosis. Mult Scler 2021; 28:1267-1276. [PMID: 34931933 DOI: 10.1177/13524585211059308] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
BACKGROUND Multiple sclerosis (MS) is commonly associated with decision-making, neurocognitive impairments, and mood and motivational symptoms. However, their relationship may be obscured by traditional scoring methods. OBJECTIVES To study the computational basis underlying decision-making impairments in MS and their interaction with neurocognitive and neuropsychiatric measures. METHODS Twenty-nine MS patients and 26 matched control subjects completed a computer version of the Iowa Gambling Task (IGT). Participants underwent neurocognitive evaluation using an expanded version of the Brief Repeatable Battery. Hierarchical Bayesian Analysis was used to estimate three established computational models to compare parameters between groups. RESULTS Patients showed increased learning rate and reduced loss-aversion during decision-making relative to control subjects. These alterations were associated with: (1) reduced net gains in the IGT; (2) processing speed, executive functioning and memory impairments; and (3) higher levels of depression and current apathy. CONCLUSION Decision-making deficits in MS patients could be described by the interplay between latent computational processes, neurocognitive impairments, and mood/motivational symptoms.
Collapse
Affiliation(s)
- Rodrigo S Fernández
- Instituto de Fisiología, Biología Molecular y Neurociencias (IFIBYNE-CONICET), Ciudad de Buenos Aires, Argentina/Laboratorio de Neurociencia de la Memoria, IFIBYNE-CONICET, Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires, Ciudad de Buenos Aires, Argentina
| | - Lucia Crivelli
- Department of Cognitive Neurology, Neuropsychiatry and Neuropsychology, Fleni, Buenos Aires, Argentina/Department of Neurology, Fleni, Buenos Aires, Argentina
| | - María E Pedreira
- Instituto de Fisiología, Biología Molecular y Neurociencias (IFIBYNE-CONICET), Ciudad de Buenos Aires, Argentina/Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires, Ciudad de Buenos Aires, Argentina
| | - Ricardo F Allegri
- Department of Cognitive Neurology, Neuropsychiatry and Neuropsychology, Fleni, Buenos Aires, Argentina/Department of Neurology, Fleni, Buenos Aires, Argentina/Universidad de la Costa (CUC), Barranquilla, Colombia
| | | |
Collapse
|
44
|
Wang S, Feng SF, Bornstein AM. Mixing memory and desire: How memory reactivation supports deliberative decision-making. WILEY INTERDISCIPLINARY REVIEWS. COGNITIVE SCIENCE 2021; 13:e1581. [PMID: 34665529 DOI: 10.1002/wcs.1581] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Revised: 08/24/2021] [Accepted: 09/16/2021] [Indexed: 11/09/2022]
Abstract
Memories affect nearly every aspect of our mental life. They allow us to both resolve uncertainty in the present and to construct plans for the future. Recently, renewed interest in the role memory plays in adaptive behavior has led to new theoretical advances and empirical observations. We review key findings, with particular emphasis on how the retrieval of many kinds of memories affects deliberative action selection. These results are interpreted in a sequential inference framework, in which reinstatements from memory serve as "samples" of potential action outcomes. The resulting model suggests a central role for the dynamics of memory reactivation in determining the influence of different kinds of memory in decisions. We propose that representation-specific dynamics can implement a bottom-up "product of experts" rule that integrates multiple sets of action-outcome predictions weighted based on their uncertainty. We close by reviewing related findings and identifying areas for further research. This article is categorized under: Psychology > Reasoning and Decision Making Neuroscience > Cognition Neuroscience > Computation.
Collapse
Affiliation(s)
- Shaoming Wang
- Department of Psychology, New York University, New York, New York, USA
| | - Samuel F Feng
- Department of Mathematics, Khalifa University of Science and Technology, Abu Dhabi, UAE.,Khalifa University Centre for Biotechnology, Khalifa University of Science and Technology, Abu Dhabi, UAE
| | - Aaron M Bornstein
- Department of Cognitive Sciences, University of California-Irvine, Irvine, California, USA.,Center for the Neurobiology of Learning & Memory, University of California-Irvine, Irvine, California, USA.,Institute for Mathematical Behavioral Sciences, University of California-Irvine, Irvine, California, USA
| |
Collapse
|
45
|
Bolenz F, Eppinger B. Valence bias in metacontrol of decision making in adolescents and young adults. Child Dev 2021; 93:e103-e116. [PMID: 34655226 DOI: 10.1111/cdev.13693] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
The development of metacontrol of decision making and its susceptibility to framing effects were investigated in a sample of 201 adolescents and adults in Germany (12-25 years, 111 female, ethnicity not recorded). In a task that dissociates model-free and model-based decision making, outcome magnitude and outcome valence were manipulated. Both adolescents and adults showed metacontrol and metacontrol tended to increase across adolescence. Furthermore, model-based decision making was more pronounced for loss compared to gain frames but there was no evidence that this framing effect differed with age. Thus, the strategic adaptation of decision making continues to develop into young adulthood and for both adolescents and adults, losses increase the motivation to invest cognitive resources into an effortful decision-making strategy.
Collapse
Affiliation(s)
- Florian Bolenz
- Faculty of Psychology, Technische Universität Dresden, Dresden, Germany.,Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany.,Cluster of Excellence "Science of Intelligence", Technische Universität Berlin, Berlin, Germany
| | - Ben Eppinger
- Faculty of Psychology, Technische Universität Dresden, Dresden, Germany.,Department of Psychology, Concordia University, Montreal, Quebec, Canada.,PERFORM centre, Concordia University, Montreal, Quebec, Canada
| |
Collapse
|
46
|
Abstract
In novel situations, where direct experience is lacking or outdated, humans must rely on mental simulations to predict future outcomes. This review discusses recent work on the neural circuits that support such inference-based behavior. We focus on two specific examples: 1) using knowledge about the associative structure of the world to infer outcomes when direct experience is lacking; 2) inferring the current value of options when the desirability of the associated outcome has changed since the original learning experience. These two examples can be studied in the sensory preconditioning and devaluation tasks, respectively. We review results from studies in animals and humans suggesting that the orbitofrontal cortex (OFC), together with the hippocampus and amygdala, is necessary for inference in both of these tasks. Together, these findings suggest that the OFC is a critical hub in the brain network that supports inference-based decision-making.
Collapse
Affiliation(s)
- Fang Wang
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA
- Center for Cognitive Neuroscience, Duke University, Durham, North Carolina, USA
| | - Thorsten Kahnt
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA
- Department of Psychology, Weinberg College of Arts and Sciences, Northwestern University, Evanston, Illinois, USA
| |
Collapse
|
47
|
Eckstein MK, Wilbrecht L, Collins AGE. What do Reinforcement Learning Models Measure? Interpreting Model Parameters in Cognition and Neuroscience. Curr Opin Behav Sci 2021; 41:128-137. [PMID: 34984213 PMCID: PMC8722372 DOI: 10.1016/j.cobeha.2021.06.004] [Citation(s) in RCA: 37] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Reinforcement learning (RL) is a concept that has been invaluable to fields including machine learning, neuroscience, and cognitive science. However, what RL entails differs between fields, leading to difficulties when interpreting and translating findings. After laying out these differences, this paper focuses on cognitive (neuro)science to discuss how we as a field might over-interpret RL modeling results. We too often assume-implicitly-that modeling results generalize between tasks, models, and participant populations, despite negative empirical evidence for this assumption. We also often assume that parameters measure specific, unique (neuro)cognitive processes, a concept we call interpretability, when evidence suggests that they capture different functions across studies and tasks. We conclude that future computational research needs to pay increased attention to implicit assumptions when using RL models, and suggest that a more systematic understanding of contextual factors will help address issues and improve the ability of RL to explain brain and behavior.
Collapse
Affiliation(s)
- Maria K Eckstein
- Department of Psychology, UC Berkeley, 2121 Berkeley Way West, Berkeley, 94720, CA, USA
| | - Linda Wilbrecht
- Department of Psychology, UC Berkeley, 2121 Berkeley Way West, Berkeley, 94720, CA, USA
- Helen Wills Neuroscience Institute, UC Berkeley, 175 Li Ka Shing Center, Berkeley, 94720, CA, USA
| | - Anne G E Collins
- Department of Psychology, UC Berkeley, 2121 Berkeley Way West, Berkeley, 94720, CA, USA
- Helen Wills Neuroscience Institute, UC Berkeley, 175 Li Ka Shing Center, Berkeley, 94720, CA, USA
| |
Collapse
|
48
|
Abstract
An organism's survival can depend on its ability to recall and navigate to spatial locations associated with rewards, such as food or a home. Accumulating research has revealed that computations of reward and its prediction occur on multiple levels across a complex set of interacting brain regions, including those that support memory and navigation. However, how the brain coordinates the encoding, recall and use of reward information to guide navigation remains incompletely understood. In this Review, we propose that the brain's classical navigation centres - the hippocampus and the entorhinal cortex - are ideally suited to coordinate this larger network by representing both physical and mental space as a series of states. These states may be linked to reward via neuromodulatory inputs to the hippocampus-entorhinal cortex system. Hippocampal outputs can then broadcast sequences of states to the rest of the brain to store reward associations or to facilitate decision-making, potentially engaging additional value signals downstream. This proposal is supported by recent advances in both experimental and theoretical neuroscience. By discussing the neural systems traditionally tied to navigation and reward at their intersection, we aim to offer an integrated framework for understanding navigation to reward as a fundamental feature of many cognitive processes.
Collapse
|
49
|
Abstract
We rely on our long-term memories to guide future behaviors, making it adaptive to prioritize the retention of goal-relevant, salient information in memory. In this review, we discuss findings from rodent and human research to demonstrate that active processes during post-encoding consolidation support the selective stabilization of recent experience into adaptive, long-term memories. Building upon literatures focused on dynamics at the cellular level, we highlight that consolidation also transforms memories at the systems level to support future goal-relevant behavior, resulting in more generalized memory traces in the brain and behavior. We synthesize previous literatures spanning animal research, human cognitive neuroscience, and cognitive psychology to propose an integrative framework for adaptive consolidation by which goal-relevant memoranda are "tagged" for subsequent consolidation, resulting in selective transformations to the structure of memories that support flexible, goal-relevant behaviors.
Collapse
|
50
|
Liu Y, Mattar MG, Behrens TEJ, Daw ND, Dolan RJ. Experience replay is associated with efficient nonlocal learning. Science 2021; 372:372/6544/eabf1357. [PMID: 34016753 DOI: 10.1126/science.abf1357] [Citation(s) in RCA: 66] [Impact Index Per Article: 22.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2020] [Accepted: 04/15/2021] [Indexed: 01/08/2023]
Abstract
To make effective decisions, people need to consider the relationship between actions and outcomes. These are often separated by time and space. The neural mechanisms by which disjoint actions and outcomes are linked remain unknown. One promising hypothesis involves neural replay of nonlocal experience. Using a task that segregates direct from indirect value learning, combined with magnetoencephalography, we examined the role of neural replay in human nonlocal learning. After receipt of a reward, we found significant backward replay of nonlocal experience, with a 160-millisecond state-to-state time lag, which was linked to efficient learning of action values. Backward replay and behavioral evidence of nonlocal learning were more pronounced for experiences of greater benefit for future behavior. These findings support nonlocal replay as a neural mechanism for solving complex credit assignment problems during learning.
Collapse
Affiliation(s)
- Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China. .,Chinese Institute for Brain Research, Beijing, China.,Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK.,Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Marcelo G Mattar
- Department of Cognitive Science, University of California, San Diego, CA, USA.
| | - Timothy E J Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, UK. .,Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Nathaniel D Daw
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA.
| | - Raymond J Dolan
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China. .,Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK.,Wellcome Centre for Human Neuroimaging, University College London, London, UK.,Department of Psychiatry, Universitätsmedizin Berlin (Campus Charité Mitte), Berlin, Germany
| |
Collapse
|