1
|
George TM, Rastogi M, de Cothi W, Clopath C, Stachenfeld K, Barry C. RatInABox, a toolkit for modelling locomotion and neuronal activity in continuous environments. eLife 2024; 13:e85274. [PMID: 38334473 PMCID: PMC10857787 DOI: 10.7554/elife.85274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Accepted: 01/03/2024] [Indexed: 02/10/2024] Open
Abstract
Generating synthetic locomotory and neural data is a useful yet cumbersome step commonly required to study theoretical models of the brain's role in spatial navigation. This process can be time consuming and, without a common framework, makes it difficult to reproduce or compare studies which each generate test data in different ways. In response, we present RatInABox, an open-source Python toolkit designed to model realistic rodent locomotion and generate synthetic neural data from spatially modulated cell types. This software provides users with (i) the ability to construct one- or two-dimensional environments with configurable barriers and visual cues, (ii) a physically realistic random motion model fitted to experimental data, (iii) rapid online calculation of neural data for many of the known self-location or velocity selective cell types in the hippocampal formation (including place cells, grid cells, boundary vector cells, head direction cells) and (iv) a framework for constructing custom cell types, multi-layer network models and data- or policy-controlled motion trajectories. The motion and neural models are spatially and temporally continuous as well as topographically sensitive to boundary conditions and walls. We demonstrate that out-of-the-box parameter settings replicate many aspects of rodent foraging behaviour such as velocity statistics and the tendency of rodents to over-explore walls. Numerous tutorial scripts are provided, including examples where RatInABox is used for decoding position from neural data or to solve a navigational reinforcement learning task. We hope this tool will significantly streamline computational research into the brain's role in navigation.
Collapse
Affiliation(s)
- Tom M George
- Sainsbury Wellcome Centre, University College LondonLondonUnited Kingdom
| | - Mehul Rastogi
- Sainsbury Wellcome Centre, University College LondonLondonUnited Kingdom
| | - William de Cothi
- Department of Cell and Developmental Biology, University College LondonLondonUnited Kingdom
| | - Claudia Clopath
- Sainsbury Wellcome Centre, University College LondonLondonUnited Kingdom
- Department of Bioengineering, Imperial College LondonLondonUnited Kingdom
| | | | - Caswell Barry
- Department of Cell and Developmental Biology, University College LondonLondonUnited Kingdom
| |
Collapse
|
2
|
Li S, Li Z, Liu Q, Ren P, Sun L, Cui Z, Liang X. Predictable navigation through spontaneous brain states with cognitive-map-like representations. Prog Neurobiol 2024; 233:102570. [PMID: 38232783 DOI: 10.1016/j.pneurobio.2024.102570] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2023] [Revised: 11/19/2023] [Accepted: 01/10/2024] [Indexed: 01/19/2024]
Abstract
Just as navigating a physical environment, navigating through the landscapes of spontaneous brain states may also require an internal cognitive map. Contemporary computation theories propose modeling a cognitive map from a reinforcement learning perspective and argue that the map would be predictive in nature, representing each state as its upcoming states. Here, we used resting-state fMRI to test the hypothesis that the spaces of spontaneously reoccurring brain states are cognitive map-like, and may exhibit future-oriented predictivity. We identified two discrete brain states of the navigation-related brain networks during rest. By combining pattern similarity and dimensional reduction analysis, we embedded the occurrences of each brain state in a two-dimensional space. Successor representation modeling analysis recognized that these brain state occurrences exhibit place cell-like representations, akin to those observed in a physical space. Moreover, we observed predictive transitions of reoccurring brain states, which strongly covaried with individual cognitive and emotional assessments. Our findings offer a novel perspective on the cognitive significance of spontaneous brain activity and support the theory of cognitive map as a unifying framework for mental navigation.
Collapse
Affiliation(s)
- Siyang Li
- School of Life Science and Technology, Harbin Institute of Technology, Harbin 150001, China; Laboratory for Space Environment and Physical Sciences, Harbin Institute of Technology, Harbin 150001, China; Research Center for Human-Machine Augmented Intelligence, Research Institute of Artificial Intelligence, Zhejiang Lab, Hangzhou, Zhejiang 311100, China
| | - Zhipeng Li
- School of Life Science and Technology, Harbin Institute of Technology, Harbin 150001, China; Laboratory for Space Environment and Physical Sciences, Harbin Institute of Technology, Harbin 150001, China
| | - Qiuyi Liu
- School of Life Science and Technology, Harbin Institute of Technology, Harbin 150001, China; Laboratory for Space Environment and Physical Sciences, Harbin Institute of Technology, Harbin 150001, China
| | - Peng Ren
- School of Life Science and Technology, Harbin Institute of Technology, Harbin 150001, China
| | - Lili Sun
- School of Life Science and Technology, Harbin Institute of Technology, Harbin 150001, China; Laboratory for Space Environment and Physical Sciences, Harbin Institute of Technology, Harbin 150001, China
| | - Zaixu Cui
- Chinese Institute for Brain Research, Beijing 102206, China
| | - Xia Liang
- Laboratory for Space Environment and Physical Sciences, Harbin Institute of Technology, Harbin 150001, China; Frontiers Science Center for Matter Behave in Space Environment, Harbin Institute of Technology, Harbin 150001, China.
| |
Collapse
|
3
|
Mehrotra D, Dubé L. Accounting for multiscale processing in adaptive real-world decision-making via the hippocampus. Front Neurosci 2023; 17:1200842. [PMID: 37732307 PMCID: PMC10508350 DOI: 10.3389/fnins.2023.1200842] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Accepted: 08/25/2023] [Indexed: 09/22/2023] Open
Abstract
For adaptive real-time behavior in real-world contexts, the brain needs to allow past information over multiple timescales to influence current processing for making choices that create the best outcome as a person goes about making choices in their everyday life. The neuroeconomics literature on value-based decision-making has formalized such choice through reinforcement learning models for two extreme strategies. These strategies are model-free (MF), which is an automatic, stimulus-response type of action, and model-based (MB), which bases choice on cognitive representations of the world and causal inference on environment-behavior structure. The emphasis of examining the neural substrates of value-based decision making has been on the striatum and prefrontal regions, especially with regards to the "here and now" decision-making. Yet, such a dichotomy does not embrace all the dynamic complexity involved. In addition, despite robust research on the role of the hippocampus in memory and spatial learning, its contribution to value-based decision making is just starting to be explored. This paper aims to better appreciate the role of the hippocampus in decision-making and advance the successor representation (SR) as a candidate mechanism for encoding state representations in the hippocampus, separate from reward representations. To this end, we review research that relates hippocampal sequences to SR models showing that the implementation of such sequences in reinforcement learning agents improves their performance. This also enables the agents to perform multiscale temporal processing in a biologically plausible manner. Altogether, we articulate a framework to advance current striatal and prefrontal-focused decision making to better account for multiscale mechanisms underlying various real-world time-related concepts such as the self that cumulates over a person's life course.
Collapse
Affiliation(s)
- Dhruv Mehrotra
- Integrated Program in Neuroscience, McGill University, Montréal, QC, Canada
- Montréal Neurological Institute, McGill University, Montréal, QC, Canada
| | - Laurette Dubé
- Desautels Faculty of Management, McGill University, Montréal, QC, Canada
- McGill Center for the Convergence of Health and Economics, McGill University, Montréal, QC, Canada
| |
Collapse
|
4
|
George TM, de Cothi W, Stachenfeld KL, Barry C. Rapid learning of predictive maps with STDP and theta phase precession. eLife 2023; 12:80663. [PMID: 36927826 PMCID: PMC10019887 DOI: 10.7554/elife.80663] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 02/26/2023] [Indexed: 03/18/2023] Open
Abstract
The predictive map hypothesis is a promising candidate principle for hippocampal function. A favoured formalisation of this hypothesis, called the successor representation, proposes that each place cell encodes the expected state occupancy of its target location in the near future. This predictive framework is supported by behavioural as well as electrophysiological evidence and has desirable consequences for both the generalisability and efficiency of reinforcement learning algorithms. However, it is unclear how the successor representation might be learnt in the brain. Error-driven temporal difference learning, commonly used to learn successor representations in artificial agents, is not known to be implemented in hippocampal networks. Instead, we demonstrate that spike-timing dependent plasticity (STDP), a form of Hebbian learning, acting on temporally compressed trajectories known as 'theta sweeps', is sufficient to rapidly learn a close approximation to the successor representation. The model is biologically plausible - it uses spiking neurons modulated by theta-band oscillations, diffuse and overlapping place cell-like state representations, and experimentally matched parameters. We show how this model maps onto known aspects of hippocampal circuitry and explains substantial variance in the temporal difference successor matrix, consequently giving rise to place cells that demonstrate experimentally observed successor representation-related phenomena including backwards expansion on a 1D track and elongation near walls in 2D. Finally, our model provides insight into the observed topographical ordering of place field sizes along the dorsal-ventral axis by showing this is necessary to prevent the detrimental mixing of larger place fields, which encode longer timescale successor representations, with more fine-grained predictions of spatial location.
Collapse
Affiliation(s)
- Tom M George
- Sainsbury Wellcome Centre for Neural Circuits and Behaviour, University College LondonLondonUnited Kingdom
| | - William de Cothi
- Research Department of Cell and Developmental Biology, University College LondonLondonUnited Kingdom
| | | | - Caswell Barry
- Research Department of Cell and Developmental Biology, University College LondonLondonUnited Kingdom
| |
Collapse
|
5
|
Fang C, Aronov D, Abbott LF, Mackevicius EL. Neural learning rules for generating flexible predictions and computing the successor representation. eLife 2023; 12:e80680. [PMID: 36928104 PMCID: PMC10019889 DOI: 10.7554/elife.80680] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 10/26/2022] [Indexed: 03/18/2023] Open
Abstract
The predictive nature of the hippocampus is thought to be useful for memory-guided cognitive behaviors. Inspired by the reinforcement learning literature, this notion has been formalized as a predictive map called the successor representation (SR). The SR captures a number of observations about hippocampal activity. However, the algorithm does not provide a neural mechanism for how such representations arise. Here, we show the dynamics of a recurrent neural network naturally calculate the SR when the synaptic weights match the transition probability matrix. Interestingly, the predictive horizon can be flexibly modulated simply by changing the network gain. We derive simple, biologically plausible learning rules to learn the SR in a recurrent network. We test our model with realistic inputs and match hippocampal data recorded during random foraging. Taken together, our results suggest that the SR is more accessible in neural circuits than previously thought and can support a broad range of cognitive functions.
Collapse
Affiliation(s)
- Ching Fang
- Zuckerman Institute, Department of Neuroscience, Columbia UniversityNew YorkUnited States
| | - Dmitriy Aronov
- Zuckerman Institute, Department of Neuroscience, Columbia UniversityNew YorkUnited States
| | - LF Abbott
- Zuckerman Institute, Department of Neuroscience, Columbia UniversityNew YorkUnited States
| | - Emily L Mackevicius
- Zuckerman Institute, Department of Neuroscience, Columbia UniversityNew YorkUnited States
- Basis Research InstituteNew YorkUnited States
| |
Collapse
|
6
|
Kang YHR, Wolpert DM, Lengyel M. Spatial uncertainty and environmental geometry in navigation. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.01.30.526278. [PMID: 36778354 PMCID: PMC9915518 DOI: 10.1101/2023.01.30.526278] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
Variations in the geometry of the environment, such as the shape and size of an enclosure, have profound effects on navigational behavior and its neural underpinning. Here, we show that these effects arise as a consequence of a single, unifying principle: to navigate efficiently, the brain must maintain and update the uncertainty about one's location. We developed an image-computable Bayesian ideal observer model of navigation, continually combining noisy visual and self-motion inputs, and a neural encoding model optimized to represent the location uncertainty computed by the ideal observer. Through mathematical analysis and numerical simulations, we show that the ideal observer accounts for a diverse range of sometimes paradoxical distortions of human homing behavior in anisotropic and deformed environments, including 'boundary tethering', and its neural encoding accounts for distortions of rodent grid cell responses under identical environmental manipulations. Our results demonstrate that spatial uncertainty plays a key role in navigation.
Collapse
Affiliation(s)
- Yul HR Kang
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK
- Department of Biological and Experimental Psychology, Queen Mary University of London, London, UK
| | - Daniel M Wolpert
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
- Department of Neuroscience, Columbia University, New York, NY, USA
| | - Máté Lengyel
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK
- Center for Cognitive Computation, Department of Cognitive Science, Central European University, Budapest, Hungary
| |
Collapse
|
7
|
Momennejad I. A rubric for human-like agents and NeuroAI. Philos Trans R Soc Lond B Biol Sci 2023; 378:20210446. [PMID: 36511409 PMCID: PMC9745874 DOI: 10.1098/rstb.2021.0446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2022] [Accepted: 10/27/2022] [Indexed: 12/15/2022] Open
Abstract
Researchers across cognitive, neuro- and computer sciences increasingly reference 'human-like' artificial intelligence and 'neuroAI'. However, the scope and use of the terms are often inconsistent. Contributed research ranges widely from mimicking behaviour, to testing machine learning methods as neurally plausible hypotheses at the cellular or functional levels, or solving engineering problems. However, it cannot be assumed nor expected that progress on one of these three goals will automatically translate to progress in others. Here, a simple rubric is proposed to clarify the scope of individual contributions, grounded in their commitments to human-like behaviour, neural plausibility or benchmark/engineering/computer science goals. This is clarified using examples of weak and strong neuroAI and human-like agents, and discussing the generative, corroborate and corrective ways in which the three dimensions interact with one another. The author maintains that future progress in artificial intelligence will need strong interactions across the disciplines, with iterative feedback loops and meticulous validity tests-leading to both known and yet-unknown advances that may span decades to come. This article is part of a discussion meeting issue 'New approaches to 3D vision'.
Collapse
Affiliation(s)
- Ida Momennejad
- Microsoft Research NYC, Reinforcement Learning Station, 300 Lafayette, New York, NY 10012, USA
| |
Collapse
|
8
|
Duvelle É, Grieves RM, van der Meer MAA. Temporal context and latent state inference in the hippocampal splitter signal. eLife 2023; 12:e82357. [PMID: 36622350 PMCID: PMC9829411 DOI: 10.7554/elife.82357] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Accepted: 12/06/2022] [Indexed: 01/10/2023] Open
Abstract
The hippocampus is thought to enable the encoding and retrieval of ongoing experience, the organization of that experience into structured representations like contexts, maps, and schemas, and the use of these structures to plan for the future. A central goal is to understand what the core computations supporting these functions are, and how these computations are realized in the collective action of single neurons. A potential access point into this issue is provided by 'splitter cells', hippocampal neurons that fire differentially on the overlapping segment of trajectories that differ in their past and/or future. However, the literature on splitter cells has been fragmented and confusing, owing to differences in terminology, behavioral tasks, and analysis methods across studies. In this review, we synthesize consistent findings from this literature, establish a common set of terms, and translate between single-cell and ensemble perspectives. Most importantly, we examine the combined findings through the lens of two major theoretical ideas about hippocampal function: representation of temporal context and latent state inference. We find that unique signature properties of each of these models are necessary to account for the data, but neither theory, by itself, explains all of its features. Specifically, the temporal gradedness of the splitter signal is strong support for temporal context, but is hard to explain using state models, while its flexibility and task-dependence is naturally accounted for using state inference, but poses a challenge otherwise. These theories suggest a number of avenues for future work, and we believe their application to splitter cells is a timely and informative domain for testing and refining theoretical ideas about hippocampal function.
Collapse
Affiliation(s)
- Éléonore Duvelle
- Department of Psychological and Brain Sciences, Dartmouth CollegeHanoverUnited States
| | - Roddy M Grieves
- Department of Psychological and Brain Sciences, Dartmouth CollegeHanoverUnited States
| | | |
Collapse
|
9
|
de Cothi W, Nyberg N, Griesbauer EM, Ghanamé C, Zisch F, Lefort JM, Fletcher L, Newton C, Renaudineau S, Bendor D, Grieves R, Duvelle É, Barry C, Spiers HJ. Predictive maps in rats and humans for spatial navigation. Curr Biol 2022; 32:3676-3689.e5. [PMID: 35863351 DOI: 10.1016/j.cub.2022.06.090] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Revised: 05/19/2022] [Accepted: 06/29/2022] [Indexed: 11/25/2022]
Abstract
Much of our understanding of navigation comes from the study of individual species, often with specific tasks tailored to those species. Here, we provide a novel experimental and analytic framework integrating across humans, rats, and simulated reinforcement learning (RL) agents to interrogate the dynamics of behavior during spatial navigation. We developed a novel open-field navigation task ("Tartarus maze") requiring dynamic adaptation (shortcuts and detours) to frequently changing obstructions on the path to a hidden goal. Humans and rats were remarkably similar in their trajectories. Both species showed the greatest similarity to RL agents utilizing a "successor representation," which creates a predictive map. Humans also displayed trajectory features similar to model-based RL agents, which implemented an optimal tree-search planning procedure. Our results help refine models seeking to explain mammalian navigation in dynamic environments and highlight the utility of modeling the behavior of different species to uncover the shared mechanisms that support behavior.
Collapse
Affiliation(s)
- William de Cothi
- Department of Cell and Developmental Biology, University College London, London, UK; Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK.
| | - Nils Nyberg
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Eva-Maria Griesbauer
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Carole Ghanamé
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Fiona Zisch
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK; The Bartlett School of Architecture, University College London, London, UK
| | - Julie M Lefort
- Department of Cell and Developmental Biology, University College London, London, UK
| | - Lydia Fletcher
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Coco Newton
- Department of Clinical Neurosciences, University of Cambridge, Cambridge, UK
| | - Sophie Renaudineau
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Daniel Bendor
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Roddy Grieves
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Éléonore Duvelle
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Caswell Barry
- Department of Cell and Developmental Biology, University College London, London, UK
| | - Hugo J Spiers
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK.
| |
Collapse
|
10
|
Tanni S, de Cothi W, Barry C. State transitions in the statistically stable place cell population correspond to rate of perceptual change. Curr Biol 2022; 32:3505-3514.e7. [PMID: 35835121 PMCID: PMC9616721 DOI: 10.1016/j.cub.2022.06.046] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2021] [Revised: 04/20/2022] [Accepted: 06/15/2022] [Indexed: 11/25/2022]
Abstract
The hippocampus occupies a central role in mammalian navigation and memory. Yet an understanding of the rules that govern the statistics and granularity of the spatial code, as well as its interactions with perceptual stimuli, is lacking. We analyzed CA1 place cell activity recorded while rats foraged in different large-scale environments. We found that place cell activity was subject to an unexpected but precise homeostasis—the distribution of activity in the population as a whole being constant at all locations within and between environments. Using a virtual reconstruction of the largest environment, we showed that the rate of transition through this statistically stable population matches the rate of change in the animals’ visual scene. Thus, place fields near boundaries were small but numerous, while in the environment’s interior, they were larger but more dispersed. These results indicate that hippocampal spatial activity is governed by a small number of simple laws and, in particular, suggest the presence of an information-theoretic bound imposed by perception on the fidelity of the spatial memory system. Neural activity in rodent CA1 place cell populations is homeostatically balanced Hippocampal place field size and frequency are governed by proximity to boundaries Transition rate through place cell population matches rate of change in visual scene
Collapse
Affiliation(s)
- Sander Tanni
- Department of Cell and Developmental Biology, University College London, London, UK
| | - William de Cothi
- Department of Cell and Developmental Biology, University College London, London, UK
| | - Caswell Barry
- Department of Cell and Developmental Biology, University College London, London, UK.
| |
Collapse
|
11
|
Neurophysiological Evidence for Cognitive Map Formation during Sequence Learning. eNeuro 2022; 9:ENEURO.0361-21.2022. [PMID: 35105662 PMCID: PMC8896554 DOI: 10.1523/eneuro.0361-21.2022] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Revised: 12/03/2021] [Accepted: 01/03/2022] [Indexed: 12/29/2022] Open
Abstract
Humans deftly parse statistics from sequences. Some theories posit that humans learn these statistics by forming cognitive maps, or underlying representations of the latent space which links items in the sequence. Here, an item in the sequence is a node, and the probability of transitioning between two items is an edge. Sequences can then be generated from walks through the latent space, with different spaces giving rise to different sequence statistics. Individual or group differences in sequence learning can be modeled by changing the time scale over which estimates of transition probabilities are built, or in other words, by changing the amount of temporal discounting. Latent space models with temporal discounting bear a resemblance to models of navigation through Euclidean spaces. However, few explicit links have been made between predictions from Euclidean spatial navigation and neural activity during human sequence learning. Here, we use a combination of behavioral modeling and intracranial encephalography (iEEG) recordings to investigate how neural activity might support the formation of space-like cognitive maps through temporal discounting during sequence learning. Specifically, we acquire human reaction times from a sequential reaction time task, to which we fit a model that formulates the amount of temporal discounting as a single free parameter. From the parameter, we calculate each individual's estimate of the latent space. We find that neural activity reflects these estimates mostly in the temporal lobe, including areas involved in spatial navigation. Similar to spatial navigation, we find that low-dimensional representations of neural activity allow for easy separation of important features, such as modules, in the latent space. Lastly, we take advantage of the high temporal resolution of iEEG data to determine the time scale on which latent spaces are learned. We find that learning typically happens within the first 500 trials, and is modulated by the underlying latent space and the amount of temporal discounting characteristic of each participant. Ultimately, this work provides important links between behavioral models of sequence learning and neural activity during the same behavior, and contextualizes these results within a broader framework of domain general cognitive maps.
Collapse
|
12
|
Nyberg N, Duvelle É, Barry C, Spiers HJ. Spatial goal coding in the hippocampal formation. Neuron 2022; 110:394-422. [PMID: 35032426 DOI: 10.1016/j.neuron.2021.12.012] [Citation(s) in RCA: 42] [Impact Index Per Article: 21.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2021] [Revised: 10/18/2021] [Accepted: 12/08/2021] [Indexed: 12/22/2022]
Abstract
The mammalian hippocampal formation contains several distinct populations of neurons involved in representing self-position and orientation. These neurons, which include place, grid, head direction, and boundary-vector cells, are thought to collectively instantiate cognitive maps supporting flexible navigation. However, to flexibly navigate, it is necessary to also maintain internal representations of goal locations, such that goal-directed routes can be planned and executed. Although it has remained unclear how the mammalian brain represents goal locations, multiple neural candidates have recently been uncovered during different phases of navigation. For example, during planning, sequential activation of spatial cells may enable simulation of future routes toward the goal. During travel, modulation of spatial cells by the prospective route, or by distance and direction to the goal, may allow maintenance of route and goal-location information, supporting navigation on an ongoing basis. As the goal is approached, an increased activation of spatial cells may enable the goal location to become distinctly represented within cognitive maps, aiding goal localization. Lastly, after arrival at the goal, sequential activation of spatial cells may represent the just-taken route, enabling route learning and evaluation. Here, we review and synthesize these and other evidence for goal coding in mammalian brains, relate the experimental findings to predictions from computational models, and discuss outstanding questions and future challenges.
Collapse
Affiliation(s)
- Nils Nyberg
- Institute of Behavioural Neuroscience, Department of Experimental Psychology, University College London, London, UK.
| | - Éléonore Duvelle
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Caswell Barry
- Department of Cell and Developmental Biology, University College London, London, UK
| | - Hugo J Spiers
- Institute of Behavioural Neuroscience, Department of Experimental Psychology, University College London, London, UK.
| |
Collapse
|
13
|
Son JY, Bhandari A, FeldmanHall O. Cognitive maps of social features enable flexible inference in social networks. Proc Natl Acad Sci U S A 2021; 118:e2021699118. [PMID: 34518372 PMCID: PMC8488581 DOI: 10.1073/pnas.2021699118] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/01/2021] [Indexed: 11/18/2022] Open
Abstract
In order to navigate a complex web of relationships, an individual must learn and represent the connections between people in a social network. However, the sheer size and complexity of the social world makes it impossible to acquire firsthand knowledge of all relations within a network, suggesting that people must make inferences about unobserved relationships to fill in the gaps. Across three studies (n = 328), we show that people can encode information about social features (e.g., hobbies, clubs) and subsequently deploy this knowledge to infer the existence of unobserved friendships in the network. Using computational models, we test various feature-based mechanisms that could support such inferences. We find that people's ability to successfully generalize depends on two representational strategies: a simple but inflexible similarity heuristic that leverages homophily, and a complex but flexible cognitive map that encodes the statistical relationships between social features and friendships. Together, our studies reveal that people can build cognitive maps encoding arbitrary patterns of latent relations in many abstract feature spaces, allowing social networks to be represented in a flexible format. Moreover, these findings shed light on open questions across disciplines about how people learn and represent social networks and may have implications for generating more human-like link prediction in machine learning algorithms.
Collapse
Affiliation(s)
- Jae-Young Son
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, RI 02912
| | - Apoorva Bhandari
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, RI 02912
| | - Oriel FeldmanHall
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, RI 02912;
- Carney Institute for Brain Sciences, Brown University, Providence, RI 02912
| |
Collapse
|
14
|
Abstract
Entorhinal cortical grid cells fire in a periodic pattern that tiles space, which is suggestive of a spatial coordinate system. However, irregularities in the grid pattern as well as responses of grid cells in contexts other than spatial navigation have presented a challenge to existing models of entorhinal function. In this Perspective, we propose that hippocampal input provides a key informative drive to the grid network in both spatial and non-spatial circumstances, particularly around salient events. We build on previous models in which neural activity propagates through the entorhinal-hippocampal network in time. This temporal contiguity in network activity points to temporal order as a necessary characteristic of representations generated by the hippocampal formation. We advocate that interactions in the entorhinal-hippocampal loop build a topological representation that is rooted in the temporal order of experience. In this way, the structure of grid cell firing supports a learned topology rather than a rigid coordinate frame that is bound to measurements of the physical world.
Collapse
|
15
|
Duvelle É, Grieves RM, Liu A, Jedidi-Ayoub S, Holeniewska J, Harris A, Nyberg N, Donnarumma F, Lefort JM, Jeffery KJ, Summerfield C, Pezzulo G, Spiers HJ. Hippocampal place cells encode global location but not connectivity in a complex space. Curr Biol 2021; 31:1221-1233.e9. [PMID: 33581073 PMCID: PMC7988036 DOI: 10.1016/j.cub.2021.01.005] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2020] [Revised: 12/22/2020] [Accepted: 01/05/2021] [Indexed: 11/20/2022]
Abstract
Flexible navigation relies on a cognitive map of space, thought to be implemented by hippocampal place cells: neurons that exhibit location-specific firing. In connected environments, optimal navigation requires keeping track of one's location and of the available connections between subspaces. We examined whether the dorsal CA1 place cells of rats encode environmental connectivity in four geometrically identical boxes arranged in a square. Rats moved between boxes by pushing saloon-type doors that could be locked in one or both directions. Although rats demonstrated knowledge of environmental connectivity, their place cells did not respond to connectivity changes, nor did they represent doorways differently from other locations. Place cells coded location in a global reference frame, with a different map for each box and minimal repetitive fields despite the repetitive geometry. These results suggest that CA1 place cells provide a spatial map that does not explicitly include connectivity.
Collapse
Affiliation(s)
- Éléonore Duvelle
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA.
| | - Roddy M Grieves
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Anyi Liu
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK
| | - Selim Jedidi-Ayoub
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK
| | - Joanna Holeniewska
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK
| | - Adam Harris
- Department of Experimental Psychology, University of Oxford, OX2 6BW Oxford, UK
| | - Nils Nyberg
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK
| | - Francesco Donnarumma
- Institute of Cognitive Sciences and Technologies, National Research Council, via S. Martino d. Battaglia 44, 00185 Rome, Italy
| | - Julie M Lefort
- University College London, Department of Cell and Developmental Biology, London, UK
| | - Kate J Jeffery
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK
| | | | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, via S. Martino d. Battaglia 44, 00185 Rome, Italy
| | - Hugo J Spiers
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK.
| |
Collapse
|
16
|
Peer M, Brunec IK, Newcombe NS, Epstein RA. Structuring Knowledge with Cognitive Maps and Cognitive Graphs. Trends Cogn Sci 2021; 25:37-54. [PMID: 33248898 PMCID: PMC7746605 DOI: 10.1016/j.tics.2020.10.004] [Citation(s) in RCA: 61] [Impact Index Per Article: 20.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2020] [Revised: 10/16/2020] [Accepted: 10/17/2020] [Indexed: 12/21/2022]
Abstract
Humans and animals use mental representations of the spatial structure of the world to navigate. The classical view is that these representations take the form of Euclidean cognitive maps, but alternative theories suggest that they are cognitive graphs consisting of locations connected by paths. We review evidence suggesting that both map-like and graph-like representations exist in the mind/brain that rely on partially overlapping neural systems. Maps and graphs can operate simultaneously or separately, and they may be applied to both spatial and nonspatial knowledge. By providing structural frameworks for complex information, cognitive maps and cognitive graphs may provide fundamental organizing schemata that allow us to navigate in physical, social, and conceptual spaces.
Collapse
Affiliation(s)
- Michael Peer
- Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Iva K Brunec
- Department of Psychology, Temple University, Philadelphia, PA 19122, USA
| | - Nora S Newcombe
- Department of Psychology, Temple University, Philadelphia, PA 19122, USA
| | - Russell A Epstein
- Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA.
| |
Collapse
|
17
|
Abstract
Humans and other animals use multiple strategies for making decisions. Reinforcement-learning theory distinguishes between stimulus-response (model-free; MF) learning and deliberative (model-based; MB) planning. The spatial-navigation literature presents a parallel dichotomy between navigation strategies. In "response learning," associated with the dorsolateral striatum (DLS), decisions are anchored to an egocentric reference frame. In "place learning," associated with the hippocampus, decisions are anchored to an allocentric reference frame. Emerging evidence suggests that the contribution of hippocampus to place learning may also underlie its contribution to MB learning by representing relational structure in a cognitive map. Here, we introduce a computational model in which hippocampus subserves place and MB learning by learning a "successor representation" of relational structure between states; DLS implements model-free response learning by learning associations between actions and egocentric representations of landmarks; and action values from either system are weighted by the reliability of its predictions. We show that this model reproduces a range of seemingly disparate behavioral findings in spatial and nonspatial decision tasks and explains the effects of lesions to DLS and hippocampus on these tasks. Furthermore, modeling place cells as driven by boundaries explains the observation that, unlike navigation guided by landmarks, navigation guided by boundaries is robust to "blocking" by prior state-reward associations due to learned associations between place cells. Our model, originally shaped by detailed constraints in the spatial literature, successfully characterizes the hippocampal-striatal system as a general system for decision making via adaptive combination of stimulus-response learning and the use of a cognitive map.
Collapse
|
18
|
Hasselmo ME. Introduction to part two of the special issue on computational models of hippocampus and related structures. Hippocampus 2020; 30:1328-1331. [PMID: 33185288 DOI: 10.1002/hipo.23279] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Extensive computational modeling has focused on the hippocampal formation and related cortical structures. This introduction describes the topics addressed by individual articles in part two of this special issue of the journal Hippocampus on the topic of computational models of the hippocampus and related structures.
Collapse
Affiliation(s)
- Michael E Hasselmo
- Center for Systems Neuroscience, Boston University, Boston, Massachusetts, USA
| |
Collapse
|
19
|
de Cothi W, Barry C. Neurobiological successor features for spatial navigation. Hippocampus 2020; 30:1347-1355. [PMID: 32584491 PMCID: PMC8432165 DOI: 10.1002/hipo.23246] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2019] [Revised: 04/27/2020] [Accepted: 06/03/2020] [Indexed: 12/03/2022]
Abstract
The hippocampus has long been observed to encode a representation of an animal's position in space. Recent evidence suggests that the nature of this representation is somewhat predictive and can be modeled by learning a successor representation (SR) between distinct positions in an environment. However, this discretization of space is subjective making it difficult to formulate predictions about how some environmental manipulations should impact the hippocampal representation. Here, we present a model of place and grid cell firing as a consequence of learning a SR from a basis set of known neurobiological features—boundary vector cells (BVCs). The model describes place cell firing as the successor features of the SR, with grid cells forming a low‐dimensional representation of these successor features. We show that the place and grid cells generated using the BVC‐SR model provide a good account of biological data for a variety of environmental manipulations, including dimensional stretches, barrier insertions, and the influence of environmental geometry on the hippocampal representation of space.
Collapse
Affiliation(s)
- William de Cothi
- Research Department of Cell and Developmental Biology, University College London, London, UK
| | - Caswell Barry
- Research Department of Cell and Developmental Biology, University College London, London, UK
| |
Collapse
|