1
|
Pulvermüller F. Neurobiological mechanisms for language, symbols and concepts: Clues from brain-constrained deep neural networks. Prog Neurobiol 2023; 230:102511. [PMID: 37482195 PMCID: PMC10518464 DOI: 10.1016/j.pneurobio.2023.102511] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2022] [Revised: 05/02/2023] [Accepted: 07/18/2023] [Indexed: 07/25/2023]
Abstract
Neural networks are successfully used to imitate and model cognitive processes. However, to provide clues about the neurobiological mechanisms enabling human cognition, these models need to mimic the structure and function of real brains. Brain-constrained networks differ from classic neural networks by implementing brain similarities at different scales, ranging from the micro- and mesoscopic levels of neuronal function, local neuronal links and circuit interaction to large-scale anatomical structure and between-area connectivity. This review shows how brain-constrained neural networks can be applied to study in silico the formation of mechanisms for symbol and concept processing and to work towards neurobiological explanations of specifically human cognitive abilities. These include verbal working memory and learning of large vocabularies of symbols, semantic binding carried by specific areas of cortex, attention focusing and modulation driven by symbol type, and the acquisition of concrete and abstract concepts partly influenced by symbols. Neuronal assembly activity in the networks is analyzed to deliver putative mechanistic correlates of higher cognitive processes and to develop candidate explanations founded in established neurobiological principles.
Collapse
Affiliation(s)
- Friedemann Pulvermüller
- Brain Language Laboratory, Department of Philosophy and Humanities, WE4, Freie Universität Berlin, 14195 Berlin, Germany; Berlin School of Mind and Brain, Humboldt Universität zu Berlin, 10099 Berlin, Germany; Einstein Center for Neurosciences Berlin, 10117 Berlin, Germany; Cluster of Excellence 'Matters of Activity', Humboldt Universität zu Berlin, 10099 Berlin, Germany.
| |
Collapse
|
2
|
Scleidorovich P, Weitzenfeld A, Fellous JM, Dominey PF. Integration of velocity-dependent spatio-temporal structure of place cell activation during navigation in a reservoir model of prefrontal cortex. BIOLOGICAL CYBERNETICS 2022; 116:585-610. [PMID: 36222887 DOI: 10.1007/s00422-022-00945-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Accepted: 09/10/2022] [Indexed: 06/16/2023]
Abstract
Sequential behavior unfolds both in space and in time. The same spatial trajectory can be realized in different manners in the same overall time by changing instantaneous speeds. The current research investigates how speed profiles might be given behavioral significance and how cortical networks might encode this information. We first demonstrate that rats can associate different speed patterns on the same trajectory with distinct behavioral choices. In this novel experimental paradigm, rats follow a small baited robot in a large megaspace environment where the rat's speed is precisely controlled by the robot's speed. Based on this proof of concept and research showing that recurrent reservoir networks are ideal for representing spatio-temporal structures, we then test reservoir networks in simulated navigation contexts and demonstrate they can discriminate between traversals of the same path with identical durations but different speed profiles. We then test the networks in an embodied robotic setup, where we use place cell representations from physically navigating robots as input and again successfully discriminate between traversals. To demonstrate that this capability is inherent to recurrent networks, we compared the model against simple linear integrators. Interestingly, although the linear integrators could also perform the speed profile discrimination, a clear difference emerged when examining information coding in both models. Reservoir neurons displayed a form of statistical mixed selectivity as a complex interaction between spatial location and speed that was not as abundant in the linear integrators. This mixed selectivity is characteristic of cortex and reservoirs and allows us to generate specific predictions about the neural activity that will be recorded in rat cortex in future experiments.
Collapse
Affiliation(s)
- Pablo Scleidorovich
- Department of Computer Science and Engineering, University of South Florida, Tampa, USA
| | - Alfredo Weitzenfeld
- Department of Computer Science and Engineering, University of South Florida, Tampa, USA
| | - Jean-Marc Fellous
- Departments of Psychology and Biomedical Engineering, University of Arizona, Tucson, USA
| | - Peter Ford Dominey
- INSERM UMR1093-CAPS, Université Bourgogne Franche-Comté, UFR Des Sciences du Sport, 21000, Dijon, France.
- Robot Cognition Laboratory, Institute Marey, Dijon, France.
| |
Collapse
|
3
|
A Brain-Inspired Model of Hippocampal Spatial Cognition Based on a Memory-Replay Mechanism. Brain Sci 2022; 12:brainsci12091176. [PMID: 36138911 PMCID: PMC9496859 DOI: 10.3390/brainsci12091176] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Revised: 08/13/2022] [Accepted: 08/19/2022] [Indexed: 11/17/2022] Open
Abstract
Since the hippocampus plays an important role in memory and spatial cognition, the study of spatial computation models inspired by the hippocampus has attracted much attention. This study relies mainly on reward signals for learning environments and planning paths. As reward signals in a complex or large-scale environment attenuate sharply, the spatial cognition and path planning performance of such models will decrease clearly as a result. Aiming to solve this problem, we present a brain-inspired mechanism, a Memory-Replay Mechanism, that is inspired by the reactivation function of place cells in the hippocampus. We classify the path memory according to the reward information and find the overlapping place cells in different categories of path memory to segment and reconstruct the memory to form a “virtual path”, replaying the memory by associating the reward information. We conducted a series of navigation experiments in a simple environment called a Morris water maze (MWM) and in a complex environment, and we compared our model with a reinforcement learning model and other brain-inspired models. The experimental results show that under the same conditions, our model has a higher rate of environmental exploration and more stable signal transmission, and the average reward obtained under stable conditions was 14.12% higher than RL with random-experience replay. Our model also shows good performance in complex maze environments where signals are easily attenuated. Moreover, the performance of our model at bifurcations is consistent with neurophysiological studies.
Collapse
|
4
|
Stoianov I, Maisto D, Pezzulo G. The hippocampal formation as a hierarchical generative model supporting generative replay and continual learning. Prog Neurobiol 2022; 217:102329. [PMID: 35870678 DOI: 10.1016/j.pneurobio.2022.102329] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Revised: 07/15/2022] [Accepted: 07/19/2022] [Indexed: 11/28/2022]
Abstract
We advance a novel computational theory of the hippocampal formation as a hierarchical generative model that organizes sequential experiences, such as rodent trajectories during spatial navigation, into coherent spatiotemporal contexts. We propose that the hippocampal generative model is endowed with inductive biases to identify individual items of experience (first hierarchical layer), organize them into sequences (second layer) and cluster them into maps (third layer). This theory entails a novel characterization of hippocampal reactivations as generative replay: the offline resampling of fictive sequences from the generative model, which supports the continual learning of multiple sequential experiences. We show that the model learns and efficiently retains multiple spatial navigation trajectories, by organizing them into spatial maps. Furthermore, the model reproduces flexible and prospective aspects of hippocampal dynamics that are challenging to explain within existing frameworks. This theory reconciles multiple roles of the hippocampal formation in map-based navigation, episodic memory and imagination.
Collapse
Affiliation(s)
- Ivilin Stoianov
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Domenico Maisto
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| |
Collapse
|
5
|
Kumar MG, Tan C, Libedinsky C, Yen SC, Tan AYY. A Nonlinear Hidden Layer Enables Actor-Critic Agents to Learn Multiple Paired Association Navigation. Cereb Cortex 2022; 32:3917-3936. [PMID: 35034127 DOI: 10.1093/cercor/bhab456] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2021] [Revised: 11/05/2021] [Accepted: 11/06/2021] [Indexed: 11/15/2022] Open
Abstract
Navigation to multiple cued reward locations has been increasingly used to study rodent learning. Though deep reinforcement learning agents have been shown to be able to learn the task, they are not biologically plausible. Biologically plausible classic actor-critic agents have been shown to learn to navigate to single reward locations, but which biologically plausible agents are able to learn multiple cue-reward location tasks has remained unclear. In this computational study, we show versions of classic agents that learn to navigate to a single reward location, and adapt to reward location displacement, but are not able to learn multiple paired association navigation. The limitation is overcome by an agent in which place cell and cue information are first processed by a feedforward nonlinear hidden layer with synapses to the actor and critic subject to temporal difference error-modulated plasticity. Faster learning is obtained when the feedforward layer is replaced by a recurrent reservoir network.
Collapse
Affiliation(s)
- M Ganesh Kumar
- Integrative Sciences and Engineering Programme, NUS Graduate School, National University of Singapore, Singapore 119077, Singapore
- The N.1 Institute for Health, National University of Singapore, Singapore 117456, Singapore
- Innovation and Design Programme, Faculty of Engineering, National University of Singapore, Singapore 117579, Singapore
| | - Cheston Tan
- Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore 138632, Singapore
| | - Camilo Libedinsky
- Integrative Sciences and Engineering Programme, NUS Graduate School, National University of Singapore, Singapore 119077, Singapore
- The N.1 Institute for Health, National University of Singapore, Singapore 117456, Singapore
- Department of Psychology, National University of Singapore, Singapore 117570, Singapore
- Institute of Molecular and Cell Biology, Agency for Science, Technology and Research, Singapore 138673, Singapore
| | - Shih-Cheng Yen
- Integrative Sciences and Engineering Programme, NUS Graduate School, National University of Singapore, Singapore 119077, Singapore
- The N.1 Institute for Health, National University of Singapore, Singapore 117456, Singapore
- Innovation and Design Programme, Faculty of Engineering, National University of Singapore, Singapore 117579, Singapore
| | - Andrew Y Y Tan
- Department of Physiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117593, Singapore
- Healthy Longevity Translational Research Programme, Yong Loo Lin School of Medicine, National University of Singapore, Singapore 119228, Singapore
- Cardiovascular Disease Translational Research Programme, Yong Loo Lin School of Medicine, National University of Singapore, Singapore 119228, Singapore
- Neurobiology Programme, Life Sciences Institute, National University of Singapore, Singapore 119077, Singapore
| |
Collapse
|
6
|
Chai J, Ruan X, Huang J. A Possible Explanation for the Generation of Habit in Navigation: a Striatal Behavioral Learning Model. Cognit Comput 2021. [DOI: 10.1007/s12559-021-09950-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
7
|
Dominey PF. Narrative event segmentation in the cortical reservoir. PLoS Comput Biol 2021; 17:e1008993. [PMID: 34618804 PMCID: PMC8525778 DOI: 10.1371/journal.pcbi.1008993] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2021] [Revised: 10/19/2021] [Accepted: 09/08/2021] [Indexed: 01/04/2023] Open
Abstract
Recent research has revealed that during continuous perception of movies or stories, humans display cortical activity patterns that reveal hierarchical segmentation of event structure. Thus, sensory areas like auditory cortex display high frequency segmentation related to the stimulus, while semantic areas like posterior middle cortex display a lower frequency segmentation related to transitions between events. These hierarchical levels of segmentation are associated with different time constants for processing. Likewise, when two groups of participants heard the same sentence in a narrative, preceded by different contexts, neural responses for the groups were initially different and then gradually aligned. The time constant for alignment followed the segmentation hierarchy: sensory cortices aligned most quickly, followed by mid-level regions, while some higher-order cortical regions took more than 10 seconds to align. These hierarchical segmentation phenomena can be considered in the context of processing related to comprehension. In a recently described model of discourse comprehension word meanings are modeled by a language model pre-trained on a billion word corpus. During discourse comprehension, word meanings are continuously integrated in a recurrent cortical network. The model demonstrates novel discourse and inference processing, in part because of two fundamental characteristics: real-world event semantics are represented in the word embeddings, and these are integrated in a reservoir network which has an inherent gradient of functional time constants due to the recurrent connections. Here we demonstrate how this model displays hierarchical narrative event segmentation properties beyond the embeddings alone, or their linear integration. The reservoir produces activation patterns that are segmented by a hidden Markov model (HMM) in a manner that is comparable to that of humans. Context construction displays a continuum of time constants across reservoir neuron subsets, while context forgetting has a fixed time constant across these subsets. Importantly, virtual areas formed by subgroups of reservoir neurons with faster time constants segmented with shorter events, while those with longer time constants preferred longer events. This neurocomputational recurrent neural network simulates narrative event processing as revealed by the fMRI event segmentation algorithm provides a novel explanation of the asymmetry in narrative forgetting and construction. The model extends the characterization of online integration processes in discourse to more extended narrative, and demonstrates how reservoir computing provides a useful model of cortical processing of narrative structure.
Collapse
Affiliation(s)
- Peter Ford Dominey
- INSERM UMR1093-CAPS, Université Bourgogne Franche-Comté, UFR des Sciences du Sport, Dijon
- Robot Cognition Laboratory, Institute Marey, Dijon
| |
Collapse
|
8
|
Wittkuhn L, Chien S, Hall-McMaster S, Schuck NW. Replay in minds and machines. Neurosci Biobehav Rev 2021; 129:367-388. [PMID: 34371078 DOI: 10.1016/j.neubiorev.2021.08.002] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2021] [Revised: 07/19/2021] [Accepted: 08/01/2021] [Indexed: 11/19/2022]
Abstract
Experience-related brain activity patterns reactivate during sleep, wakeful rest, and brief pauses from active behavior. In parallel, machine learning research has found that experience replay can lead to substantial performance improvements in artificial agents. Together, these lines of research suggest replay has a variety of computational benefits for decision-making and learning. Here, we provide an overview of putative computational functions of replay as suggested by machine learning and neuroscientific research. We show that replay can lead to faster learning, less forgetting, reorganization or augmentation of experiences, and support planning and generalization. In addition, we highlight the benefits of reactivating abstracted internal representations rather than veridical memories, and discuss how replay could provide a mechanism to build internal representations that improve learning and decision-making.
Collapse
Affiliation(s)
- Lennart Wittkuhn
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Lentzeallee 94, D-14195 Berlin, Germany; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Lentzeallee 94, D-14195 Berlin, Germany.
| | - Samson Chien
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Lentzeallee 94, D-14195 Berlin, Germany; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Lentzeallee 94, D-14195 Berlin, Germany
| | - Sam Hall-McMaster
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Lentzeallee 94, D-14195 Berlin, Germany; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Lentzeallee 94, D-14195 Berlin, Germany
| | - Nicolas W Schuck
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Lentzeallee 94, D-14195 Berlin, Germany; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Lentzeallee 94, D-14195 Berlin, Germany.
| |
Collapse
|
9
|
Biological constraints on neural network models of cognitive function. Nat Rev Neurosci 2021; 22:488-502. [PMID: 34183826 PMCID: PMC7612527 DOI: 10.1038/s41583-021-00473-5] [Citation(s) in RCA: 36] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/17/2021] [Indexed: 02/06/2023]
Abstract
Neural network models are potential tools for improving our understanding of complex brain functions. To address this goal, these models need to be neurobiologically realistic. However, although neural networks have advanced dramatically in recent years and even achieve human-like performance on complex perceptual and cognitive tasks, their similarity to aspects of brain anatomy and physiology is imperfect. Here, we discuss different types of neural models, including localist, auto-associative, hetero-associative, deep and whole-brain networks, and identify aspects under which their biological plausibility can be improved. These aspects range from the choice of model neurons and of mechanisms of synaptic plasticity and learning to implementation of inhibition and control, along with neuroanatomical properties including areal structure and local and long-range connectivity. We highlight recent advances in developing biologically grounded cognitive theories and in mechanistically explaining, on the basis of these brain-constrained neural models, hitherto unaddressed issues regarding the nature, localization and ontogenetic and phylogenetic development of higher brain functions. In closing, we point to possible future clinical applications of brain-constrained modelling.
Collapse
|
10
|
NLM-HS: Navigation Learning Model Based on a Hippocampal-Striatal Circuit for Explaining Navigation Mechanisms in Animal Brains. Brain Sci 2021; 11:brainsci11060803. [PMID: 34204482 PMCID: PMC8235547 DOI: 10.3390/brainsci11060803] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2021] [Revised: 06/04/2021] [Accepted: 06/15/2021] [Indexed: 11/17/2022] Open
Abstract
Neurophysiological studies have shown that the hippocampus, striatum, and prefrontal cortex play different roles in animal navigation, but it is still less clear how these structures work together. In this paper, we establish a navigation learning model based on the hippocampal-striatal circuit (NLM-HS), which provides a possible explanation for the navigation mechanism in the animal brain. The hippocampal model generates a cognitive map of the environment and performs goal-directed navigation by using a place cell sequence planning algorithm. The striatal model performs reward-related habitual navigation by using the classic temporal difference learning algorithm. Since the two models may produce inconsistent behavioral decisions, the prefrontal cortex model chooses the most appropriate strategies by using a strategy arbitration mechanism. The cognitive and learning mechanism of the NLM-HS works in two stages of exploration and navigation. First, the agent uses a hippocampal model to construct the cognitive map of the unknown environment. Then, the agent uses the strategy arbitration mechanism in the prefrontal cortex model to directly decide which strategy to choose. To test the validity of the NLM-HS, the classical Tolman detour experiment was reproduced. The results show that the NLM-HS not only makes agents show environmental cognition and navigation behavior similar to animals, but also makes behavioral decisions faster and achieves better adaptivity than hippocampal or striatal models alone.
Collapse
|
11
|
Bermudez-Contreras E, Clark BJ, Wilber A. The Neuroscience of Spatial Navigation and the Relationship to Artificial Intelligence. Front Comput Neurosci 2020; 14:63. [PMID: 32848684 PMCID: PMC7399088 DOI: 10.3389/fncom.2020.00063] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2019] [Accepted: 05/28/2020] [Indexed: 11/13/2022] Open
Abstract
Recent advances in artificial intelligence (AI) and neuroscience are impressive. In AI, this includes the development of computer programs that can beat a grandmaster at GO or outperform human radiologists at cancer detection. A great deal of these technological developments are directly related to progress in artificial neural networks-initially inspired by our knowledge about how the brain carries out computation. In parallel, neuroscience has also experienced significant advances in understanding the brain. For example, in the field of spatial navigation, knowledge about the mechanisms and brain regions involved in neural computations of cognitive maps-an internal representation of space-recently received the Nobel Prize in medicine. Much of the recent progress in neuroscience has partly been due to the development of technology used to record from very large populations of neurons in multiple regions of the brain with exquisite temporal and spatial resolution in behaving animals. With the advent of the vast quantities of data that these techniques allow us to collect there has been an increased interest in the intersection between AI and neuroscience, many of these intersections involve using AI as a novel tool to explore and analyze these large data sets. However, given the common initial motivation point-to understand the brain-these disciplines could be more strongly linked. Currently much of this potential synergy is not being realized. We propose that spatial navigation is an excellent area in which these two disciplines can converge to help advance what we know about the brain. In this review, we first summarize progress in the neuroscience of spatial navigation and reinforcement learning. We then turn our attention to discuss how spatial navigation has been modeled using descriptive, mechanistic, and normative approaches and the use of AI in such models. Next, we discuss how AI can advance neuroscience, how neuroscience can advance AI, and the limitations of these approaches. We finally conclude by highlighting promising lines of research in which spatial navigation can be the point of intersection between neuroscience and AI and how this can contribute to the advancement of the understanding of intelligent behavior.
Collapse
Affiliation(s)
| | - Benjamin J. Clark
- Department of Psychology, University of New Mexico, Albuquerque, NM, United States
| | - Aaron Wilber
- Department of Psychology, Program in Neuroscience, Florida State University, Tallahassee, FL, United States
| |
Collapse
|
12
|
Xiao Z, Lin K, Fellous JM. Conjunctive reward-place coding properties of dorsal distal CA1 hippocampus cells. BIOLOGICAL CYBERNETICS 2020; 114:285-301. [PMID: 32266474 DOI: 10.1007/s00422-020-00830-0] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/17/2019] [Accepted: 03/25/2020] [Indexed: 06/11/2023]
Abstract
Autonomous motivated spatial navigation in animals or robots requires the association between spatial location and value. Hippocampal place cells are involved in goal-directed spatial navigation and the consolidation of spatial memories. Recently, Gauthier and Tank (Neuron 99(1):179-193, 2018. https://doi.org/10.1016/j.neuron.2018.06.008) have identified a subpopulation of hippocampal cells selectively activated in relation to rewarded goals. However, the relationship between these cells' spiking activity and goal representation remains elusive. We analyzed data from experiments in which rats underwent five consecutive tasks in which reward locations and spatial context were manipulated. We found CA1 populations with properties continuously ranging from place cells to reward cells. Specifically, we found typical place cells insensitive to reward locations, reward cells that only fired at correct rewarded feeders in each task regardless of context, and "hybrid cells" that responded to spatial locations and change of reward locations. Reward cells responded mostly to the reward delivery rather than to its expectation. In addition, we found a small group of neurons that transitioned between place and reward cells properties within the 5-task session. We conclude that some pyramidal cells (if not all) integrate both spatial and reward inputs to various degrees. These results provide insights into the integrative coding properties of CA1 pyramidal cells, focusing on their abilities to carry both spatial and reward information in a mixed and plastic manner. This conjunctive coding property prompts a re-thinking of current computational models of spatial navigation in which hippocampal spatial and subcortical value representations are independent.
Collapse
Affiliation(s)
- Zhuocheng Xiao
- Program in Applied Mathematics, University of Arizona, Tucson, AZ, 85721, USA
| | - Kevin Lin
- Program in Applied Mathematics, University of Arizona, Tucson, AZ, 85721, USA
- Department of Mathematics, University of Arizona, Tucson, AZ, 85721, USA
- Neuroscience Graduate Interdisciplinary Program, University of Arizona, Tucson, AZ, 85721, USA
| | - Jean-Marc Fellous
- Program in Applied Mathematics, University of Arizona, Tucson, AZ, 85721, USA.
- Department of Psychology, University of Arizona, 1503 E University Blvd, Suite 312, Tucson, AZ, 85721, USA.
- Neuroscience Graduate Interdisciplinary Program, University of Arizona, Tucson, AZ, 85721, USA.
| |
Collapse
|
13
|
Cazin N, Scleidorovich P, Weitzenfeld A, Dominey PF. Real-time sensory-motor integration of hippocampal place cell replay and prefrontal sequence learning in simulated and physical rat robots for novel path optimization. BIOLOGICAL CYBERNETICS 2020; 114:249-268. [PMID: 32095878 DOI: 10.1007/s00422-020-00820-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/03/2019] [Accepted: 02/04/2020] [Indexed: 06/10/2023]
Abstract
An open problem in the cognitive dimensions of navigation concerns how previous exploratory experience is reorganized in order to allow the creation of novel efficient navigation trajectories. This behavior is revealed in the "traveling salesrat problem" (TSP) when rats discover the shortest path linking baited food wells after a few exploratory traversals. We have recently published a model of navigation sequence learning, where sharp wave ripple replay of hippocampal place cells transmit "snippets" of the recent trajectories that the animal has explored to the prefrontal cortex (PFC) (Cazin et al. in PLoS Comput Biol 15:e1006624, 2019). PFC is modeled as a recurrent reservoir network that is able to assemble these snippets into the efficient sequence (trajectory of spatial locations coded by place cell activation). The model of hippocampal replay generates a distribution of snippets as a function of their proximity to a reward, thus implementing a form of spatial credit assignment that solves the TSP task. The integrative PFC reservoir reconstructs the efficient TSP sequence based on exposure to this distribution of snippets that favors paths that are most proximal to rewards. While this demonstrates the theoretical feasibility of the PFC-HIPP interaction, the integration of such a dynamic system into a real-time sensory-motor system remains a challenge. In the current research, we test the hypothesis that the PFC reservoir model can operate in a real-time sensory-motor loop. Thus, the main goal of the paper is to validate the model in simulated and real robot scenarios. Place cell activation encoding the current position of the simulated and physical rat robot feeds the PFC reservoir which generates the successor place cell activation that represents the next step in the reproduced sequence in the readout. This is input to the robot, which advances to the coded location and then generates de novo the current place cell activation. This allows demonstration of the crucial role of embodiment. If the spatial code readout from PFC is played back directly into PFC, error can accumulate, and the system can diverge from desired trajectories. This required a spatial filter to decode the PFC code to a location and then recode a new place cell code for that location. In the robot, the place cell vector output of PFC is used to physically displace the robot and then generate a new place cell coded input to the PFC, replacing part of the software recoding procedure that was required otherwise. We demonstrate how this integrated sensory-motor system can learn simple navigation sequences and then, importantly, how it can synthesize novel efficient sequences based on prior experience, as previously demonstrated (Cazin et al. 2019). This contributes to the understanding of hippocampal replay in novel navigation sequence formation and the important role of embodiment.
Collapse
Affiliation(s)
- Nicolas Cazin
- INSERM UMR1093-CAPS, Université Bourgogne Franche-Comté, UFR des Sciences du Sport, 21000, Dijon, France
- Robot Cognition Laboratory, Institut Marey, INSERM U1093 CAPS, UBFC, Dijon, France
| | | | | | - Peter Ford Dominey
- INSERM UMR1093-CAPS, Université Bourgogne Franche-Comté, UFR des Sciences du Sport, 21000, Dijon, France.
- Robot Cognition Laboratory, Institut Marey, INSERM U1093 CAPS, UBFC, Dijon, France.
| |
Collapse
|
14
|
Arbib MA. From spatial navigation via visual construction to episodic memory and imagination. BIOLOGICAL CYBERNETICS 2020; 114:139-167. [PMID: 32285205 PMCID: PMC7152744 DOI: 10.1007/s00422-020-00829-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/18/2019] [Accepted: 03/25/2020] [Indexed: 06/11/2023]
Abstract
This hybrid of review and personal essay argues that models of visual construction are essential to extend spatial navigation models to models that link episodic memory and imagination. The starting point is the TAM-WG model, combining the Taxon Affordance Model and the World Graph model of spatial navigation. The key here is to reject approaches in which memory is restricted to unanalyzed views from familiar places, and their later recall. Instead, we will seek mechanisms for imagining truly novel scenes and episodes. We thus introduce a specific variant of schema theory and VISIONS, a cooperative computation model of visual scene understanding in which a scene is represented by an assemblage of schema instances with links to lower-level "patches" of relevant visual data. We sketch a new conceptual framework for future modeling, Visual Integration of Diverse Multi-Modal Aspects, by extending VISIONS from static scenes to episodes combining agents, actions and objects and assess its relevance to both navigation and episodic memory. We can then analyze imagination as a constructive process that combines aspects of memories of prior episodes along with other schemas and adjusts them into a coherent whole which, through expectations associated with diverse episodes and schemas, may yield the linkage of episodes that constitutes a dream or a narrative. The result is IBSEN, a conceptual model of Imagination in Brain Systems for Episodes and Navigation. The essay closes by analyzing other papers in this Special Issue to assess to what extent their results relate to the research proposed here.
Collapse
|