1
|
Son JY, Vives ML, Bhandari A, FeldmanHall O. Replay shapes abstract cognitive maps for efficient social navigation. Nat Hum Behav 2024:10.1038/s41562-024-01990-w. [PMID: 39300309 DOI: 10.1038/s41562-024-01990-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2024] [Accepted: 08/16/2024] [Indexed: 09/22/2024]
Abstract
To make adaptive social decisions, people must anticipate how information flows through their social network. While this requires knowledge of how people are connected, networks are too large to have first-hand experience with every possible route between individuals. How, then, are people able to accurately track information flow through social networks? Here we find that people immediately cache abstract knowledge about social network structure as they learn who is friends with whom, which enables the identification of efficient routes between remotely connected individuals. These cognitive maps of social networks, which are built while learning, are then reshaped through overnight rest. During these extended periods of rest, a replay-like mechanism helps to make these maps increasingly abstract, which privileges improvements in social navigation accuracy for the longest communication paths that span distinct communities within the network. Together, these findings provide mechanistic insight into the sophisticated mental representations humans use for social navigation.
Collapse
Affiliation(s)
- Jae-Young Son
- Department of Cognitive and Psychological Sciences, Brown University, Providence, RI, USA
| | - Marc-Lluís Vives
- Institute of Psychology, Leiden University, Leiden, The Netherlands
| | - Apoorva Bhandari
- Department of Cognitive and Psychological Sciences, Brown University, Providence, RI, USA.
| | - Oriel FeldmanHall
- Department of Cognitive and Psychological Sciences, Brown University, Providence, RI, USA.
- Robert J. and Nancy D. Carney Institute for Brain Science, Brown University, Providence, RI, USA.
| |
Collapse
|
2
|
Kowadlo G, Ahmed A, Mayan A, Rawlinson D. Expanding continual few-shot learning benchmarks to include recognition of specific instances. PLoS One 2024; 19:e0305856. [PMID: 38968250 PMCID: PMC11226023 DOI: 10.1371/journal.pone.0305856] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2024] [Accepted: 06/05/2024] [Indexed: 07/07/2024] Open
Abstract
Continual learning and few-shot learning are important frontiers in progress toward broader Machine Learning (ML) capabilities. Recently, there has been intense interest in combining both. One of the first examples to do so was the Continual few-shot Learning (CFSL) framework of Antoniou et al. (2020). In this study, we extend CFSL in two ways that capture a broader range of challenges, important for intelligent agent behaviour in real-world conditions. First, we increased the number of classes by an order of magnitude, making the results more comparable to standard continual learning experiments. Second, we introduced an 'instance test' which requires recognition of specific instances of classes-a capability of animal cognition that is usually neglected in ML. For an initial exploration of ML model performance under these conditions, we selected representative baseline models from the original CFSL work and added a model variant with replay. As expected, learning more classes is more difficult than the original CFSL experiments, and interestingly, the way in which image instances and classes are presented affects classification performance. Surprisingly, accuracy in the baseline instance test is comparable to other classification tasks, but poor given significant occlusion and noise. The use of replay for consolidation substantially improves performance for both types of tasks, but particularly for the instance test.
Collapse
|
3
|
Rolls ET. The memory systems of the human brain and generative artificial intelligence. Heliyon 2024; 10:e31965. [PMID: 38841455 PMCID: PMC11152951 DOI: 10.1016/j.heliyon.2024.e31965] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Revised: 05/11/2024] [Accepted: 05/24/2024] [Indexed: 06/07/2024] Open
Abstract
Generative Artificial Intelligence foundation models (for example Generative Pre-trained Transformer - GPT - models) can generate the next token given a sequence of tokens. How can this 'generative AI' be compared with the 'real' intelligence of the human brain, when for example a human generates a whole memory in response to an incomplete retrieval cue, and then generates further prospective thoughts? Here these two types of generative intelligence, artificial in machines and real in the human brain are compared, and it is shown how when whole memories are generated by hippocampal recall in response to an incomplete retrieval cue, what the human brain computes, and how it computes it, are very different from generative AI. Key differences are the use of local associative learning rules in the hippocampal memory system, and of non-local backpropagation of error learning in AI. Indeed, it is argued that the whole operation of the human brain is performed computationally very differently to what is implemented in generative AI. Moreover, it is emphasized that the primate including human hippocampal system includes computations about spatial view and where objects and people are in scenes, whereas in rodents the emphasis is on place cells and path integration by movements between places. This comparison with generative memory and processing in the human brain has interesting implications for the further development of generative AI and for neuroscience research.
Collapse
Affiliation(s)
- Edmund T. Rolls
- Oxford Centre for Computational Neuroscience, Oxford, UK
- Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai, 200403, China
| |
Collapse
|
4
|
Esmaeilzadeh P. Challenges and strategies for wide-scale artificial intelligence (AI) deployment in healthcare practices: A perspective for healthcare organizations. Artif Intell Med 2024; 151:102861. [PMID: 38555850 DOI: 10.1016/j.artmed.2024.102861] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2023] [Revised: 03/19/2024] [Accepted: 03/25/2024] [Indexed: 04/02/2024]
Abstract
Healthcare organizations have realized that Artificial intelligence (AI) can provide a competitive edge through personalized patient experiences, improved patient outcomes, early diagnosis, augmented clinician capabilities, enhanced operational efficiencies, or improved medical service accessibility. However, deploying AI-driven tools in the healthcare ecosystem could be challenging. This paper categorizes AI applications in healthcare and comprehensively examines the challenges associated with deploying AI in medical practices at scale. As AI continues to make strides in healthcare, its integration presents various challenges, including production timelines, trust generation, privacy concerns, algorithmic biases, and data scarcity. The paper highlights that flawed business models and wrong workflows in healthcare practices cannot be rectified merely by deploying AI-driven tools. Healthcare organizations should re-evaluate root problems such as misaligned financial incentives (e.g., fee-for-service models), dysfunctional medical workflows (e.g., high rates of patient readmissions), poor care coordination between different providers, fragmented electronic health records systems, and inadequate patient education and engagement models in tandem with AI adoption. This study also explores the need for a cultural shift in viewing AI not as a threat but as an enabler that can enhance healthcare delivery and create new employment opportunities while emphasizing the importance of addressing underlying operational issues. The necessity of investments beyond finance is discussed, emphasizing the importance of human capital, continuous learning, and a supportive environment for AI integration. The paper also highlights the crucial role of clear regulations in building trust, ensuring safety, and guiding the ethical use of AI, calling for coherent frameworks addressing transparency, model accuracy, data quality control, liability, and ethics. Furthermore, this paper underscores the importance of advancing AI literacy within academia to prepare future healthcare professionals for an AI-driven landscape. Through careful navigation and proactive measures addressing these challenges, the healthcare community can harness AI's transformative power responsibly and effectively, revolutionizing healthcare delivery and patient care. The paper concludes with a vision and strategic suggestions for the future of healthcare with AI, emphasizing thoughtful, responsible, and innovative engagement as the pathway to realizing its full potential to unlock immense benefits for healthcare organizations, physicians, nurses, and patients while proactively mitigating risks.
Collapse
Affiliation(s)
- Pouyan Esmaeilzadeh
- Department of Information Systems and Business Analytics, College of Business, Florida International University (FIU), Modesto A. Maidique Campus, 11200 S.W. 8th St, RB 261B, Miami, FL 33199, United States.
| |
Collapse
|
5
|
Pezzulo G, D'Amato L, Mannella F, Priorelli M, Van de Maele T, Stoianov IP, Friston K. Neural representation in active inference: Using generative models to interact with-and understand-the lived world. Ann N Y Acad Sci 2024; 1534:45-68. [PMID: 38528782 DOI: 10.1111/nyas.15118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/27/2024]
Abstract
This paper considers neural representation through the lens of active inference, a normative framework for understanding brain function. It delves into how living organisms employ generative models to minimize the discrepancy between predictions and observations (as scored with variational free energy). The ensuing analysis suggests that the brain learns generative models to navigate the world adaptively, not (or not solely) to understand it. Different living organisms may possess an array of generative models, spanning from those that support action-perception cycles to those that underwrite planning and imagination; namely, from explicit models that entail variables for predicting concurrent sensations, like objects, faces, or people-to action-oriented models that predict action outcomes. It then elucidates how generative models and belief dynamics might link to neural representation and the implications of different types of generative models for understanding an agent's cognitive capabilities in relation to its ecological niche. The paper concludes with open questions regarding the evolution of generative models and the development of advanced cognitive abilities-and the gradual transition from pragmatic to detached neural representations. The analysis on offer foregrounds the diverse roles that generative models play in cognitive processes and the evolution of neural representation.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Leo D'Amato
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
- Polytechnic University of Turin, Turin, Italy
| | - Francesco Mannella
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Matteo Priorelli
- Institute of Cognitive Sciences and Technologies, National Research Council, Padua, Italy
| | - Toon Van de Maele
- IDLab, Department of Information Technology, Ghent University - imec, Ghent, Belgium
| | - Ivilin Peev Stoianov
- Institute of Cognitive Sciences and Technologies, National Research Council, Padua, Italy
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London, UK
- VERSES Research Lab, Los Angeles, California, USA
| |
Collapse
|
6
|
Berger JI, Billig AJ, Sedley W, Kumar S, Griffiths TD, Gander PE. What is the role of the hippocampus and parahippocampal gyrus in the persistence of tinnitus? Hum Brain Mapp 2024; 45:e26627. [PMID: 38376166 PMCID: PMC10878198 DOI: 10.1002/hbm.26627] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Revised: 01/12/2024] [Accepted: 02/04/2024] [Indexed: 02/21/2024] Open
Abstract
The hippocampus and parahippocampal gyrus have been implicated as part of a tinnitus network by a number of studies. These structures are usually considered in the context of a "limbic system," a concept typically invoked to explain the emotional response to tinnitus. Despite this common framing, it is not apparent from current literature that this is necessarily the main functional role of these structures in persistent tinnitus. Here, we highlight a different role that encompasses their most commonly implicated functional position within the brain-that is, as a memory system. We consider tinnitus as an auditory object that is held in memory, which may be made persistent by associated activity from the hippocampus and parahippocampal gyrus. Evidence from animal and human studies implicating these structures in tinnitus is reviewed and used as an anchor for this hypothesis. We highlight the potential for the hippocampus/parahippocampal gyrus to facilitate maintenance of the memory of the tinnitus percept via communication with auditory cortex, rather than (or in addition to) mediating emotional responses to this percept.
Collapse
Affiliation(s)
- Joel I. Berger
- Department of NeurosurgeryUniversity of IowaIowa CityIowaUSA
| | | | | | | | | | - Phillip E. Gander
- Department of NeurosurgeryUniversity of IowaIowa CityIowaUSA
- Department of RadiologyUniversity of IowaIowa CityIowaUSA
| |
Collapse
|
7
|
Pezzulo G, Parr T, Friston K. Active inference as a theory of sentient behavior. Biol Psychol 2024; 186:108741. [PMID: 38182015 DOI: 10.1016/j.biopsycho.2023.108741] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 12/05/2023] [Accepted: 12/29/2023] [Indexed: 01/07/2024]
Abstract
This review paper offers an overview of the history and future of active inference-a unifying perspective on action and perception. Active inference is based upon the idea that sentient behavior depends upon our brains' implicit use of internal models to predict, infer, and direct action. Our focus is upon the conceptual roots and development of this theory of (basic) sentience and does not follow a rigid chronological narrative. We trace the evolution from Helmholtzian ideas on unconscious inference, through to a contemporary understanding of action and perception. In doing so, we touch upon related perspectives, the neural underpinnings of active inference, and the opportunities for future development. Key steps in this development include the formulation of predictive coding models and related theories of neuronal message passing, the use of sequential models for planning and policy optimization, and the importance of hierarchical (temporally) deep internal (i.e., generative or world) models. Active inference has been used to account for aspects of anatomy and neurophysiology, to offer theories of psychopathology in terms of aberrant precision control, and to unify extant psychological theories. We anticipate further development in all these areas and note the exciting early work applying active inference beyond neuroscience. This suggests a future not just in biology, but in robotics, machine learning, and artificial intelligence.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| | - Thomas Parr
- Nuffield Department of Clinical Neurosciences, University of Oxford, UK
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London, UK; VERSES AI Research Lab, Los Angeles, CA 90016, USA
| |
Collapse
|
8
|
Pezzulo G, Parr T, Cisek P, Clark A, Friston K. Generating meaning: active inference and the scope and limits of passive AI. Trends Cogn Sci 2024; 28:97-112. [PMID: 37973519 DOI: 10.1016/j.tics.2023.10.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Revised: 10/03/2023] [Accepted: 10/05/2023] [Indexed: 11/19/2023]
Abstract
Prominent accounts of sentient behavior depict brains as generative models of organismic interaction with the world, evincing intriguing similarities with current advances in generative artificial intelligence (AI). However, because they contend with the control of purposive, life-sustaining sensorimotor interactions, the generative models of living organisms are inextricably anchored to the body and world. Unlike the passive models learned by generative AI systems, they must capture and control the sensory consequences of action. This allows embodied agents to intervene upon their worlds in ways that constantly put their best models to the test, thus providing a solid bedrock that is - we argue - essential to the development of genuine understanding. We review the resulting implications and consider future directions for generative AI.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| | - Thomas Parr
- Nuffield Department of Clinical Neurosciences, University of Oxford
| | - Paul Cisek
- Department of Neuroscience, University of Montréal, Montréal, Québec, Canada
| | - Andy Clark
- Department of Philosophy, University of Sussex, Brighton, UK; Department of Informatics, University of Sussex, Brighton, UK; Department of Philosophy, Macquarie University, Sydney, New South Wales, Australia
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London, UK; VERSES AI Research Lab, Los Angeles, CA, USA
| |
Collapse
|
9
|
de Tinguy D, Van de Maele T, Verbelen T, Dhoedt B. Spatial and Temporal Hierarchy for Autonomous Navigation Using Active Inference in Minigrid Environment. ENTROPY (BASEL, SWITZERLAND) 2024; 26:83. [PMID: 38248208 PMCID: PMC11154534 DOI: 10.3390/e26010083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 01/05/2024] [Accepted: 01/12/2024] [Indexed: 01/23/2024]
Abstract
Robust evidence suggests that humans explore their environment using a combination of topological landmarks and coarse-grained path integration. This approach relies on identifiable environmental features (topological landmarks) in tandem with estimations of distance and direction (coarse-grained path integration) to construct cognitive maps of the surroundings. This cognitive map is believed to exhibit a hierarchical structure, allowing efficient planning when solving complex navigation tasks. Inspired by human behaviour, this paper presents a scalable hierarchical active inference model for autonomous navigation, exploration, and goal-oriented behaviour. The model uses visual observation and motion perception to combine curiosity-driven exploration with goal-oriented behaviour. Motion is planned using different levels of reasoning, i.e., from context to place to motion. This allows for efficient navigation in new spaces and rapid progress toward a target. By incorporating these human navigational strategies and their hierarchical representation of the environment, this model proposes a new solution for autonomous navigation and exploration. The approach is validated through simulations in a mini-grid environment.
Collapse
Affiliation(s)
| | | | - Tim Verbelen
- VERSES AI Research Lab, Los Angeles, CA 90016, USA;
| | | |
Collapse
|
10
|
Friston KJ, Parr T, Heins C, Constant A, Friedman D, Isomura T, Fields C, Verbelen T, Ramstead M, Clippinger J, Frith CD. Federated inference and belief sharing. Neurosci Biobehav Rev 2024; 156:105500. [PMID: 38056542 PMCID: PMC11139662 DOI: 10.1016/j.neubiorev.2023.105500] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Revised: 11/08/2023] [Accepted: 12/01/2023] [Indexed: 12/08/2023]
Abstract
This paper concerns the distributed intelligence or federated inference that emerges under belief-sharing among agents who share a common world-and world model. Imagine, for example, several animals keeping a lookout for predators. Their collective surveillance rests upon being able to communicate their beliefs-about what they see-among themselves. But, how is this possible? Here, we show how all the necessary components arise from minimising free energy. We use numerical studies to simulate the generation, acquisition and emergence of language in synthetic agents. Specifically, we consider inference, learning and selection as minimising the variational free energy of posterior (i.e., Bayesian) beliefs about the states, parameters and structure of generative models, respectively. The common theme-that attends these optimisation processes-is the selection of actions that minimise expected free energy, leading to active inference, learning and model selection (a.k.a., structure learning). We first illustrate the role of communication in resolving uncertainty about the latent states of a partially observed world, on which agents have complementary perspectives. We then consider the acquisition of the requisite language-entailed by a likelihood mapping from an agent's beliefs to their overt expression (e.g., speech)-showing that language can be transmitted across generations by active learning. Finally, we show that language is an emergent property of free energy minimisation, when agents operate within the same econiche. We conclude with a discussion of various perspectives on these phenomena; ranging from cultural niche construction, through federated learning, to the emergence of complexity in ensembles of self-organising systems.
Collapse
Affiliation(s)
- Karl J Friston
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, UK; VERSES AI Research Lab, Los Angeles, CA 90016, USA.
| | - Thomas Parr
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, UK
| | - Conor Heins
- VERSES AI Research Lab, Los Angeles, CA 90016, USA; Department of Collective Behaviour, Max Planck Institute of Animal Behavior, 78457 Konstanz, Germany; Centre for the Advanced Study of Collective Behaviour, 78457 Konstanz, Germany; Department of Biology, University of Konstanz, 78457 Konstanz, Germany
| | - Axel Constant
- VERSES AI Research Lab, Los Angeles, CA 90016, USA; School of Engineering and Informatics, The University of Sussex, Brighton, UK
| | - Daniel Friedman
- Department of Entomology and Nematology, University of California, Davis, Davis, CA, USA; Active Inference Institute, Davis, CA 95616, USA
| | - Takuya Isomura
- Brain Intelligence Theory Unit, RIKEN Center for Brain Science, Wako, Saitama 351-0198, Japan
| | - Chris Fields
- Allen Discovery Center at Tufts University, Medford, MA 02155, USA
| | - Tim Verbelen
- VERSES AI Research Lab, Los Angeles, CA 90016, USA
| | - Maxwell Ramstead
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, UK; VERSES AI Research Lab, Los Angeles, CA 90016, USA
| | | | - Christopher D Frith
- Institute of Philosophy, School of Advanced Studies, University of London, UK
| |
Collapse
|
11
|
Schwartenbeck P, Baram A, Liu Y, Mark S, Muller T, Dolan R, Botvinick M, Kurth-Nelson Z, Behrens T. Generative replay underlies compositional inference in the hippocampal-prefrontal circuit. Cell 2023; 186:4885-4897.e14. [PMID: 37804832 PMCID: PMC10914680 DOI: 10.1016/j.cell.2023.09.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 01/23/2023] [Accepted: 09/06/2023] [Indexed: 10/09/2023]
Abstract
Human reasoning depends on reusing pieces of information by putting them together in new ways. However, very little is known about how compositional computation is implemented in the brain. Here, we ask participants to solve a series of problems that each require constructing a whole from a set of elements. With fMRI, we find that representations of novel constructed objects in the frontal cortex and hippocampus are relational and compositional. With MEG, we find that replay assembles elements into compounds, with each replay sequence constituting a hypothesis about a possible configuration of elements. The content of sequences evolves as participants solve each puzzle, progressing from predictable to uncertain elements and gradually converging on the correct configuration. Together, these results suggest a computational bridge between apparently distinct functions of hippocampal-prefrontal circuitry and a role for generative replay in compositional inference and hypothesis testing.
Collapse
Affiliation(s)
- Philipp Schwartenbeck
- University of Tübingen, Tübingen, Germany; Max Planck Institute for Biological Cybernetics, Tübingen, Baden-Württemberg, Germany; Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK.
| | - Alon Baram
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Shirley Mark
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK
| | - Timothy Muller
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK; Institute of Neurology, University College London, London WC1N 3BG, UK
| | - Raymond Dolan
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Department of Psychiatry, Universitätsmedizin Berlin (Campus Charité Mitte), Berlin, Germany
| | - Matthew Botvinick
- Google DeepMind, London, UK; Gatsby Computational Neuroscience Unit, University College London, London, UK
| | - Zeb Kurth-Nelson
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Google DeepMind, London, UK
| | - Timothy Behrens
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK; Sainsbury Wellcome Centre for Neural Circuits and Behaviour, UCL, London W1T 4JG, UK
| |
Collapse
|
12
|
Mehrotra D, Dubé L. Accounting for multiscale processing in adaptive real-world decision-making via the hippocampus. Front Neurosci 2023; 17:1200842. [PMID: 37732307 PMCID: PMC10508350 DOI: 10.3389/fnins.2023.1200842] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Accepted: 08/25/2023] [Indexed: 09/22/2023] Open
Abstract
For adaptive real-time behavior in real-world contexts, the brain needs to allow past information over multiple timescales to influence current processing for making choices that create the best outcome as a person goes about making choices in their everyday life. The neuroeconomics literature on value-based decision-making has formalized such choice through reinforcement learning models for two extreme strategies. These strategies are model-free (MF), which is an automatic, stimulus-response type of action, and model-based (MB), which bases choice on cognitive representations of the world and causal inference on environment-behavior structure. The emphasis of examining the neural substrates of value-based decision making has been on the striatum and prefrontal regions, especially with regards to the "here and now" decision-making. Yet, such a dichotomy does not embrace all the dynamic complexity involved. In addition, despite robust research on the role of the hippocampus in memory and spatial learning, its contribution to value-based decision making is just starting to be explored. This paper aims to better appreciate the role of the hippocampus in decision-making and advance the successor representation (SR) as a candidate mechanism for encoding state representations in the hippocampus, separate from reward representations. To this end, we review research that relates hippocampal sequences to SR models showing that the implementation of such sequences in reinforcement learning agents improves their performance. This also enables the agents to perform multiscale temporal processing in a biologically plausible manner. Altogether, we articulate a framework to advance current striatal and prefrontal-focused decision making to better account for multiscale mechanisms underlying various real-world time-related concepts such as the self that cumulates over a person's life course.
Collapse
Affiliation(s)
- Dhruv Mehrotra
- Integrated Program in Neuroscience, McGill University, Montréal, QC, Canada
- Montréal Neurological Institute, McGill University, Montréal, QC, Canada
| | - Laurette Dubé
- Desautels Faculty of Management, McGill University, Montréal, QC, Canada
- McGill Center for the Convergence of Health and Economics, McGill University, Montréal, QC, Canada
| |
Collapse
|
13
|
Yesiltepe D, Fernández Velasco P, Coutrot A, Ozbil Torun A, Wiener JM, Holscher C, Hornberger M, Conroy Dalton R, Spiers HJ. Entropy and a sub-group of geometric measures of paths predict the navigability of an environment. Cognition 2023; 236:105443. [PMID: 37003236 DOI: 10.1016/j.cognition.2023.105443] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 02/01/2023] [Accepted: 03/12/2023] [Indexed: 04/03/2023]
Abstract
Despite extensive research on navigation, it remains unclear which features of an environment predict how difficult it will be to navigate. We analysed 478,170 trajectories from 10,626 participants who navigated 45 virtual environments in the research app-based game Sea Hero Quest. Virtual environments were designed to vary in a range of properties such as their layout, number of goals, visibility (varying fog) and map condition. We calculated 58 spatial measures grouped into four families: task-specific metrics, space syntax configurational metrics, space syntax geometric metrics, and general geometric metrics. We used Lasso, a variable selection method, to select the most predictive measures of navigation difficulty. Geometric features such as entropy, area of navigable space, number of rings and closeness centrality of path networks were among the most significant factors determining the navigational difficulty. By contrast a range of other measures did not predict difficulty, including measures of intelligibility. Unsurprisingly, other task-specific features (e.g. number of destinations) and fog also predicted navigation difficulty. These findings have implications for the study of spatial behaviour in ecological settings, as well as predicting human movements in different settings, such as complex buildings and transport networks and may aid the design of more navigable environments.
Collapse
Affiliation(s)
- D Yesiltepe
- School of Architecture, University of Sheffield, Sheffield, UK.
| | - P Fernández Velasco
- Department of Philosophy, Trinity College Dublin, University of Dublin, Dublin, Ireland
| | - A Coutrot
- LIRIS, CNRS, University of Lyon, Lyon, France
| | - A Ozbil Torun
- Department of Architecture and Built Environment, Northumbria University, Newcastle upon Tyne, UK
| | - J M Wiener
- Department of Psychology, Ageing and Dementia Research Centre, Bournemouth University, Poole, UK
| | - C Holscher
- ETH Zürich, Swiss Federal Institute of Technology, Zürich, Switzerland
| | - M Hornberger
- Norwich Medical School, University of East Anglia, Norwich, UK
| | - R Conroy Dalton
- Department of Architecture and Built Environment, Northumbria University, Newcastle upon Tyne, UK.
| | - H J Spiers
- Institute of Behavioural Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK.
| |
Collapse
|
14
|
Chen ZS, Wilson MA. How our understanding of memory replay evolves. J Neurophysiol 2023; 129:552-580. [PMID: 36752404 PMCID: PMC9988534 DOI: 10.1152/jn.00454.2022] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 01/20/2023] [Accepted: 01/20/2023] [Indexed: 02/09/2023] Open
Abstract
Memory reactivations and replay, widely reported in the hippocampus and cortex across species, have been implicated in memory consolidation, planning, and spatial and skill learning. Technological advances in electrophysiology, calcium imaging, and human neuroimaging techniques have enabled neuroscientists to measure large-scale neural activity with increasing spatiotemporal resolution and have provided opportunities for developing robust analytic methods to identify memory replay. In this article, we first review a large body of historically important and representative memory replay studies from the animal and human literature. We then discuss our current understanding of memory replay functions in learning, planning, and memory consolidation and further discuss the progress in computational modeling that has contributed to these improvements. Next, we review past and present analytic methods for replay analyses and discuss their limitations and challenges. Finally, looking ahead, we discuss some promising analytic methods for detecting nonstereotypical, behaviorally nondecodable structures from large-scale neural recordings. We argue that seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be essential for delineating the role of memory replay in a wide range of cognitive and motor functions.
Collapse
Affiliation(s)
- Zhe Sage Chen
- Department of Psychiatry, New York University Grossman School of Medicine, New York, New York, United States
- Department of Neuroscience and Physiology, New York University Grossman School of Medicine, New York, New York, United States
- Neuroscience Institute, New York University Grossman School of Medicine, New York, New York, United States
- Department of Biomedical Engineering, New York University Tandon School of Engineering, Brooklyn, New York, United States
| | - Matthew A Wilson
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
| |
Collapse
|
15
|
Rens N, Lancia GL, Eluchans M, Schwartenbeck P, Cunnington R, Pezzulo G. Evidence for entropy maximisation in human free choice behaviour. Cognition 2023; 232:105328. [PMID: 36463639 DOI: 10.1016/j.cognition.2022.105328] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2021] [Revised: 11/10/2022] [Accepted: 11/12/2022] [Indexed: 12/05/2022]
Abstract
The freedom to choose between options is strongly linked to notions of free will. Accordingly, several studies have shown that individuals demonstrate a preference for choice, or the availability of multiple options, over and above utilitarian value. Yet we lack a decision-making framework that integrates preference for choice with traditional utility maximisation in free choice behaviour. Here we test the predictions of an inference-based model of decision-making in which an agent actively seeks states yielding entropy (availability of options) in addition to utility (economic reward). We designed a study in which participants freely navigated a virtual environment consisting of two consecutive choices leading to reward locations in separate rooms. Critically, the choice of one room always led to two final doors while, in the second room, only one door was permissible to choose. This design allowed us to separately determine the influence of utility and entropy on participants' choice behaviour and their self-evaluation of free will. We found that choice behaviour was better predicted by an inference-based model than by expected utility alone, and that both the availability of options and the value of the context positively influenced participants' perceived freedom of choice. Moreover, this consideration of options was apparent in the ongoing motion dynamics as individuals navigated the environment. In a second study, in which participants selected between rooms that gave access to three or four doors, we observed a similar pattern of results, with participants preferring the room that gave access to more options and feeling freer in it. These results suggest that free choice behaviour is well explained by an inference-based framework in which both utility and entropy are optimised and supports the idea that the feeling of having free will is tightly related to options availability.
Collapse
Affiliation(s)
- Natalie Rens
- Queensland Brain Institute, The University of Queensland, St Lucia, Queensland 4072, Australia
| | - Gian Luca Lancia
- Institute of Cognitive Sciences and Technologies, National Research Council, Via S. Martino della Battaglia, 44, 00185 Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Mattia Eluchans
- Institute of Cognitive Sciences and Technologies, National Research Council, Via S. Martino della Battaglia, 44, 00185 Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Philipp Schwartenbeck
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom; Oxford Centre for Functional MRI of the Brain, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom; University of Tübingen, Tübingen, Germany; Max Planck Institute for Biological Cybernetics, Tübingen, Baden-Württemberg, Germany
| | - Ross Cunnington
- School of Psychology, The University of Queensland, St Lucia, Queensland 4072, Australia
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Via S. Martino della Battaglia, 44, 00185 Rome, Italy.
| |
Collapse
|
16
|
Priorelli M, Stoianov IP. Flexible intentions: An Active Inference theory. Front Comput Neurosci 2023; 17:1128694. [PMID: 37021085 PMCID: PMC10067605 DOI: 10.3389/fncom.2023.1128694] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Accepted: 03/03/2023] [Indexed: 04/07/2023] Open
Abstract
We present a normative computational theory of how the brain may support visually-guided goal-directed actions in dynamically changing environments. It extends the Active Inference theory of cortical processing according to which the brain maintains beliefs over the environmental state, and motor control signals try to fulfill the corresponding sensory predictions. We propose that the neural circuitry in the Posterior Parietal Cortex (PPC) compute flexible intentions-or motor plans from a belief over targets-to dynamically generate goal-directed actions, and we develop a computational formalization of this process. A proof-of-concept agent embodying visual and proprioceptive sensors and an actuated upper limb was tested on target-reaching tasks. The agent behaved correctly under various conditions, including static and dynamic targets, different sensory feedbacks, sensory precisions, intention gains, and movement policies; limit conditions were individuated, too. Active Inference driven by dynamic and flexible intentions can thus support goal-directed behavior in constantly changing environments, and the PPC might putatively host its core intention mechanism. More broadly, the study provides a normative computational basis for research on goal-directed behavior in end-to-end settings and further advances mechanistic theories of active biological systems.
Collapse
|
17
|
Whittington JCR, McCaffary D, Bakermans JJW, Behrens TEJ. How to build a cognitive map. Nat Neurosci 2022; 25:1257-1272. [PMID: 36163284 DOI: 10.1038/s41593-022-01153-y] [Citation(s) in RCA: 45] [Impact Index Per Article: 22.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2021] [Accepted: 07/25/2022] [Indexed: 11/08/2022]
Abstract
Learning and interpreting the structure of the environment is an innate feature of biological systems, and is integral to guiding flexible behaviors for evolutionary viability. The concept of a cognitive map has emerged as one of the leading metaphors for these capacities, and unraveling the learning and neural representation of such a map has become a central focus of neuroscience. In recent years, many models have been developed to explain cellular responses in the hippocampus and other brain areas. Because it can be difficult to see how these models differ, how they relate and what each model can contribute, this Review aims to organize these models into a clear ontology. This ontology reveals parallels between existing empirical results, and implies new approaches to understand hippocampal-cortical interactions and beyond.
Collapse
Affiliation(s)
- James C R Whittington
- Department of Applied Physics, Stanford University, Stanford, CA, USA.
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| | - David McCaffary
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Jacob J W Bakermans
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Timothy E J Behrens
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Sainsbury Wellcome Centre for Neural Circuits and Behaviour, University College London, London, UK
| |
Collapse
|
18
|
Safron A, Çatal O, Verbelen T. Generalized Simultaneous Localization and Mapping (G-SLAM) as unification framework for natural and artificial intelligences: towards reverse engineering the hippocampal/entorhinal system and principles of high-level cognition. Front Syst Neurosci 2022; 16:787659. [PMID: 36246500 PMCID: PMC9563348 DOI: 10.3389/fnsys.2022.787659] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Accepted: 09/02/2022] [Indexed: 11/24/2022] Open
Abstract
Simultaneous localization and mapping (SLAM) represents a fundamental problem for autonomous embodied systems, for which the hippocampal/entorhinal system (H/E-S) has been optimized over the course of evolution. We have developed a biologically-inspired SLAM architecture based on latent variable generative modeling within the Free Energy Principle and Active Inference (FEP-AI) framework, which affords flexible navigation and planning in mobile robots. We have primarily focused on attempting to reverse engineer H/E-S "design" properties, but here we consider ways in which SLAM principles from robotics may help us better understand nervous systems and emergent minds. After reviewing LatentSLAM and notable features of this control architecture, we consider how the H/E-S may realize these functional properties not only for physical navigation, but also with respect to high-level cognition understood as generalized simultaneous localization and mapping (G-SLAM). We focus on loop-closure, graph-relaxation, and node duplication as particularly impactful architectural features, suggesting these computational phenomena may contribute to understanding cognitive insight (as proto-causal-inference), accommodation (as integration into existing schemas), and assimilation (as category formation). All these operations can similarly be describable in terms of structure/category learning on multiple levels of abstraction. However, here we adopt an ecological rationality perspective, framing H/E-S functions as orchestrating SLAM processes within both concrete and abstract hypothesis spaces. In this navigation/search process, adaptive cognitive equilibration between assimilation and accommodation involves balancing tradeoffs between exploration and exploitation; this dynamic equilibrium may be near optimally realized in FEP-AI, wherein control systems governed by expected free energy objective functions naturally balance model simplicity and accuracy. With respect to structure learning, such a balance would involve constructing models and categories that are neither too inclusive nor exclusive. We propose these (generalized) SLAM phenomena may represent some of the most impactful sources of variation in cognition both within and between individuals, suggesting that modulators of H/E-S functioning may potentially illuminate their adaptive significances as fundamental cybernetic control parameters. Finally, we discuss how understanding H/E-S contributions to G-SLAM may provide a unifying framework for high-level cognition and its potential realization in artificial intelligences.
Collapse
Affiliation(s)
- Adam Safron
- Center for Psychedelic and Consciousness Research, Johns Hopkins University School of Medicine, Baltimore, MD, United States
- Cognitive Science Program, Indiana University, Bloomington, IN, United States
- Institute for Advanced Consciousness Studies, Santa Monica, CA, United States
| | - Ozan Çatal
- IDLab, Department of Information Technology, Ghent University—imec, Ghent, Belgium
| | - Tim Verbelen
- IDLab, Department of Information Technology, Ghent University—imec, Ghent, Belgium
| |
Collapse
|