1
|
Li C, Brenner J, Boesky A, Ramanathan S, Kreiman G. Neuron-level Prediction and Noise can Implement Flexible Reward-Seeking Behavior. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.22.595306. [PMID: 38826332 PMCID: PMC11142161 DOI: 10.1101/2024.05.22.595306] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2024]
Abstract
We show that neural networks can implement reward-seeking behavior using only local predictive updates and internal noise. These networks are capable of autonomous interaction with an environment and can switch between explore and exploit behavior, which we show is governed by attractor dynamics. Networks can adapt to changes in their architectures, environments, or motor interfaces without any external control signals. When networks have a choice between different tasks, they can form preferences that depend on patterns of noise and initialization, and we show that these preferences can be biased by network architectures or by changing learning rates. Our algorithm presents a flexible, biologically plausible way of interacting with environments without requiring an explicit environmental reward function, allowing for behavior that is both highly adaptable and autonomous. Code is available at https://github.com/ccli3896/PaN.
Collapse
Affiliation(s)
- Chenguang Li
- Biophysics Program, Harvard College, Cambridge, MA 02138
| | | | | | - Sharad Ramanathan
- Department of Molecular and Cellular Biology, Harvard University Cambridge, MA 02138
| | - Gabriel Kreiman
- Boston Children's Hospital, Harvard Medical School, Boston, MA 02115
| |
Collapse
|
2
|
Pezzulo G, D'Amato L, Mannella F, Priorelli M, Van de Maele T, Stoianov IP, Friston K. Neural representation in active inference: Using generative models to interact with-and understand-the lived world. Ann N Y Acad Sci 2024; 1534:45-68. [PMID: 38528782 DOI: 10.1111/nyas.15118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/27/2024]
Abstract
This paper considers neural representation through the lens of active inference, a normative framework for understanding brain function. It delves into how living organisms employ generative models to minimize the discrepancy between predictions and observations (as scored with variational free energy). The ensuing analysis suggests that the brain learns generative models to navigate the world adaptively, not (or not solely) to understand it. Different living organisms may possess an array of generative models, spanning from those that support action-perception cycles to those that underwrite planning and imagination; namely, from explicit models that entail variables for predicting concurrent sensations, like objects, faces, or people-to action-oriented models that predict action outcomes. It then elucidates how generative models and belief dynamics might link to neural representation and the implications of different types of generative models for understanding an agent's cognitive capabilities in relation to its ecological niche. The paper concludes with open questions regarding the evolution of generative models and the development of advanced cognitive abilities-and the gradual transition from pragmatic to detached neural representations. The analysis on offer foregrounds the diverse roles that generative models play in cognitive processes and the evolution of neural representation.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Leo D'Amato
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
- Polytechnic University of Turin, Turin, Italy
| | - Francesco Mannella
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Matteo Priorelli
- Institute of Cognitive Sciences and Technologies, National Research Council, Padua, Italy
| | - Toon Van de Maele
- IDLab, Department of Information Technology, Ghent University - imec, Ghent, Belgium
| | - Ivilin Peev Stoianov
- Institute of Cognitive Sciences and Technologies, National Research Council, Padua, Italy
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London, UK
- VERSES Research Lab, Los Angeles, California, USA
| |
Collapse
|
3
|
Pezzulo G, Parr T, Cisek P, Clark A, Friston K. Generating meaning: active inference and the scope and limits of passive AI. Trends Cogn Sci 2024; 28:97-112. [PMID: 37973519 DOI: 10.1016/j.tics.2023.10.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Revised: 10/03/2023] [Accepted: 10/05/2023] [Indexed: 11/19/2023]
Abstract
Prominent accounts of sentient behavior depict brains as generative models of organismic interaction with the world, evincing intriguing similarities with current advances in generative artificial intelligence (AI). However, because they contend with the control of purposive, life-sustaining sensorimotor interactions, the generative models of living organisms are inextricably anchored to the body and world. Unlike the passive models learned by generative AI systems, they must capture and control the sensory consequences of action. This allows embodied agents to intervene upon their worlds in ways that constantly put their best models to the test, thus providing a solid bedrock that is - we argue - essential to the development of genuine understanding. We review the resulting implications and consider future directions for generative AI.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| | - Thomas Parr
- Nuffield Department of Clinical Neurosciences, University of Oxford
| | - Paul Cisek
- Department of Neuroscience, University of Montréal, Montréal, Québec, Canada
| | - Andy Clark
- Department of Philosophy, University of Sussex, Brighton, UK; Department of Informatics, University of Sussex, Brighton, UK; Department of Philosophy, Macquarie University, Sydney, New South Wales, Australia
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London, UK; VERSES AI Research Lab, Los Angeles, CA, USA
| |
Collapse
|
4
|
Seymour B, Crook RJ, Chen ZS. Post-injury pain and behaviour: a control theory perspective. Nat Rev Neurosci 2023; 24:378-392. [PMID: 37165018 PMCID: PMC10465160 DOI: 10.1038/s41583-023-00699-5] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/28/2023] [Indexed: 05/12/2023]
Abstract
Injuries of various types occur commonly in the lives of humans and other animals and lead to a pattern of persistent pain and recuperative behaviour that allows safe and effective recovery. In this Perspective, we propose a control-theoretic framework to explain the adaptive processes in the brain that drive physiological post-injury behaviour. We set out an evolutionary and ethological view on how animals respond to injury, illustrating how the behavioural state associated with persistent pain and recuperation may be just as important as phasic pain in ensuring survival. Adopting a normative approach, we suggest that the brain implements a continuous optimal inference of the current state of injury from diverse sensory and physiological signals. This drives the various effector control mechanisms of behavioural homeostasis, which span the modulation of ongoing motivation and perception to drive rest and hyper-protective behaviours. However, an inherent problem with this is that these protective behaviours may partially obscure information about whether injury has resolved. Such information restriction may seed a tendency to aberrantly or persistently infer injury, and may thus promote the transition to pathological chronic pain states.
Collapse
Affiliation(s)
- Ben Seymour
- Institute for Biomedical Engineering, University of Oxford, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging, John Radcliffe Hospital, Headington, Oxford, UK.
| | - Robyn J Crook
- Department of Biology, San Francisco State University, San Francisco, CA, USA.
| | - Zhe Sage Chen
- Departments of Psychiatry, Neuroscience and Physiology, Neuroscience Institute, New York University Grossman School of Medicine, New York, NY, USA.
- Department of Biomedical Engineering, New York University Tandon School of Engineering, Brooklyn, NY, USA.
- Interdisciplinary Pain Research Program, NYU Langone Health, New York, NY, USA.
| |
Collapse
|
5
|
Yang Z, Diaz GJ, Fajen BR, Bailey R, Ororbia AG. A neural active inference model of perceptual-motor learning. Front Comput Neurosci 2023; 17:1099593. [PMID: 36890967 PMCID: PMC9986490 DOI: 10.3389/fncom.2023.1099593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Accepted: 01/30/2023] [Indexed: 02/22/2023] Open
Abstract
The active inference framework (AIF) is a promising new computational framework grounded in contemporary neuroscience that can produce human-like behavior through reward-based learning. In this study, we test the ability for the AIF to capture the role of anticipation in the visual guidance of action in humans through the systematic investigation of a visual-motor task that has been well-explored-that of intercepting a target moving over a ground plane. Previous research demonstrated that humans performing this task resorted to anticipatory changes in speed intended to compensate for semi-predictable changes in target speed later in the approach. To capture this behavior, our proposed "neural" AIF agent uses artificial neural networks to select actions on the basis of a very short term prediction of the information about the task environment that these actions would reveal along with a long-term estimate of the resulting cumulative expected free energy. Systematic variation revealed that anticipatory behavior emerged only when required by limitations on the agent's movement capabilities, and only when the agent was able to estimate accumulated free energy over sufficiently long durations into the future. In addition, we present a novel formulation of the prior mapping function that maps a multi-dimensional world-state to a uni-dimensional distribution of free-energy/reward. Together, these results demonstrate the use of AIF as a plausible model of anticipatory visually guided behavior in humans.
Collapse
Affiliation(s)
- Zhizhuo Yang
- Golisano College of Computing and Information Sciences, Rochester Institute of Technology, Rochester, NY, United States
| | - Gabriel J Diaz
- Chester F. Carlson Center for Imaging Science, Rochester Institute of Technology, Rochester, NY, United States
| | - Brett R Fajen
- Department of Cognitive Science, Rensselaer Polytechnic Institute, Troy, NY, United States
| | - Reynold Bailey
- Golisano College of Computing and Information Sciences, Rochester Institute of Technology, Rochester, NY, United States
| | - Alexander G Ororbia
- Golisano College of Computing and Information Sciences, Rochester Institute of Technology, Rochester, NY, United States
| |
Collapse
|
6
|
Miller M, Albarracin M, Pitliya RJ, Kiefer A, Mago J, Gorman C, Friston KJ, Ramstead MJD. Resilience and active inference. Front Psychol 2022; 13:1059117. [PMID: 36619023 PMCID: PMC9815108 DOI: 10.3389/fpsyg.2022.1059117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Accepted: 11/28/2022] [Indexed: 12/24/2022] Open
Abstract
In this article, we aim to conceptualize and formalize the construct of resilience using the tools of active inference, a new physics-based modeling approach apt for the description and analysis of complex adaptive systems. We intend this as a first step toward a computational model of resilient systems. We begin by offering a conceptual analysis of resilience, to clarify its meaning, as established in the literature. We examine an orthogonal, threefold distinction between meanings of the word "resilience": (i) inertia, or the ability to resist change (ii) elasticity, or the ability to bounce back from a perturbation, and (iii) plasticity, or the ability to flexibly expand the repertoire of adaptive states. We then situate all three senses of resilience within active inference. We map resilience as inertia onto high precision beliefs, resilience as elasticity onto relaxation back to characteristic (i.e., attracting) states, and resilience as plasticity onto functional redundancy and structural degeneracy.
Collapse
Affiliation(s)
- Mark Miller
- Center for Consciousness and Contemplative Studies, Monash University, Melbourne, VIC, Australia
| | - Mahault Albarracin
- VERSES Research Lab, Los Angeles, CA, United States
- Department of Computing, Université du Québec à Montréal, Montreal, QC, Canada
| | - Riddhi J. Pitliya
- VERSES Research Lab, Los Angeles, CA, United States
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Alex Kiefer
- VERSES Research Lab, Los Angeles, CA, United States
- Department of Philosophy, Monash University, Melbourne, VIC, Australia
| | - Jonas Mago
- Integrated Program in Neuroscience, Department of Neuroscience, McGill University, Montreal, QC, Canada
- Division of Social and Transcultural Psychiatry, McGill University, Montreal, QC, Canada
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
| | - Claire Gorman
- MIT Senseable City Lab, Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Karl J. Friston
- VERSES Research Lab, Los Angeles, CA, United States
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
| | - Maxwell J. D. Ramstead
- VERSES Research Lab, Los Angeles, CA, United States
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
| |
Collapse
|
7
|
Structure learning enhances concept formation in synthetic Active Inference agents. PLoS One 2022; 17:e0277199. [PMID: 36374909 PMCID: PMC9662737 DOI: 10.1371/journal.pone.0277199] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2021] [Accepted: 10/24/2022] [Indexed: 11/16/2022] Open
Abstract
Humans display astonishing skill in learning about the environment in which they operate. They assimilate a rich set of affordances and interrelations among different elements in particular contexts, and form flexible abstractions (i.e., concepts) that can be generalised and leveraged with ease. To capture these abilities, we present a deep hierarchical Active Inference model of goal-directed behaviour, and the accompanying belief update schemes implied by maximising model evidence. Using simulations, we elucidate the potential mechanisms that underlie and influence concept learning in a spatial foraging task. We show that the representations formed–as a result of foraging–reflect environmental structure in a way that is enhanced and nuanced by Bayesian model reduction, a special case of structure learning that typifies learning in the absence of new evidence. Synthetic agents learn associations and form concepts about environmental context and configuration as a result of inferential, parametric learning, and structure learning processes–three processes that can produce a diversity of beliefs and belief structures. Furthermore, the ensuing representations reflect symmetries for environments with identical configurations.
Collapse
|
8
|
Buddy System: An Adaptive Mental State Support System Based on Active Inference and Free-Energy Principles. IEEE Trans Cogn Dev Syst 2022. [DOI: 10.1109/tcds.2021.3102993] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
9
|
Kiverstein J, Kirchhoff MD, Froese T. The Problem of Meaning: The Free Energy Principle and Artificial Agency. Front Neurorobot 2022; 16:844773. [PMID: 35812784 PMCID: PMC9260223 DOI: 10.3389/fnbot.2022.844773] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2021] [Accepted: 05/17/2022] [Indexed: 12/13/2022] Open
Abstract
Biological agents can act in ways that express a sensitivity to context-dependent relevance. So far it has proven difficult to engineer this capacity for context-dependent sensitivity to relevance in artificial agents. We give this problem the label the “problem of meaning”. The problem of meaning could be circumvented if artificial intelligence researchers were to design agents based on the assumption of the continuity of life and mind. In this paper, we focus on the proposal made by enactive cognitive scientists to design artificial agents that possess sensorimotor autonomy—stable, self-sustaining patterns of sensorimotor interaction that can ground values, norms and goals necessary for encountering a meaningful environment. More specifically, we consider whether the Free Energy Principle (FEP) can provide formal tools for modeling sensorimotor autonomy. There is currently no consensus on how to understand the relationship between enactive cognitive science and the FEP. However, a number of recent papers have argued that the two frameworks are fundamentally incompatible. Some argue that biological systems exhibit historical path-dependent learning that is absent from systems that minimize free energy. Others have argued that a free energy minimizing system would fail to satisfy a key condition for sensorimotor agency referred to as “interactional asymmetry”. These critics question the claim we defend in this paper that the FEP can be used to formally model autonomy and adaptivity. We will argue it is too soon to conclude that the two frameworks are incompatible. There are undeniable conceptual differences between the two frameworks but in our view each has something important and necessary to offer. The FEP needs enactive cognitive science for the solution it provides to the problem of meaning. Enactive cognitive science needs the FEP to formally model the properties it argues to be constitutive of agency. Our conclusion will be that active inference models based on the FEP provides a way by which scientists can think about how to address the problems of engineering autonomy and adaptivity in artificial agents in formal terms. In the end engaging more closely with this formalism and its further developments will benefit those working within the enactive framework.
Collapse
Affiliation(s)
- Julian Kiverstein
- Academic Medical Center, Amsterdam, Netherlands
- Amsterdam Brain and Cognition, University of Amsterdam, Amsterdam, Netherlands
- *Correspondence: Julian Kiverstein
| | - Michael D. Kirchhoff
- Faculty of Arts, Social Sciences, and Humanities, School of Liberal Arts, University of Wollongong, Wollongong, NSW, Australia
| | - Tom Froese
- Embodied Cognitive Science Unit, Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan
| |
Collapse
|
10
|
Hauke G, Lohr C. Piloting the Update: The Use of Therapeutic Relationship for Change - A Free Energy Account. Front Psychol 2022; 13:842488. [PMID: 35478746 PMCID: PMC9036100 DOI: 10.3389/fpsyg.2022.842488] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2021] [Accepted: 03/02/2022] [Indexed: 11/13/2022] Open
Abstract
We apply the Free Energy Principle (FEP) to cognitive behavioral therapy (CBT). FEP describes the basic functioning of the brain as a predictive organ and states that any self-organizing system that is in equilibrium with its environment must minimize its free energy. Based on an internal model of the world and the self, predictions-so-called priors-are created, which are matched with the information input. The sum of prediction errors corresponds to the Free Energy, which must be minimized. Internal models can be identified with the cognitive-affective schemas of the individual that has become dysfunctional in patients. The role of CBT in this picture is to help the patient update her/his priors. They have evolved in learning history and no longer provide adaptive predictions. We discuss the process of updating in terms of the exploration-exploitation dilemma. This consists of the extent to which one relies on what one already has, i.e., whether one continues to maintain and "exploit" one's previous priors ("better safe than sorry") or whether one does explore new data that lead to an update of priors. Questioning previous priors triggers stress, which is associated with increases in Free Energy in short term. The role of therapeutic relationship is to buffer this increase in Free Energy, thereby increasing the level of perceived safety. The therapeutic relationship is represented in a dual model of affective alliance and goal attainment alliance and is aligned with FEP. Both forms of alliance support exploration and updating of priors. All aspects are illustrated with the help of a clinical case example.
Collapse
|
11
|
Smith R, Friston KJ, Whyte CJ. A step-by-step tutorial on active inference and its application to empirical data. JOURNAL OF MATHEMATICAL PSYCHOLOGY 2022; 107:102632. [PMID: 35340847 PMCID: PMC8956124 DOI: 10.1016/j.jmp.2021.102632] [Citation(s) in RCA: 37] [Impact Index Per Article: 18.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
The active inference framework, and in particular its recent formulation as a partially observable Markov decision process (POMDP), has gained increasing popularity in recent years as a useful approach for modeling neurocognitive processes. This framework is highly general and flexible in its ability to be customized to model any cognitive process, as well as simulate predicted neuronal responses based on its accompanying neural process theory. It also affords both simulation experiments for proof of principle and behavioral modeling for empirical studies. However, there are limited resources that explain how to build and run these models in practice, which limits their widespread use. Most introductions assume a technical background in programming, mathematics, and machine learning. In this paper we offer a step-by-step tutorial on how to build POMDPs, run simulations using standard MATLAB routines, and fit these models to empirical data. We assume a minimal background in programming and mathematics, thoroughly explain all equations, and provide exemplar scripts that can be customized for both theoretical and empirical studies. Our goal is to provide the reader with the requisite background knowledge and practical tools to apply active inference to their own research. We also provide optional technical sections and multiple appendices, which offer the interested reader additional technical details. This tutorial should provide the reader with all the tools necessary to use these models and to follow emerging advances in active inference research.
Collapse
Affiliation(s)
- Ryan Smith
- Laureate Institute for Brain Research, Tulsa, OK, USA
| | - Karl J. Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, WC1N 3AR, UK
| | | |
Collapse
|
12
|
Albarracin M, Demekas D, Ramstead MJD, Heins C. Epistemic Communities under Active Inference. ENTROPY 2022; 24:e24040476. [PMID: 35455140 PMCID: PMC9027706 DOI: 10.3390/e24040476] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Revised: 03/11/2022] [Accepted: 03/24/2022] [Indexed: 02/04/2023]
Abstract
The spread of ideas is a fundamental concern of today’s news ecology. Understanding the dynamics of the spread of information and its co-option by interested parties is of critical importance. Research on this topic has shown that individuals tend to cluster in echo-chambers and are driven by confirmation bias. In this paper, we leverage the active inference framework to provide an in silico model of confirmation bias and its effect on echo-chamber formation. We build a model based on active inference, where agents tend to sample information in order to justify their own view of reality, which eventually leads to them to have a high degree of certainty about their own beliefs. We show that, once agents have reached a certain level of certainty about their beliefs, it becomes very difficult to get them to change their views. This system of self-confirming beliefs is upheld and reinforced by the evolving relationship between an agent’s beliefs and observations, which over time will continue to provide evidence for their ingrained ideas about the world. The epistemic communities that are consolidated by these shared beliefs, in turn, tend to produce perceptions of reality that reinforce those shared beliefs. We provide an active inference account of this community formation mechanism. We postulate that agents are driven by the epistemic value that they obtain from sampling or observing the behaviours of other agents. Inspired by digital social networks like Twitter, we build a generative model in which agents generate observable social claims or posts (e.g., ‘tweets’) while reading the socially observable claims of other agents that lend support to one of two mutually exclusive abstract topics. Agents can choose which other agent they pay attention to at each timestep, and crucially who they attend to and what they choose to read influences their beliefs about the world. Agents also assess their local network’s perspective, influencing which kinds of posts they expect to see other agents making. The model was built and simulated using the freely available Python package pymdp. The proposed active inference model can reproduce the formation of echo-chambers over social networks, and gives us insight into the cognitive processes that lead to this phenomenon.
Collapse
Affiliation(s)
- Mahault Albarracin
- Department of Cognitive Computing, Université du Québec a Montreal, Montreal, QC H2K 4M1, Canada;
- VERSES Labs, Los Angeles, CA 90016, USA;
| | - Daphne Demekas
- Department of Computing, Imperial College London, London SW7 5NH, UK;
| | - Maxwell J. D. Ramstead
- VERSES Labs, Los Angeles, CA 90016, USA;
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3AR, UK
| | - Conor Heins
- VERSES Labs, Los Angeles, CA 90016, USA;
- Department of Collective Behaviour, Max Planck Institute of Animal Behaviour, 78315 Radolfzell am Bodensee, Germany
- Centre for the Advanced Study of Collective Behaviour, University of Konstanz, 78464 Konstanz, Germany
- Department of Biology, University of Konstanz, 78457 Konstanz, Germany
- Correspondence:
| |
Collapse
|
13
|
Ramstead MJD, Seth AK, Hesp C, Sandved-Smith L, Mago J, Lifshitz M, Pagnoni G, Smith R, Dumas G, Lutz A, Friston K, Constant A. From Generative Models to Generative Passages: A Computational Approach to (Neuro) Phenomenology. REVIEW OF PHILOSOPHY AND PSYCHOLOGY 2022; 13:829-857. [PMID: 35317021 PMCID: PMC8932094 DOI: 10.1007/s13164-021-00604-y] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Accepted: 10/28/2021] [Indexed: 12/16/2022]
Abstract
This paper presents a version of neurophenomenology based on generative modelling techniques developed in computational neuroscience and biology. Our approach can be described as computational phenomenology because it applies methods originally developed in computational modelling to provide a formal model of the descriptions of lived experience in the phenomenological tradition of philosophy (e.g., the work of Edmund Husserl, Maurice Merleau-Ponty, etc.). The first section presents a brief review of the overall project to naturalize phenomenology. The second section presents and evaluates philosophical objections to that project and situates our version of computational phenomenology with respect to these projects. The third section reviews the generative modelling framework. The final section presents our approach in detail. We conclude by discussing how our approach differs from previous attempts to use generative modelling to help understand consciousness. In summary, we describe a version of computational phenomenology which uses generative modelling to construct a computational model of the inferential or interpretive processes that best explain this or that kind of lived experience.
Collapse
Affiliation(s)
- Maxwell J. D. Ramstead
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- VERSES Research Lab and Spatial Web Foundation, Los Angeles, California USA
| | - Anil K. Seth
- School of Engineering and Informatics, University of Sussex, Brighton, BN1 9QJ UK
- Canadian Institute for Advanced Research (CIFAR), Program on Brain, Mind, and Consciousness, Toronto, Ontario, M5G 1M1 Canada
| | - Casper Hesp
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Department of Psychology, University of Amsterdam, Science Park 904, 1098 XH Amsterdam, Netherlands
- Amsterdam Brain and Cognition Centre, University of Amsterdam, Science Park 904, 1098 XH Amsterdam, Netherlands
- Institute for Advanced Study, University of Amsterdam, Oude Turfmarkt 147, 1012 GC Amsterdam, Netherlands
| | - Lars Sandved-Smith
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Lyon Neuroscience Research Centre, INSERM U1028, CNRS UMR5292, Lyon 1 University, Lyon, France
| | - Jonas Mago
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Integrated Program in Neuroscience, Department of Neuroscience, McGill University, Montreal, Canada
- Division of Social and Transcultural Psychiatry, McGill University, Montreal, Canada
| | - Michael Lifshitz
- Division of Social and Transcultural Psychiatry, McGill University, Montreal, Canada
- Lady Davis Institute for Medical Research, Montreal Jewish General Hospital, Montreal, Canada
| | - Giuseppe Pagnoni
- Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, Modena, Italy
- Center for Neuroscience and Neurotechnology, University of Modena and Reggio Emilia, Modena, Italy
| | - Ryan Smith
- Laureate Institute for Brain Research, Tulsa, Oklahoma USA
| | - Guillaume Dumas
- CHU Sainte-Justine Research Center, Department of Psychiatry, University of Montreal, Montreal, Canada
- Mila – Quebec Artificial Intelligence Institute, University of Montreal, Montreal, Canada
| | - Antoine Lutz
- Lyon Neuroscience Research Centre, INSERM U1028, CNRS UMR5292, Lyon 1 University, Lyon, France
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- VERSES Research Lab and Spatial Web Foundation, Los Angeles, California USA
| | - Axel Constant
- Charles Perkins Centre, The University of Sydney, Sydney, Australia
| |
Collapse
|
14
|
Pezzulo G, Parr T, Friston K. The evolution of brain architectures for predictive coding and active inference. Philos Trans R Soc Lond B Biol Sci 2022; 377:20200531. [PMID: 34957844 PMCID: PMC8710884 DOI: 10.1098/rstb.2020.0531] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Accepted: 09/08/2021] [Indexed: 01/13/2023] Open
Abstract
This article considers the evolution of brain architectures for predictive processing. We argue that brain mechanisms for predictive perception and action are not late evolutionary additions of advanced creatures like us. Rather, they emerged gradually from simpler predictive loops (e.g. autonomic and motor reflexes) that were a legacy from our earlier evolutionary ancestors-and were key to solving their fundamental problems of adaptive regulation. We characterize simpler-to-more-complex brains formally, in terms of generative models that include predictive loops of increasing hierarchical breadth and depth. These may start from a simple homeostatic motif and be elaborated during evolution in four main ways: these include the multimodal expansion of predictive control into an allostatic loop; its duplication to form multiple sensorimotor loops that expand an animal's behavioural repertoire; and the gradual endowment of generative models with hierarchical depth (to deal with aspects of the world that unfold at different spatial scales) and temporal depth (to select plans in a future-oriented manner). In turn, these elaborations underwrite the solution to biological regulation problems faced by increasingly sophisticated animals. Our proposal aligns neuroscientific theorising-about predictive processing-with evolutionary and comparative data on brain architectures in different animal species. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Via S. Martino della Battaglia, 44, 00185 Rome, Italy
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK
| |
Collapse
|
15
|
Tschantz A, Barca L, Maisto D, Buckley CL, Seth AK, Pezzulo G. Simulating homeostatic, allostatic and goal-directed forms of interoceptive control using active inference. Biol Psychol 2022; 169:108266. [DOI: 10.1016/j.biopsycho.2022.108266] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2021] [Revised: 01/06/2022] [Accepted: 01/14/2022] [Indexed: 12/28/2022]
|
16
|
Abstract
We offer an account of mental health and well-being using the predictive processing framework (PPF). According to this framework, the difference between mental health and psychopathology can be located in the goodness of the predictive model as a regulator of action. What is crucial for avoiding the rigid patterns of thinking, feeling and acting associated with psychopathology is the regulation of action based on the valence of affective states. In PPF, valence is modelled as error dynamics—the change in prediction errors over time . Our aim in this paper is to show how error dynamics can account for both momentary happiness and longer term well-being. What will emerge is a new neurocomputational framework for making sense of human flourishing.
Collapse
Affiliation(s)
- Mark Miller
- Center for Consciousness and Contemplative StudiesMonash University, Melbourne, Australia
| | - Erik Rietveld
- ILLC/Department of Philosophy, University of Amsterdam, Amsterdamhe Netherlands Department of PhilosophyUniversity of Twente, Enschede, the Netherlands
| | - Julian Kiverstein
- ILLC/Department of Philosophy, University of Amsterdam, Amsterdamhe Netherlands Department of PhilosophyUniversity of Twente, Enschede, the Netherlands
| |
Collapse
|
17
|
Koudahl MT, Kouw WM, de Vries B. On Epistemics in Expected Free Energy for Linear Gaussian State Space Models. ENTROPY 2021; 23:e23121565. [PMID: 34945871 PMCID: PMC8700494 DOI: 10.3390/e23121565] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Revised: 11/19/2021] [Accepted: 11/23/2021] [Indexed: 01/20/2023]
Abstract
Active Inference (AIF) is a framework that can be used both to describe information processing in naturally intelligent systems, such as the human brain, and to design synthetic intelligent systems (agents). In this paper we show that Expected Free Energy (EFE) minimisation, a core feature of the framework, does not lead to purposeful explorative behaviour in linear Gaussian dynamical systems. We provide a simple proof that, due to the specific construction used for the EFE, the terms responsible for the exploratory (epistemic) drive become constant in the case of linear Gaussian systems. This renders AIF equivalent to KL control. From a theoretical point of view this is an interesting result since it is generally assumed that EFE minimisation will always introduce an exploratory drive in AIF agents. While the full EFE objective does not lead to exploration in linear Gaussian dynamical systems, the principles of its construction can still be used to design objectives that include an epistemic drive. We provide an in-depth analysis of the mechanics behind the epistemic drive of AIF agents and show how to design objectives for linear Gaussian dynamical systems that do include an epistemic drive. Concretely, we show that focusing solely on epistemics and dispensing with goal-directed terms leads to a form of maximum entropy exploration that is heavily dependent on the type of control signals driving the system. Additive controls do not permit such exploration. From a practical point of view this is an important result since linear Gaussian dynamical systems with additive controls are an extensively used model class, encompassing for instance Linear Quadratic Gaussian controllers. On the other hand, linear Gaussian dynamical systems driven by multiplicative controls such as switching transition matrices do permit an exploratory drive.
Collapse
Affiliation(s)
- Magnus T. Koudahl
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands; (W.M.K.); (B.d.V.)
- Correspondence:
| | - Wouter M. Kouw
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands; (W.M.K.); (B.d.V.)
| | - Bert de Vries
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands; (W.M.K.); (B.d.V.)
- GN Hearing, JF Kennedylaan 2, 5612 AB Eindhoven, The Netherlands
| |
Collapse
|
18
|
Lanillos P, Franklin S, Maselli A, Franklin DW. Active strategies for multisensory conflict suppression in the virtual hand illusion. Sci Rep 2021; 11:22844. [PMID: 34819563 PMCID: PMC8613215 DOI: 10.1038/s41598-021-02200-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2021] [Accepted: 11/08/2021] [Indexed: 11/09/2022] Open
Abstract
The perception of our body in space is flexible and manipulable. The predictive brain hypothesis explains this malleability as a consequence of the interplay between incoming sensory information and our body expectations. However, given the interaction between perception and action, we might also expect that actions would arise due to prediction errors, especially in conflicting situations. Here we describe a computational model, based on the free-energy principle, that forecasts involuntary movements in sensorimotor conflicts. We experimentally confirm those predictions in humans using a virtual reality rubber-hand illusion. Participants generated movements (forces) towards the virtual hand, regardless of its location with respect to the real arm, with little to no forces produced when the virtual hand overlaid their physical hand. The congruency of our model predictions and human observations indicates that the brain-body is generating actions to reduce the prediction error between the expected arm location and the new visual arm. This observed unconscious mechanism is an empirical validation of the perception-action duality in body adaptation to uncertain situations and evidence of the active component of predictive processing.
Collapse
Affiliation(s)
- Pablo Lanillos
- Donders Institute for Brain, Cognition and Behaviour, Artificial Intelligence Department, Radboud University, Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands.
| | - Sae Franklin
- Institute for Cognitive Systems, Technical University of Munich, Munich, Germany
| | - Antonella Maselli
- Institute of Cognitive Sciences and Technology, National Research Council (CNR), Rome, Italy
| | - David W Franklin
- Neuromuscular Diagnostics, Department of Sport and Health Sciences, Technical University of Munich, Munich, Germany.,Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich, Munich, Germany
| |
Collapse
|
19
|
How particular is the physics of the free energy principle? Phys Life Rev 2021; 40:24-50. [PMID: 34895862 PMCID: PMC8902446 DOI: 10.1016/j.plrev.2021.11.001] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2021] [Accepted: 11/01/2021] [Indexed: 12/15/2022]
Abstract
The free energy principle (FEP) states that any dynamical system can be interpreted as performing Bayesian inference upon its surrounding environment. Although, in theory, the FEP applies to a wide variety of systems, there has been almost no direct exploration or demonstration of the principle in concrete systems. In this work, we examine in depth the assumptions required to derive the FEP in the simplest possible set of systems – weakly-coupled non-equilibrium linear stochastic systems. Specifically, we explore (i) how general the requirements imposed on the statistical structure of a system are and (ii) how informative the FEP is about the behaviour of such systems. We discover that two requirements of the FEP – the Markov blanket condition (i.e. a statistical boundary precluding direct coupling between internal and external states) and stringent restrictions on its solenoidal flows (i.e. tendencies driving a system out of equilibrium) – are only valid for a very narrow space of parameters. Suitable systems require an absence of perception-action asymmetries that is highly unusual for living systems interacting with an environment. More importantly, we observe that a mathematically central step in the argument, connecting the behaviour of a system to variational inference, relies on an implicit equivalence between the dynamics of the average states of a system with the average of the dynamics of those states. This equivalence does not hold in general even for linear stochastic systems, since it requires an effective decoupling from the system's history of interactions. These observations are critical for evaluating the generality and applicability of the FEP and indicate the existence of significant problems of the theory in its current form. These issues make the FEP, as it stands, not straightforwardly applicable to the simple linear systems studied here and suggest that more development is needed before the theory could be applied to the kind of complex systems that describe living and cognitive processes. We review the free energy principle (FEP) in a family of analytically tractable non-equilibrium linear systems. We study how general and how informative the FEP is in this class of systems. We find that crucial assumptions of the FEP restrict its applicability to a very narrow space of the studied linear systems. Secondly, a core step relies on an equivalence between the dynamics of the average and the average of the dynamics. As a result, the FEP does not result in a good description of the (history-dependent) behaviour of a system.
Collapse
|
20
|
Parr T, Pezzulo G. Understanding, Explanation, and Active Inference. Front Syst Neurosci 2021; 15:772641. [PMID: 34803619 PMCID: PMC8602880 DOI: 10.3389/fnsys.2021.772641] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2021] [Accepted: 10/15/2021] [Indexed: 11/13/2022] Open
Abstract
While machine learning techniques have been transformative in solving a range of problems, an important challenge is to understand why they arrive at the decisions they output. Some have argued that this necessitates augmenting machine intelligence with understanding such that, when queried, a machine is able to explain its behaviour (i.e., explainable AI). In this article, we address the issue of machine understanding from the perspective of active inference. This paradigm enables decision making based upon a model of how data are generated. The generative model contains those variables required to explain sensory data, and its inversion may be seen as an attempt to explain the causes of these data. Here we are interested in explanations of one's own actions. This implies a deep generative model that includes a model of the world, used to infer policies, and a higher-level model that attempts to predict which policies will be selected based upon a space of hypothetical (i.e., counterfactual) explanations-and which can subsequently be used to provide (retrospective) explanations about the policies pursued. We illustrate the construct validity of this notion of understanding in relation to human understanding by highlighting the similarities in computational architecture and the consequences of its dysfunction.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London, United Kingdom
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| |
Collapse
|
21
|
Mannella F, Maggiore F, Baltieri M, Pezzulo G. Active inference through whiskers. Neural Netw 2021; 144:428-437. [PMID: 34563752 DOI: 10.1016/j.neunet.2021.08.037] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2021] [Revised: 08/29/2021] [Accepted: 08/31/2021] [Indexed: 10/20/2022]
Abstract
Rodents use whisking to probe actively their environment and to locate objects in space, hence providing a paradigmatic biological example of active sensing. Numerous studies show that the control of whisking has anticipatory aspects. For example, rodents target their whisker protraction to the distance at which they expect objects, rather than just reacting fast to contacts with unexpected objects. Here we characterize the anticipatory control of whisking in rodents as an active inference process. In this perspective, the rodent is endowed with a prior belief that it will touch something at the end of the whisker protraction, and it continuously modulates its whisking amplitude to minimize (proprioceptive and somatosensory) prediction errors arising from an unexpected whisker-object contact, or from a lack of an expected contact. We will use the model to qualitatively reproduce key empirical findings about the ways rodents modulate their whisker amplitude during exploration and the scanning of (expected or unexpected) objects. Furthermore, we will discuss how the components of active inference model can in principle map to the neurobiological circuits of rodent whisking.
Collapse
Affiliation(s)
- Francesco Mannella
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| | - Federico Maggiore
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| | - Manuel Baltieri
- Laboratory for Neural Computation and Adaptation, RIKEN Centre for Brain Science, Wako-shi, Japan.
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| |
Collapse
|
22
|
Deane G. Consciousness in active inference: Deep self-models, other minds, and the challenge of psychedelic-induced ego-dissolution. Neurosci Conscious 2021; 2021:niab024. [PMID: 34484808 PMCID: PMC8408766 DOI: 10.1093/nc/niab024] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Revised: 07/26/2021] [Accepted: 08/02/2021] [Indexed: 11/16/2022] Open
Abstract
Predictive processing approaches to brain function are increasingly delivering promise for illuminating the computational underpinnings of a wide range of phenomenological states. It remains unclear, however, whether predictive processing is equipped to accommodate a theory of consciousness itself. Furthermore, objectors have argued that without specification of the core computational mechanisms of consciousness, predictive processing is unable to inform the attribution of consciousness to other non-human (biological and artificial) systems. In this paper, I argue that an account of consciousness in the predictive brain is within reach via recent accounts of phenomenal self-modelling in the active inference framework. The central claim here is that phenomenal consciousness is underpinned by 'subjective valuation'-a deep inference about the precision or 'predictability' of the self-evidencing ('fitness-promoting') outcomes of action. Based on this account, I argue that this approach can critically inform the distribution of experience in other systems, paying particular attention to the complex sensory attenuation mechanisms associated with deep self-models. I then consider an objection to the account: several recent papers argue that theories of consciousness that invoke self-consciousness as constitutive or necessary for consciousness are undermined by states (or traits) of 'selflessness'; in particular the 'totally selfless' states of ego-dissolution occasioned by psychedelic drugs. Drawing on existing work that accounts for psychedelic-induced ego-dissolution in the active inference framework, I argue that these states do not threaten to undermine an active inference theory of consciousness. Instead, these accounts corroborate the view that subjective valuation is the constitutive facet of experience, and they highlight the potential of psychedelic research to inform consciousness science, computational psychiatry and computational phenomenology.
Collapse
Affiliation(s)
- George Deane
- School of Philosophy, Psychology and Language Sciences, The University of Edinburgh, 3 Charles Street, Edinburgh EH8 9AD, UK
| |
Collapse
|
23
|
Martin JM, Solms M, Sterzer P. Useful misrepresentation: perception as embodied proactive inference. Trends Neurosci 2021; 44:619-628. [PMID: 33994015 DOI: 10.1016/j.tins.2021.04.007] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2020] [Revised: 03/01/2021] [Accepted: 04/21/2021] [Indexed: 12/17/2022]
Abstract
According to the predictive processing framework, perception is geared to represent the environment in terms of embodied action opportunities as opposed to objective truth. Here, we argue that such an optimisation is reflected by biases in expectations (i.e., prior predictive information) that facilitate 'useful' inferences of external sensory causes. To support this, we highlight a body of literature suggesting that perception is systematically biased away from accurate estimates under conditions where utility and accuracy conflict with one another. We interpret this to reflect the brain's attempt to adjudicate between conflicting sources of prediction error, as external accuracy is sacrificed to facilitate actions that proactively avoid physiologically surprising outcomes. This carries important theoretical implications and offers new insights into psychopathology.
Collapse
Affiliation(s)
- Joshua M Martin
- Berlin School of Mind and Brain, Faculty of Philosophy, Humboldt-Universität zu Berlin, Berlin, Germany; Department of Psychiatry and Psychotherapy, Charité-Universitätsmedizin Berlin, Campus Charité Mitte, Berlin, Germany
| | - Mark Solms
- Department of Psychology, University of Cape Town, Cape Town, South Africa
| | - Philipp Sterzer
- Department of Psychiatry and Psychotherapy, Charité-Universitätsmedizin Berlin, Campus Charité Mitte, Berlin, Germany.
| |
Collapse
|
24
|
Ciria A, Schillaci G, Pezzulo G, Hafner VV, Lara B. Predictive Processing in Cognitive Robotics: A Review. Neural Comput 2021; 33:1402-1432. [PMID: 34496394 DOI: 10.1162/neco_a_01383] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2020] [Accepted: 12/31/2020] [Indexed: 11/04/2022]
Abstract
Predictive processing has become an influential framework in cognitive sciences. This framework turns the traditional view of perception upside down, claiming that the main flow of information processing is realized in a top-down, hierarchical manner. Furthermore, it aims at unifying perception, cognition, and action as a single inferential process. However, in the related literature, the predictive processing framework and its associated schemes, such as predictive coding, active inference, perceptual inference, and free-energy principle, tend to be used interchangeably. In the field of cognitive robotics, there is no clear-cut distinction on which schemes have been implemented and under which assumptions. In this letter, working definitions are set with the main aim of analyzing the state of the art in cognitive robotics research working under the predictive processing framework as well as some related nonrobotic models. The analysis suggests that, first, research in both cognitive robotics implementations and nonrobotic models needs to be extended to the study of how multiple exteroceptive modalities can be integrated into prediction error minimization schemes. Second, a relevant distinction found here is that cognitive robotics implementations tend to emphasize the learning of a generative model, while in nonrobotics models, it is almost absent. Third, despite the relevance for active inference, few cognitive robotics implementations examine the issues around control and whether it should result from the substitution of inverse models with proprioceptive predictions. Finally, limited attention has been placed on precision weighting and the tracking of prediction error dynamics. These mechanisms should help to explore more complex behaviors and tasks in cognitive robotics research under the predictive processing framework.
Collapse
Affiliation(s)
- Alejandra Ciria
- Facultad de Psicología, Universidad Nacional Autónoma de México, Mexico City, CP 04510, Mexico
| | - Guido Schillaci
- BioRobotics Institute, Scuola Superiore Sant'Anna, 34 56025 Pontedera, Italy
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, 44 00185 Rome, Italy
| | - Verena V Hafner
- Adaptive Systems Group, Department of Computer Science, Humboldt-Universität zu Berlin, D-12489, Germany
| | - Bruno Lara
- Laboratorio de Robótica Cognitiva, Centro de Investigación en Ciencias, Universidad Autónoma del Estado de Morelos, Cuernavaca CP 62209, Mexico
| |
Collapse
|
25
|
Da Costa L, Parr T, Sengupta B, Friston K. Neural Dynamics under Active Inference: Plausibility and Efficiency of Information Processing. ENTROPY (BASEL, SWITZERLAND) 2021; 23:454. [PMID: 33921298 PMCID: PMC8069154 DOI: 10.3390/e23040454] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/16/2021] [Accepted: 04/06/2021] [Indexed: 02/07/2023]
Abstract
Active inference is a normative framework for explaining behaviour under the free energy principle-a theory of self-organisation originating in neuroscience. It specifies neuronal dynamics for state-estimation in terms of a descent on (variational) free energy-a measure of the fit between an internal (generative) model and sensory observations. The free energy gradient is a prediction error-plausibly encoded in the average membrane potentials of neuronal populations. Conversely, the expected probability of a state can be expressed in terms of neuronal firing rates. We show that this is consistent with current models of neuronal dynamics and establish face validity by synthesising plausible electrophysiological responses. We then show that these neuronal dynamics approximate natural gradient descent, a well-known optimisation algorithm from information geometry that follows the steepest descent of the objective in information space. We compare the information length of belief updating in both schemes, a measure of the distance travelled in information space that has a direct interpretation in terms of metabolic cost. We show that neural dynamics under active inference are metabolically efficient and suggest that neural representations in biological agents may evolve by approximating steepest descent in information space towards the point of optimal inference.
Collapse
Affiliation(s)
- Lancelot Da Costa
- Department of Mathematics, Imperial College London, London SW7 2AZ, UK
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK; (T.P.); (B.S.); (K.F.)
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK; (T.P.); (B.S.); (K.F.)
| | - Biswa Sengupta
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK; (T.P.); (B.S.); (K.F.)
- Core Machine Learning Group, Zebra AI, London WC2H 8TJ, UK
- Department of Bioengineering, Imperial College London, London SW7 2AZ, UK
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK; (T.P.); (B.S.); (K.F.)
| |
Collapse
|
26
|
Skora LI, Seth AK, Scott RB. Sensorimotor predictions shape reported conscious visual experience in a breaking continuous flash suppression task. Neurosci Conscious 2021; 2021:niab003. [PMID: 33763234 PMCID: PMC7970722 DOI: 10.1093/nc/niab003] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2020] [Revised: 01/04/2021] [Accepted: 01/05/2021] [Indexed: 11/12/2022] Open
Abstract
Accounts of predictive processing propose that conscious experience is influenced not only by passive predictions about the world, but also by predictions encompassing how the world changes in relation to our actions-that is, on predictions about sensorimotor contingencies. We tested whether valid sensorimotor predictions, in particular learned associations between stimuli and actions, shape reports about conscious visual experience. Two experiments used instrumental conditioning to build sensorimotor predictions linking different stimuli with distinct actions. Conditioning was followed by a breaking continuous flash suppression task, measuring the speed of reported breakthrough for different pairings between the stimuli and prepared actions, comparing those congruent and incongruent with the trained sensorimotor predictions. In Experiment 1, counterbalancing of the response actions within the breaking continuous flash suppression task was achieved by repeating the same action within each block but having them differ across the two blocks. Experiment 2 sought to increase the predictive salience of the actions by avoiding the repetition within blocks. In Experiment 1, breakthrough times were numerically shorter for congruent than incongruent pairings, but Bayesian analysis supported the null hypothesis of no influence from the sensorimotor predictions. In Experiment 2, reported conscious perception was significantly faster for congruent than for incongruent pairings. A meta-analytic Bayes factor combining the two experiments confirmed this effect. Altogether, we provide evidence for a key implication of the action-oriented predictive processing approach to conscious perception, namely that sensorimotor predictions shape our conscious experience of the world.
Collapse
Affiliation(s)
- Lina I Skora
- School of Psychology, University of Sussex, Pevensey Building, Falmer, Brighton BN1 9RH, UK
- Sackler Centre for Consciousness Science, University of Sussex, Falmer, Brighton, Brighton BN1 9RH, UK
| | - Anil K Seth
- Sackler Centre for Consciousness Science, University of Sussex, Falmer, Brighton, Brighton BN1 9RH, UK
- School of Engineering and Informatics, University of Sussex, Falmer, Brighton, Brighton BN1 9RH, UK
- Canadian Institute for Advanced Research, Program on Brain, Mind and Consciousness, 661 University Ave, Toronto, ON M5G 1M1, Canada
| | - Ryan B Scott
- School of Psychology, University of Sussex, Pevensey Building, Falmer, Brighton BN1 9RH, UK
- Sackler Centre for Consciousness Science, University of Sussex, Falmer, Brighton, Brighton BN1 9RH, UK
| |
Collapse
|
27
|
Gao H, Ou Y, Zhang Z, Ni M, Zhou X, Liao L. The Relationship Between Family Support and e-Learning Engagement in College Students: The Mediating Role of e-Learning Normative Consciousness and Behaviors and Self-Efficacy. Front Psychol 2021; 12:573779. [PMID: 33613373 PMCID: PMC7890012 DOI: 10.3389/fpsyg.2021.573779] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2020] [Accepted: 01/06/2021] [Indexed: 01/26/2023] Open
Abstract
Due to the current COVID-19 pandemic, colleges and universities have implemented network teaching. E-learning engagement is the most important concern of educators and parents because this will directly affect student academic performance. Hence, this study focuses on students’ perceived family support and their e-learning engagement and analyzes the effects of e-learning normative consciousness and behaviors and self-efficacy on the relationship between family support and e-learning engagement in college students. Prior to this study, the relationship between these variables was unknown. Four structural equation models revealed the multiple mediating roles of e-learning normative consciousness and behaviors and self-efficacy in the relationship between family support and e-learning engagement. A total of 1,317 college students (mean age=19.51; 52.2% freshman) voluntarily participated in our study. The results showed that e-learning normative consciousness and behaviors and self-efficacy played significant and mediating roles between students’ perceived family support and e-learning engagement. Specifically, these two individual variables fully mediated the relationship between students’ perceived family support and e-learning engagement. The multiple mediation model showed that family members can increase family support of their children by creating a household environment conducive to learning, displaying positive emotions, demonstrating the capability to assist their children, advocating the significance of learning normative consciousness and behaviors, and encouraging dedicated and efficient learning. The findings complement and extend the understanding of factors influencing student e-learning engagement.
Collapse
Affiliation(s)
- Hong Gao
- School of Nursing, University of South China, Hengyang, China
| | - Yangli Ou
- School of Nursing, University of South China, Hengyang, China
| | - Zhiyuan Zhang
- Emergency Department, The Second Hospital University of South China, Hengyang, China
| | - Menghui Ni
- School of Nursing, University of South China, Hengyang, China
| | - Xinlian Zhou
- Emergency Department, The Second Hospital University of South China, Hengyang, China
| | - Li Liao
- School of Nursing, University of South China, Hengyang, China
| |
Collapse
|
28
|
Kim CS. Bayesian mechanics of perceptual inference and motor control in the brain. BIOLOGICAL CYBERNETICS 2021; 115:87-102. [PMID: 33471182 PMCID: PMC7925488 DOI: 10.1007/s00422-021-00859-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/21/2020] [Accepted: 01/06/2021] [Indexed: 06/12/2023]
Abstract
The free energy principle (FEP) in the neurosciences stipulates that all viable agents induce and minimize informational free energy in the brain to fit their environmental niche. In this study, we continue our effort to make the FEP a more physically principled formalism by implementing free energy minimization based on the principle of least action. We build a Bayesian mechanics (BM) by casting the formulation reported in the earlier publication (Kim in Neural Comput 30:2616-2659, 2018, https://doi.org/10.1162/neco_a_01115 ) to considering active inference beyond passive perception. The BM is a neural implementation of variational Bayes under the FEP in continuous time. The resulting BM is provided as an effective Hamilton's equation of motion and subject to the control signal arising from the brain's prediction errors at the proprioceptive level. To demonstrate the utility of our approach, we adopt a simple agent-based model and present a concrete numerical illustration of the brain performing recognition dynamics by integrating BM in neural phase space. Furthermore, we recapitulate the major theoretical architectures in the FEP by comparing our approach with the common state-space formulations.
Collapse
Affiliation(s)
- Chang Sub Kim
- Department of Physics, Chonnam National University, Gwangju, 61186, Republic of Korea.
| |
Collapse
|
29
|
A generative spiking neural-network model of goal-directed behaviour and one-step planning. PLoS Comput Biol 2020; 16:e1007579. [PMID: 33290414 PMCID: PMC7748287 DOI: 10.1371/journal.pcbi.1007579] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2019] [Revised: 12/18/2020] [Accepted: 10/01/2020] [Indexed: 11/21/2022] Open
Abstract
In mammals, goal-directed and planning processes support flexible behaviour used to face new situations that cannot be tackled through more efficient but rigid habitual behaviours. Within the Bayesian modelling approach of brain and behaviour, models have been proposed to perform planning as probabilistic inference but this approach encounters a crucial problem: explaining how such inference might be implemented in brain spiking networks. Recently, the literature has proposed some models that face this problem through recurrent spiking neural networks able to internally simulate state trajectories, the core function at the basis of planning. However, the proposed models have relevant limitations that make them biologically implausible, namely their world model is trained ‘off-line’ before solving the target tasks, and they are trained with supervised learning procedures that are biologically and ecologically not plausible. Here we propose two novel hypotheses on how brain might overcome these problems, and operationalise them in a novel architecture pivoting on a spiking recurrent neural network. The first hypothesis allows the architecture to learn the world model in parallel with its use for planning: to this purpose, a new arbitration mechanism decides when to explore, for learning the world model, or when to exploit it, for planning, based on the entropy of the world model itself. The second hypothesis allows the architecture to use an unsupervised learning process to learn the world model by observing the effects of actions. The architecture is validated by reproducing and accounting for the learning profiles and reaction times of human participants learning to solve a visuomotor learning task that is new for them. Overall, the architecture represents the first instance of a model bridging probabilistic planning and spiking-processes that has a degree of autonomy analogous to the one of real organisms. Goal-directed behaviour relies on brain processes supporting planning of actions based on their expected consequences before performing them in the environment. An important computational modelling approach proposes that the brain performs goal-directed processes on the basis of probability distributions and computations on them. A key challenge of this approach is to explain how these probabilistic processes can rely on the spiking processes of the brain. The literature has recently proposed some models that do so by ‘thinking ahead’ alternative possible action-outcomes based on low-level neuronal stochastic events. However, these models have a limited autonomy as they require to learn how the environment works (‘world model’) before solving the tasks, and use a biologically implausible learning process requiring an ‘external teacher’ to tell how their internal units should respond. Here we present a novel architecture proposing how organisms might overcome these challenging problems. First, the architecture can decide if exploring, to learn the world model, or planning, using such model, by evaluating how confident it is on the model knowledge. Second, the architecture can autonomously learn the world model based on experience. The architecture represents a first fully autonomous planning model relying on a spiking neural network.
Collapse
|
30
|
Leptourgos P, Fortier-Davy M, Carhart-Harris R, Corlett PR, Dupuis D, Halberstadt AL, Kometer M, Kozakova E, LarØi F, Noorani TN, Preller KH, Waters F, Zaytseva Y, Jardri R. Hallucinations Under Psychedelics and in the Schizophrenia Spectrum: An Interdisciplinary and Multiscale Comparison. Schizophr Bull 2020; 46:1396-1408. [PMID: 32944778 PMCID: PMC7707069 DOI: 10.1093/schbul/sbaa117] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/28/2023]
Abstract
The recent renaissance of psychedelic science has reignited interest in the similarity of drug-induced experiences to those more commonly observed in psychiatric contexts such as the schizophrenia-spectrum. This report from a multidisciplinary working group of the International Consortium on Hallucinations Research (ICHR) addresses this issue, putting special emphasis on hallucinatory experiences. We review evidence collected at different scales of understanding, from pharmacology to brain-imaging, phenomenology and anthropology, highlighting similarities and differences between hallucinations under psychedelics and in the schizophrenia-spectrum disorders. Finally, we attempt to integrate these findings using computational approaches and conclude with recommendations for future research.
Collapse
Affiliation(s)
- Pantelis Leptourgos
- Department of Psychiatry, Connecticut Mental Health Center, Yale University, New Haven, CT
| | - Martin Fortier-Davy
- Institut Jean Nicod, École des Hautes Études en Sciences Sociales, École Normale Supérieure, PSL Research University, Paris France
| | | | - Philip R Corlett
- Department of Psychiatry, Connecticut Mental Health Center, Yale University, New Haven, CT
| | - David Dupuis
- Department of Anthropology, University of Durham, Durham, UK
| | - Adam L Halberstadt
- Department of Psychiatry, University of California San Diego, La Jolla, CA
- Research Service, VA San Diego Healthcare System, San Diego, CA
| | - Michael Kometer
- Pharmaco-Neuroimaging and Cognitive-Emotional Processing, Department of Psychiatry, Psychotherapy and Psychosomatics, Psychiatric University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Eva Kozakova
- Department of Applied Neurosciences and Brain Imaging, National Institute of Mental Health, Klecany, Czechia
- Department of Psychology, Faculty of Arts, Charles University, Prague, Czechia
| | - Frank LarØi
- Department of Biological and Medical Psychology, Faculty of Psychology, University of Bergen, Bergen, Norway
- Psychology and Neuroscience of Cognition Research Unit, University of Liège, Liège, Belgium
- Norwegian Center of Excellence for Mental Disorders Research, University of Oslo, Oslo, Norway
| | | | - Katrin H Preller
- Pharmaco-Neuroimaging and Cognitive-Emotional Processing, Department of Psychiatry, Psychotherapy and Psychosomatics, Psychiatric University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Flavie Waters
- School of Psychological Sciences, The University of Western Australia, Perth, Western Australia
| | - Yuliya Zaytseva
- Department of Applied Neurosciences and Brain Imaging, National Institute of Mental Health, Klecany, Czechia
- Department of Psychiatry and Medical Psychology, 3rd Faculty of Medicine, Charles University in Prague, Prague, Czechia
| | - Renaud Jardri
- Univ. Lille, INSERM U1172, CHU Lille, Lille Neuroscience & Cognition Centre (LiNC), Plasticity & SubjectivitY team, Lille, France
- Laboratoire de Neurosciences Cognitives et Computationnelles, ENS, INSERM U960, PSL Research University, Paris, France
| |
Collapse
|
31
|
Seth AK, Millidge B, Buckley CL, Tschantz A. Curious Inferences: Reply to Sun and Firestone on the Dark Room Problem. Trends Cogn Sci 2020; 24:681-683. [DOI: 10.1016/j.tics.2020.05.011] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2020] [Accepted: 05/22/2020] [Indexed: 10/23/2022]
|
32
|
Smith R, Steklis HD, Steklis NG, Weihs KL, Lane RD. The evolution and development of the uniquely human capacity for emotional awareness: A synthesis of comparative anatomical, cognitive, neurocomputational, and evolutionary psychological perspectives. Biol Psychol 2020; 154:107925. [DOI: 10.1016/j.biopsycho.2020.107925] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Revised: 06/17/2020] [Accepted: 06/23/2020] [Indexed: 01/09/2023]
|
33
|
Smith R, Schwartenbeck P, Parr T, Friston KJ. An Active Inference Approach to Modeling Structure Learning: Concept Learning as an Example Case. Front Comput Neurosci 2020; 14:41. [PMID: 32508611 PMCID: PMC7250191 DOI: 10.3389/fncom.2020.00041] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2019] [Accepted: 04/17/2020] [Indexed: 11/13/2022] Open
Abstract
Within computational neuroscience, the algorithmic and neural basis of structure learning remains poorly understood. Concept learning is one primary example, which requires both a type of internal model expansion process (adding novel hidden states that explain new observations), and a model reduction process (merging different states into one underlying cause and thus reducing model complexity via meta-learning). Although various algorithmic models of concept learning have been proposed within machine learning and cognitive science, many are limited to various degrees by an inability to generalize, the need for very large amounts of training data, and/or insufficiently established biological plausibility. Using concept learning as an example case, we introduce a novel approach for modeling structure learning-and specifically state-space expansion and reduction-within the active inference framework and its accompanying neural process theory. Our aim is to demonstrate its potential to facilitate a novel line of active inference research in this area. The approach we lay out is based on the idea that a generative model can be equipped with extra (hidden state or cause) "slots" that can be engaged when an agent learns about novel concepts. This can be combined with a Bayesian model reduction process, in which any concept learning-associated with these slots-can be reset in favor of a simpler model with higher model evidence. We use simulations to illustrate this model's ability to add new concepts to its state space (with relatively few observations) and increase the granularity of the concepts it currently possesses. We also simulate the predicted neural basis of these processes. We further show that it can accomplish a simple form of "one-shot" generalization to new stimuli. Although deliberately simple, these simulation results highlight ways in which active inference could offer useful resources in developing neurocomputational models of structure learning. They provide a template for how future active inference research could apply this approach to real-world structure learning problems and assess the added utility it may offer.
Collapse
Affiliation(s)
- Ryan Smith
- Laureate Institute for Brain Research, Tulsa, OK, United States
| | - Philipp Schwartenbeck
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, United Kingdom
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, United Kingdom
| | - Karl J. Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, United Kingdom
| |
Collapse
|
34
|
Parr T, Sajid N, Friston KJ. Modules or Mean-Fields? ENTROPY 2020; 22:e22050552. [PMID: 33286324 PMCID: PMC7517075 DOI: 10.3390/e22050552] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/01/2020] [Revised: 05/03/2020] [Accepted: 05/12/2020] [Indexed: 12/15/2022]
Abstract
The segregation of neural processing into distinct streams has been interpreted by some as evidence in favour of a modular view of brain function. This implies a set of specialised ‘modules’, each of which performs a specific kind of computation in isolation of other brain systems, before sharing the result of this operation with other modules. In light of a modern understanding of stochastic non-equilibrium systems, like the brain, a simpler and more parsimonious explanation presents itself. Formulating the evolution of a non-equilibrium steady state system in terms of its density dynamics reveals that such systems appear on average to perform a gradient ascent on their steady state density. If this steady state implies a sufficiently sparse conditional independency structure, this endorses a mean-field dynamical formulation. This decomposes the density over all states in a system into the product of marginal probabilities for those states. This factorisation lends the system a modular appearance, in the sense that we can interpret the dynamics of each factor independently. However, the argument here is that it is factorisation, as opposed to modularisation, that gives rise to the functional anatomy of the brain or, indeed, any sentient system. In the following, we briefly overview mean-field theory and its applications to stochastic dynamical systems. We then unpack the consequences of this factorisation through simple numerical simulations and highlight the implications for neuronal message passing and the computational architecture of sentience.
Collapse
|