1
|
Onagawa R, Kudo K, Watanabe K. Systematic bias in representation of reaction time distribution. Q J Exp Psychol (Hove) 2024; 77:2084-2097. [PMID: 38336626 DOI: 10.1177/17470218241234650] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/12/2024]
Abstract
A correct perception of one's own abilities is essential for making appropriate decisions. A well-known bias in probability perception is that rare events are overestimated. Here, we examined whether such a bias also exists for action outcomes using a simple reaction task. In Experiment 1, after completing a set of 30 trials of the simple reaction task, participants were required to judge the probability that they would be able to respond before a given reference time when performing the task next. We assessed the difference between the actual reaction times and the probability judgement and found that the represented probability distribution was more widely distributed than the actual one, suggesting that low-probability events were overestimated and high-probability events were underestimated. Experiment 2 confirmed the presence of such a bias in the representation of both one's own and another's reaction times. In addition, Experiment 3 showed the presence of such a bias regardless of the difference between the representation of another's reaction times and the mere numerical representation. Furthermore, Experiment 4 found the presence of such a bias even when the information regarding actual reaction times was visually shown before the representation. The present results reveal the existence of a highly robust bias in the representation of motor performance, which reflects the ubiquitous bias in probability perception and is difficult to eliminate.
Collapse
Affiliation(s)
- Ryoji Onagawa
- Faculty of Science and Engineering, Waseda University, Tokyo, Japan
- Japan Society for the Promotion of Science, Tokyo, Japan
| | - Kazutoshi Kudo
- Department of Life Sciences, Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, Japan
| | - Katsumi Watanabe
- Faculty of Science and Engineering, Waseda University, Tokyo, Japan
| |
Collapse
|
2
|
Hodson R, Mehta M, Smith R. The empirical status of predictive coding and active inference. Neurosci Biobehav Rev 2024; 157:105473. [PMID: 38030100 DOI: 10.1016/j.neubiorev.2023.105473] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Revised: 10/27/2023] [Accepted: 11/16/2023] [Indexed: 12/01/2023]
Abstract
Research on predictive processing models has focused largely on two specific algorithmic theories: Predictive Coding for perception and Active Inference for decision-making. While these interconnected theories possess broad explanatory potential, they have only recently begun to receive direct empirical evaluation. Here, we review recent studies of Predictive Coding and Active Inference with a focus on evaluating the degree to which they are empirically supported. For Predictive Coding, we find that existing empirical evidence offers modest support. However, some positive results can also be explained by alternative feedforward (e.g., feature detection-based) models. For Active Inference, most empirical studies have focused on fitting these models to behavior as a means of identifying and explaining individual or group differences. While Active Inference models tend to explain behavioral data reasonably well, there has not been a focus on testing empirical validity of active inference theory per se, which would require formal comparison to other models (e.g., non-Bayesian or model-free reinforcement learning models). This review suggests that, while promising, a number of specific research directions are still necessary to evaluate the empirical adequacy and explanatory power of these algorithms.
Collapse
Affiliation(s)
| | | | - Ryan Smith
- Laureate Institute for Brain Research, USA.
| |
Collapse
|
3
|
Pezzulo G, Parr T, Cisek P, Clark A, Friston K. Generating meaning: active inference and the scope and limits of passive AI. Trends Cogn Sci 2024; 28:97-112. [PMID: 37973519 DOI: 10.1016/j.tics.2023.10.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Revised: 10/03/2023] [Accepted: 10/05/2023] [Indexed: 11/19/2023]
Abstract
Prominent accounts of sentient behavior depict brains as generative models of organismic interaction with the world, evincing intriguing similarities with current advances in generative artificial intelligence (AI). However, because they contend with the control of purposive, life-sustaining sensorimotor interactions, the generative models of living organisms are inextricably anchored to the body and world. Unlike the passive models learned by generative AI systems, they must capture and control the sensory consequences of action. This allows embodied agents to intervene upon their worlds in ways that constantly put their best models to the test, thus providing a solid bedrock that is - we argue - essential to the development of genuine understanding. We review the resulting implications and consider future directions for generative AI.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| | - Thomas Parr
- Nuffield Department of Clinical Neurosciences, University of Oxford
| | - Paul Cisek
- Department of Neuroscience, University of Montréal, Montréal, Québec, Canada
| | - Andy Clark
- Department of Philosophy, University of Sussex, Brighton, UK; Department of Informatics, University of Sussex, Brighton, UK; Department of Philosophy, Macquarie University, Sydney, New South Wales, Australia
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London, UK; VERSES AI Research Lab, Los Angeles, CA, USA
| |
Collapse
|
4
|
Ward EK, Press C. Sixty years of predictive perception. Cortex 2024; 170:57-63. [PMID: 38104029 DOI: 10.1016/j.cortex.2023.11.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Revised: 11/14/2023] [Accepted: 11/14/2023] [Indexed: 12/19/2023]
Affiliation(s)
- Emma K Ward
- School of Psychological Sciences, Birkbeck, University of London, UK.
| | - Clare Press
- School of Psychological Sciences, Birkbeck, University of London, UK; Wellcome Centre for Human Neuroimaging, UCL, UK
| |
Collapse
|
5
|
Smith R. The path forward for modeling action-oriented cognition as active inference: Comment on "An active inference model of hierarchical action understanding, learning and imitation" by Riccardo Proietti, Giovanni Pezzulo, Alessia Tessari. Phys Life Rev 2023; 46:152-154. [PMID: 37437406 DOI: 10.1016/j.plrev.2023.06.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Accepted: 06/27/2023] [Indexed: 07/14/2023]
Affiliation(s)
- Ryan Smith
- Laureate Institute for Brain Research, United States of America.
| |
Collapse
|
6
|
Parr T, Holmes E, Friston KJ, Pezzulo G. Cognitive effort and active inference. Neuropsychologia 2023; 184:108562. [PMID: 37080424 PMCID: PMC10636588 DOI: 10.1016/j.neuropsychologia.2023.108562] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Revised: 04/03/2023] [Accepted: 04/11/2023] [Indexed: 04/22/2023]
Abstract
This paper aims to integrate some key constructs in the cognitive neuroscience of cognitive control and executive function by formalising the notion of cognitive (or mental) effort in terms of active inference. To do so, we call upon a task used in neuropsychology to assess impulse inhibition-a Stroop task. In this task, participants must suppress the impulse to read a colour word and instead report the colour of the text of the word. The Stroop task is characteristically effortful, and we unpack a theory of mental effort in which, to perform this task accurately, participants must overcome prior beliefs about how they would normally act. However, our interest here is not in overt action, but in covert (mental) action. Mental actions change our beliefs but have no (direct) effect on the outside world-much like deploying covert attention. This account of effort as mental action lets us generate multimodal (choice, reaction time, and electrophysiological) data of the sort we might expect from a human participant engaging in this task. We analyse how parameters determining cognitive effort influence simulated responses and demonstrate that-when provided only with performance data-these parameters can be recovered, provided they are within a certain range.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, UK.
| | - Emma Holmes
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, UK
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, UK
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| |
Collapse
|
7
|
Gijsen S, Grundei M, Blankenburg F. Active inference and the two-step task. Sci Rep 2022; 12:17682. [PMID: 36271279 PMCID: PMC9586964 DOI: 10.1038/s41598-022-21766-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Accepted: 09/30/2022] [Indexed: 01/18/2023] Open
Abstract
Sequential decision problems distill important challenges frequently faced by humans. Through repeated interactions with an uncertain world, unknown statistics need to be learned while balancing exploration and exploitation. Reinforcement learning is a prominent method for modeling such behaviour, with a prevalent application being the two-step task. However, recent studies indicate that the standard reinforcement learning model sometimes describes features of human task behaviour inaccurately and incompletely. We investigated whether active inference, a framework proposing a trade-off to the exploration-exploitation dilemma, could better describe human behaviour. Therefore, we re-analysed four publicly available datasets of the two-step task, performed Bayesian model selection, and compared behavioural model predictions. Two datasets, which revealed more model-based inference and behaviour indicative of directed exploration, were better described by active inference, while the models scored similarly for the remaining datasets. Learning using probability distributions appears to contribute to the improved model fits. Further, approximately half of all participants showed sensitivity to information gain as formulated under active inference, although behavioural exploration effects were not fully captured. These results contribute to the empirical validation of active inference as a model of human behaviour and the study of alternative models for the influential two-step task.
Collapse
Affiliation(s)
- Sam Gijsen
- Neurocomputation and Neuroimaging Unit, Freie Universität Berlin, 14195, Berlin, Germany.
- Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, 10117, Berlin, Germany.
| | - Miro Grundei
- Neurocomputation and Neuroimaging Unit, Freie Universität Berlin, 14195, Berlin, Germany
- Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, 10117, Berlin, Germany
| | - Felix Blankenburg
- Neurocomputation and Neuroimaging Unit, Freie Universität Berlin, 14195, Berlin, Germany
- Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, 10117, Berlin, Germany
| |
Collapse
|
8
|
Abstract
Sometimes agents choose to occupy environments that are neither traditionally rewarding nor worth exploring, but which rather promise to help minimise uncertainty related to what they can control. Selecting environments that afford inferences about agency seems a foundational aspect of environment selection dynamics - if an agent can't form reliable beliefs about what they can and can't control, then they can't act efficiently to achieve rewards. This relatively neglected aspect of environment selection is important to study so that we can better understand why agents occupy certain environments over others - something that may also be relevant for mental and developmental conditions, such as autism. This online experiment investigates the impact of uncertainty about agency on the way participants choose to freely move between two environments, one that has greater irreducible variability and one that is more complex to model. We hypothesise that increasingly erroneous predictions about the expected outcome of agency-exploring actions can be a driver of switching environments, and we explore which type of environment agents prefer. Results show that participants actively switch between the two environments following increases in prediction error, and that the tolerance for prediction error before switching is modulated by individuals' autism traits. Further, we find that participants more frequently occupy the variable environment, which is predicted by greater accuracy and higher confidence than the complex environment. This is the first online study to investigate relatively unconstrained ongoing foraging dynamics in support of judgements of agency, and in doing so represents a significant methodological advance.
Collapse
|
9
|
Chen WH. Perspective view of autonomous control in unknown environment: Dual control for exploitation and exploration vs reinforcement learning. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.04.131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
10
|
Sarasso P, Francesetti G, Roubal J, Gecele M, Ronga I, Neppi-Modona M, Sacco K. Beauty and Uncertainty as Transformative Factors: A Free Energy Principle Account of Aesthetic Diagnosis and Intervention in Gestalt Psychotherapy. Front Hum Neurosci 2022; 16:906188. [PMID: 35911596 PMCID: PMC9325967 DOI: 10.3389/fnhum.2022.906188] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Accepted: 06/09/2022] [Indexed: 11/13/2022] Open
Abstract
Drawing from field theory, Gestalt therapy conceives psychological suffering and psychotherapy as two intentional field phenomena, where unprocessed and chaotic experiences seek the opportunity to emerge and be assimilated through the contact between the patient and the therapist (i.e., the intentionality of contacting). This therapeutic approach is based on the therapist’s aesthetic experience of his/her embodied presence in the flow of the healing process because (1) the perception of beauty can provide the therapist with feedback on the assimilation of unprocessed experiences; (2) the therapist’s attentional focus on intrinsic aesthetic diagnostic criteria can facilitate the modification of rigid psychopathological fields by supporting the openness to novel experiences. The aim of the present manuscript is to review recent evidence from psychophysiology, neuroaesthetic research, and neurocomputational models of cognition, such as the free energy principle (FEP), which support the notion of the therapeutic potential of aesthetic sensibility in Gestalt psychotherapy. Drawing from neuroimaging data, psychophysiology and recent neurocognitive accounts of aesthetic perception, we propose a novel interpretation of the sense of beauty as a self-generated reward motivating us to assimilate an ever-greater spectrum of sensory and affective states in our predictive representation of ourselves and the world and supporting the intentionality of contact. Expecting beauty, in the psychotherapeutic encounter, can help therapists tolerate uncertainty avoiding impulsive behaviours and to stay tuned to the process of change.
Collapse
Affiliation(s)
- Pietro Sarasso
- BraIn Plasticity and Behaviour Changes Research Group, Department of Psychology, University of Turin, Turin, Italy
- *Correspondence: Pietro Sarasso,
| | - Gianni Francesetti
- International Institute for Gestalt Therapy and Psychopathology, Turin Center for Gestalt Therapy, Turin, Italy
| | - Jan Roubal
- Psychotherapy Training Gestalt Studia, Training in Psychotherapy Integration, Center for Psychotherapy Research in Brno, Masaryk University, Brno, Czechia
| | - Michela Gecele
- International Institute for Gestalt Therapy and Psychopathology, Turin Center for Gestalt Therapy, Turin, Italy
| | - Irene Ronga
- BraIn Plasticity and Behaviour Changes Research Group, Department of Psychology, University of Turin, Turin, Italy
| | - Marco Neppi-Modona
- BraIn Plasticity and Behaviour Changes Research Group, Department of Psychology, University of Turin, Turin, Italy
| | - Katiuscia Sacco
- BraIn Plasticity and Behaviour Changes Research Group, Department of Psychology, University of Turin, Turin, Italy
| |
Collapse
|
11
|
Lin CHS, Garrido MI. Towards a cross-level understanding of Bayesian inference in the brain. Neurosci Biobehav Rev 2022; 137:104649. [PMID: 35395333 DOI: 10.1016/j.neubiorev.2022.104649] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2021] [Revised: 02/28/2022] [Accepted: 03/29/2022] [Indexed: 10/18/2022]
Abstract
Perception emerges from unconscious probabilistic inference, which guides behaviour in our ubiquitously uncertain environment. Bayesian decision theory is a prominent computational model that describes how people make rational decisions using noisy and ambiguous sensory observations. However, critical questions have been raised about the validity of the Bayesian framework in explaining the mental process of inference. Firstly, some natural behaviours deviate from Bayesian optimum. Secondly, the neural mechanisms that support Bayesian computations in the brain are yet to be understood. Taking Marr's cross level approach, we review the recent progress made in addressing these challenges. We first review studies that combined behavioural paradigms and modelling approaches to explain both optimal and suboptimal behaviours. Next, we evaluate the theoretical advances and the current evidence for ecologically feasible algorithms and neural implementations in the brain, which may enable probabilistic inference. We argue that this cross-level approach is necessary for the worthwhile pursuit to uncover mechanistic accounts of human behaviour.
Collapse
Affiliation(s)
- Chin-Hsuan Sophie Lin
- Melbourne School of Psychological Sciences, The University of Melbourne, Australia; Australian Research Council for Integrative Brain Function, Australia.
| | - Marta I Garrido
- Melbourne School of Psychological Sciences, The University of Melbourne, Australia; Australian Research Council for Integrative Brain Function, Australia
| |
Collapse
|
12
|
Smith R, Friston KJ, Whyte CJ. A step-by-step tutorial on active inference and its application to empirical data. JOURNAL OF MATHEMATICAL PSYCHOLOGY 2022; 107:102632. [PMID: 35340847 PMCID: PMC8956124 DOI: 10.1016/j.jmp.2021.102632] [Citation(s) in RCA: 37] [Impact Index Per Article: 18.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
The active inference framework, and in particular its recent formulation as a partially observable Markov decision process (POMDP), has gained increasing popularity in recent years as a useful approach for modeling neurocognitive processes. This framework is highly general and flexible in its ability to be customized to model any cognitive process, as well as simulate predicted neuronal responses based on its accompanying neural process theory. It also affords both simulation experiments for proof of principle and behavioral modeling for empirical studies. However, there are limited resources that explain how to build and run these models in practice, which limits their widespread use. Most introductions assume a technical background in programming, mathematics, and machine learning. In this paper we offer a step-by-step tutorial on how to build POMDPs, run simulations using standard MATLAB routines, and fit these models to empirical data. We assume a minimal background in programming and mathematics, thoroughly explain all equations, and provide exemplar scripts that can be customized for both theoretical and empirical studies. Our goal is to provide the reader with the requisite background knowledge and practical tools to apply active inference to their own research. We also provide optional technical sections and multiple appendices, which offer the interested reader additional technical details. This tutorial should provide the reader with all the tools necessary to use these models and to follow emerging advances in active inference research.
Collapse
Affiliation(s)
- Ryan Smith
- Laureate Institute for Brain Research, Tulsa, OK, USA
| | - Karl J. Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, WC1N 3AR, UK
| | | |
Collapse
|
13
|
Tschantz A, Barca L, Maisto D, Buckley CL, Seth AK, Pezzulo G. Simulating homeostatic, allostatic and goal-directed forms of interoceptive control using active inference. Biol Psychol 2022; 169:108266. [DOI: 10.1016/j.biopsycho.2022.108266] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2021] [Revised: 01/06/2022] [Accepted: 01/14/2022] [Indexed: 12/28/2022]
|
14
|
Villiger D. How Psychedelic-Assisted Treatment Works in the Bayesian Brain. Front Psychiatry 2022; 13:812180. [PMID: 35360137 PMCID: PMC8963812 DOI: 10.3389/fpsyt.2022.812180] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Accepted: 02/11/2022] [Indexed: 11/25/2022] Open
Abstract
Psychedelics are experiencing a renaissance in clinical research. In recent years, an increasing number of studies on psychedelic-assisted treatment have been conducted. So far, the results are promising, suggesting that this new (or rather, rediscovered) form of therapy has great potential. One particular reason for that appears to be the synergistic combination of the pharmacological and psychotherapeutic interventions in psychedelic-assisted treatment. But how exactly do these two interventions complement each other? This paper provides the first account of the interaction between pharmacological and psychological effects in psychedelic-assisted treatment. Building on the relaxed beliefs under psychedelics (REBUS) hypothesis of Carhart-Harris and Friston and the contextual model of Wampold, it argues that psychedelics amplify the common factors and thereby the remedial effects of psychotherapy. More precisely, psychedelics are assumed to attenuate the precision of high-level predictions, making them more revisable by bottom-up input. Psychotherapy constitutes an important source of such input. At best, it signalizes a safe and supportive environment (cf. setting) and induces remedial expectations (cf. set). During treatment, these signals should become incorporated when high-level predictions are revised: a process that is hypothesized to occur as a matter of course in psychotherapy but to get reinforced and accelerated under psychedelics. Ultimately, these revisions should lead to a relief of symptoms.
Collapse
Affiliation(s)
- Daniel Villiger
- Department of Psychosomatics and Psychotherapy, Psychiatric University Hospital Basel, University of Basel, Basel, Switzerland.,Institute of Philosophy, University of Zurich, Zurich, Switzerland
| |
Collapse
|
15
|
Panitz C, Endres D, Buchholz M, Khosrowtaj Z, Sperl MFJ, Mueller EM, Schubö A, Schütz AC, Teige-Mocigemba S, Pinquart M. A Revised Framework for the Investigation of Expectation Update Versus Maintenance in the Context of Expectation Violations: The ViolEx 2.0 Model. Front Psychol 2021; 12:726432. [PMID: 34858264 PMCID: PMC8632008 DOI: 10.3389/fpsyg.2021.726432] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2021] [Accepted: 10/20/2021] [Indexed: 12/22/2022] Open
Abstract
Expectations are probabilistic beliefs about the future that shape and influence our perception, affect, cognition, and behavior in many contexts. This makes expectations a highly relevant concept across basic and applied psychological disciplines. When expectations are confirmed or violated, individuals can respond by either updating or maintaining their prior expectations in light of the new evidence. Moreover, proactive and reactive behavior can change the probability with which individuals encounter expectation confirmations or violations. The investigation of predictors and mechanisms underlying expectation update and maintenance has been approached from many research perspectives. However, in many instances there has been little exchange between different research fields. To further advance research on expectations and expectation violations, collaborative efforts across different disciplines in psychology, cognitive (neuro)science, and other life sciences are warranted. For fostering and facilitating such efforts, we introduce the ViolEx 2.0 model, a revised framework for interdisciplinary research on cognitive and behavioral mechanisms of expectation update and maintenance in the context of expectation violations. To support different goals and stages in interdisciplinary exchange, the ViolEx 2.0 model features three model levels with varying degrees of specificity in order to address questions about the research synopsis, central concepts, or functional processes and relationships, respectively. The framework can be applied to different research fields and has high potential for guiding collaborative research efforts in expectation research.
Collapse
Affiliation(s)
- Christian Panitz
- Department of Psychology, University of Marburg, Marburg, Germany.,Department of Psychology, University of Leipzig, Leipzig, Germany.,Center for the Study of Emotion and Attention, University of Florida, Gainesville, FL, United States
| | - Dominik Endres
- Department of Psychology, University of Marburg, Marburg, Germany
| | - Merle Buchholz
- Department of Psychology, University of Marburg, Marburg, Germany
| | - Zahra Khosrowtaj
- Department of Psychology, University of Marburg, Marburg, Germany
| | - Matthias F J Sperl
- Department of Psychology, University of Marburg, Marburg, Germany.,Department of Psychology, University of Giessen, Giessen, Germany
| | - Erik M Mueller
- Department of Psychology, University of Marburg, Marburg, Germany
| | - Anna Schubö
- Department of Psychology, University of Marburg, Marburg, Germany
| | | | | | - Martin Pinquart
- Department of Psychology, University of Marburg, Marburg, Germany
| |
Collapse
|
16
|
Parr T, Pezzulo G. Understanding, Explanation, and Active Inference. Front Syst Neurosci 2021; 15:772641. [PMID: 34803619 PMCID: PMC8602880 DOI: 10.3389/fnsys.2021.772641] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2021] [Accepted: 10/15/2021] [Indexed: 11/13/2022] Open
Abstract
While machine learning techniques have been transformative in solving a range of problems, an important challenge is to understand why they arrive at the decisions they output. Some have argued that this necessitates augmenting machine intelligence with understanding such that, when queried, a machine is able to explain its behaviour (i.e., explainable AI). In this article, we address the issue of machine understanding from the perspective of active inference. This paradigm enables decision making based upon a model of how data are generated. The generative model contains those variables required to explain sensory data, and its inversion may be seen as an attempt to explain the causes of these data. Here we are interested in explanations of one's own actions. This implies a deep generative model that includes a model of the world, used to infer policies, and a higher-level model that attempts to predict which policies will be selected based upon a space of hypothetical (i.e., counterfactual) explanations-and which can subsequently be used to provide (retrospective) explanations about the policies pursued. We illustrate the construct validity of this notion of understanding in relation to human understanding by highlighting the similarities in computational architecture and the consequences of its dysfunction.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London, United Kingdom
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| |
Collapse
|
17
|
Friston K, Moran RJ, Nagai Y, Taniguchi T, Gomi H, Tenenbaum J. World model learning and inference. Neural Netw 2021; 144:573-590. [PMID: 34634605 DOI: 10.1016/j.neunet.2021.09.011] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2021] [Revised: 07/28/2021] [Accepted: 09/09/2021] [Indexed: 11/19/2022]
Abstract
Understanding information processing in the brain-and creating general-purpose artificial intelligence-are long-standing aspirations of scientists and engineers worldwide. The distinctive features of human intelligence are high-level cognition and control in various interactions with the world including the self, which are not defined in advance and are vary over time. The challenge of building human-like intelligent machines, as well as progress in brain science and behavioural analyses, robotics, and their associated theoretical formalisations, speaks to the importance of the world-model learning and inference. In this article, after briefly surveying the history and challenges of internal model learning and probabilistic learning, we introduce the free energy principle, which provides a useful framework within which to consider neuronal computation and probabilistic world models. Next, we showcase examples of human behaviour and cognition explained under that principle. We then describe symbol emergence in the context of probabilistic modelling, as a topic at the frontiers of cognitive robotics. Lastly, we review recent progress in creating human-like intelligence by using novel probabilistic programming languages. The striking consensus that emerges from these studies is that probabilistic descriptions of learning and inference are powerful and effective ways to create human-like artificial intelligent machines and to understand intelligence in the context of how humans interact with their world.
Collapse
Affiliation(s)
- Karl Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London (UCL), WC1N 3BG, UK.
| | - Rosalyn J Moran
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, SE5 8AF, UK.
| | - Yukie Nagai
- International Research Center for Neurointelligence (IRCN), The University of Tokyo, Tokyo, Japan.
| | - Tadahiro Taniguchi
- College of Information Science and Engineering, Ritsumeikan University, Shiga, Japan.
| | - Hiroaki Gomi
- NTT Communication Science Labs., Nippon Telegraph and Telephone, Kanawaga, Japan.
| | - Josh Tenenbaum
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA, USA; The Center for Brains, Minds and Machines, MIT, Cambridge, MA, USA.
| |
Collapse
|
18
|
Deane G. Consciousness in active inference: Deep self-models, other minds, and the challenge of psychedelic-induced ego-dissolution. Neurosci Conscious 2021; 2021:niab024. [PMID: 34484808 PMCID: PMC8408766 DOI: 10.1093/nc/niab024] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Revised: 07/26/2021] [Accepted: 08/02/2021] [Indexed: 11/16/2022] Open
Abstract
Predictive processing approaches to brain function are increasingly delivering promise for illuminating the computational underpinnings of a wide range of phenomenological states. It remains unclear, however, whether predictive processing is equipped to accommodate a theory of consciousness itself. Furthermore, objectors have argued that without specification of the core computational mechanisms of consciousness, predictive processing is unable to inform the attribution of consciousness to other non-human (biological and artificial) systems. In this paper, I argue that an account of consciousness in the predictive brain is within reach via recent accounts of phenomenal self-modelling in the active inference framework. The central claim here is that phenomenal consciousness is underpinned by 'subjective valuation'-a deep inference about the precision or 'predictability' of the self-evidencing ('fitness-promoting') outcomes of action. Based on this account, I argue that this approach can critically inform the distribution of experience in other systems, paying particular attention to the complex sensory attenuation mechanisms associated with deep self-models. I then consider an objection to the account: several recent papers argue that theories of consciousness that invoke self-consciousness as constitutive or necessary for consciousness are undermined by states (or traits) of 'selflessness'; in particular the 'totally selfless' states of ego-dissolution occasioned by psychedelic drugs. Drawing on existing work that accounts for psychedelic-induced ego-dissolution in the active inference framework, I argue that these states do not threaten to undermine an active inference theory of consciousness. Instead, these accounts corroborate the view that subjective valuation is the constitutive facet of experience, and they highlight the potential of psychedelic research to inform consciousness science, computational psychiatry and computational phenomenology.
Collapse
Affiliation(s)
- George Deane
- School of Philosophy, Psychology and Language Sciences, The University of Edinburgh, 3 Charles Street, Edinburgh EH8 9AD, UK
| |
Collapse
|
19
|
Abstract
Human social interactions depend on the ability to resolve uncertainty about the mental states of others. The context in which social interactions take place is crucial for mental state attribution as sensory inputs may be perceived differently depending on the context. In this paper, we introduce a mental state attribution task where a target-face with either an ambiguous or an unambiguous emotion is embedded in different social contexts. The social context is determined by the emotions conveyed by other faces in the scene. This task involves mental state attribution to a target-face (either happy or sad) depending on the social context. Using active inference models, we provide a proof of concept that an agent's perception of sensory stimuli may be altered by social context. We show with simulations that context congruency and facial expression coherency improve behavioural performance in terms of decision times. Furthermore, we show through simulations that the abnormal viewing strategies employed by patients with schizophrenia may be due to (i) an imbalance between the precisions of local and global features in the scene and (ii) a failure to modulate the sensory precision to contextualise emotions.
Collapse
|
20
|
Parr T, Limanowski J, Rawji V, Friston K. The computational neurology of movement under active inference. Brain 2021; 144:1799-1818. [PMID: 33704439 PMCID: PMC8320263 DOI: 10.1093/brain/awab085] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2020] [Revised: 11/08/2020] [Accepted: 12/20/2020] [Indexed: 12/31/2022] Open
Abstract
We propose a computational neurology of movement based on the convergence of theoretical neurobiology and clinical neurology. A significant development in the former is the idea that we can frame brain function as a process of (active) inference, in which the nervous system makes predictions about its sensory data. These predictions depend upon an implicit predictive (generative) model used by the brain. This means neural dynamics can be framed as generating actions to ensure sensations are consistent with these predictions-and adjusting predictions when they are not. We illustrate the significance of this formulation for clinical neurology by simulating a clinical examination of the motor system using an upper limb coordination task. Specifically, we show how tendon reflexes emerge naturally under the right kind of generative model. Through simulated perturbations, pertaining to prior probabilities of this model's variables, we illustrate the emergence of hyperreflexia and pendular reflexes, reminiscent of neurological lesions in the corticospinal tract and cerebellum. We then turn to the computational lesions causing hypokinesia and deficits of coordination. This in silico lesion-deficit analysis provides an opportunity to revisit classic neurological dichotomies (e.g. pyramidal versus extrapyramidal systems) from the perspective of modern approaches to theoretical neurobiology-and our understanding of the neurocomputational architecture of movement control based on first principles.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London WC1N 3BG, UK
| | - Jakub Limanowski
- Faculty of Psychology and Center for Tactile Internet with Human-in-the-Loop, Technische Universität Dresden, Dresden, Germany
| | - Vishal Rawji
- Department of Clinical and Movement Neurosciences, Queen Square Institute of Neurology, University College London, London WC1N 3BG, UK
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London WC1N 3BG, UK
| |
Collapse
|
21
|
Champion T, Grześ M, Bowman H. Realizing Active Inference in Variational Message Passing: The Outcome-Blind Certainty Seeker. Neural Comput 2021; 33:2762-2826. [PMID: 34280302 DOI: 10.1162/neco_a_01422] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Accepted: 04/20/2021] [Indexed: 11/04/2022]
Abstract
Active inference is a state-of-the-art framework in neuroscience that offers a unified theory of brain function. It is also proposed as a framework for planning in AI. Unfortunately, the complex mathematics required to create new models can impede application of active inference in neuroscience and AI research. This letter addresses this problem by providing a complete mathematical treatment of the active inference framework in discrete time and state spaces and the derivation of the update equations for any new model. We leverage the theoretical connection between active inference and variational message passing as described by John Winn and Christopher M. Bishop in 2005. Since variational message passing is a well-defined methodology for deriving Bayesian belief update equations, this letter opens the door to advanced generative models for active inference. We show that using a fully factorized variational distribution simplifies the expected free energy, which furnishes priors over policies so that agents seek unambiguous states. Finally, we consider future extensions that support deep tree searches for sequential policy optimization based on structure learning and belief propagation.
Collapse
Affiliation(s)
| | - Marek Grześ
- University of Kent, School of Computing, Canterbury CT2 7NZ, U.K.
| | - Howard Bowman
- University of Birmingham, School of Psychology, Birmingham B15 2TT, U.K., and University of Kent, School of Computing, Canterbury CT2 7NZ, U.K.
| |
Collapse
|
22
|
Parr T, Rikhye RV, Halassa MM, Friston KJ. Prefrontal Computation as Active Inference. Cereb Cortex 2021; 30:682-695. [PMID: 31298270 PMCID: PMC7444741 DOI: 10.1093/cercor/bhz118] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2019] [Revised: 05/01/2019] [Accepted: 05/11/2019] [Indexed: 12/22/2022] Open
Abstract
The prefrontal cortex is vital for a range of cognitive processes, including working memory, attention, and decision-making. Notably, its absence impairs the performance of tasks requiring the maintenance of information through a delay period. In this paper, we formulate a rodent task—which requires maintenance of delay-period activity—as a Markov decision process and treat optimal task performance as an (active) inference problem. We simulate the behavior of a Bayes optimal mouse presented with 1 of 2 cues that instructs the selection of concurrent visual and auditory targets on a trial-by-trial basis. Formulating inference as message passing, we reproduce features of neuronal coupling within and between prefrontal regions engaged by this task. We focus on the micro-circuitry that underwrites delay-period activity and relate it to functional specialization within the prefrontal cortex in primates. Finally, we simulate the electrophysiological correlates of inference and demonstrate the consequences of lesions to each part of our in silico prefrontal cortex. In brief, this formulation suggests that recurrent excitatory connections—which support persistent neuronal activity—encode beliefs about transition probabilities over time. We argue that attentional modulation can be understood as the contextualization of sensory input by these persistent beliefs.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, WC1N 3BG, UK
| | - Rajeev Vijay Rikhye
- Department of Brain and Cognitive Science, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.,McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Michael M Halassa
- Department of Brain and Cognitive Science, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.,McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.,Stanley Center for Psychiatric Genetics, Broad Institute, Cambridge, MA 02139, USA
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, WC1N 3BG, UK
| |
Collapse
|
23
|
Alexithymia explains atypical spatiotemporal dynamics of eye gaze in autism. Cognition 2021; 212:104710. [PMID: 33862441 DOI: 10.1016/j.cognition.2021.104710] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2020] [Revised: 03/24/2021] [Accepted: 03/26/2021] [Indexed: 11/23/2022]
Abstract
Recognition of emotional facial expressions is considered to be atypical in autism. This difficulty is thought to be due to the way that facial expressions are visually explored. Evidence for atypical visual exploration of emotional faces in autism is, however, equivocal. We propose that, where observed, atypical visual exploration of emotional facial expressions is due to alexithymia, a distinct but frequently co-occurring condition. In this eye-tracking study we tested the alexithymia hypothesis using a number of recent methodological advances to study eye gaze during several emotion processing tasks (emotion recognition, intensity judgements, free gaze), in 25 adults with, and 45 without, autism. A multilevel polynomial modelling strategy was used to describe the spatiotemporal dynamics of eye gaze to emotional facial expressions. Converging evidence from traditional and novel analysis methods revealed that atypical gaze to the eyes is best predicted by alexithymia in both autistic and non-autistic individuals. Information theoretic analyses also revealed differential effects of task on gaze patterns as a function of alexithymia, but not autism. These findings highlight factors underlying atypical emotion processing in autistic individuals, with wide-ranging implications for emotion research.
Collapse
|
24
|
Parr T, Sajid N, Da Costa L, Mirza MB, Friston KJ. Generative Models for Active Vision. Front Neurorobot 2021; 15:651432. [PMID: 33927605 PMCID: PMC8076554 DOI: 10.3389/fnbot.2021.651432] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2021] [Accepted: 03/15/2021] [Indexed: 11/13/2022] Open
Abstract
The active visual system comprises the visual cortices, cerebral attention networks, and oculomotor system. While fascinating in its own right, it is also an important model for sensorimotor networks in general. A prominent approach to studying this system is active inference-which assumes the brain makes use of an internal (generative) model to predict proprioceptive and visual input. This approach treats action as ensuring sensations conform to predictions (i.e., by moving the eyes) and posits that visual percepts are the consequence of updating predictions to conform to sensations. Under active inference, the challenge is to identify the form of the generative model that makes these predictions-and thus directs behavior. In this paper, we provide an overview of the generative models that the brain must employ to engage in active vision. This means specifying the processes that explain retinal cell activity and proprioceptive information from oculomotor muscle fibers. In addition to the mechanics of the eyes and retina, these processes include our choices about where to move our eyes. These decisions rest upon beliefs about salient locations, or the potential for information gain and belief-updating. A key theme of this paper is the relationship between "looking" and "seeing" under the brain's implicit generative model of the visual world.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, London, United Kingdom
| | - Noor Sajid
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, London, United Kingdom
| | - Lancelot Da Costa
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, London, United Kingdom
- Department of Mathematics, Imperial College London, London, United Kingdom
| | - M. Berk Mirza
- Department of Neuroimaging, Centre for Neuroimaging Sciences, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom
| | - Karl J. Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, London, United Kingdom
| |
Collapse
|
25
|
Da Costa L, Parr T, Sengupta B, Friston K. Neural Dynamics under Active Inference: Plausibility and Efficiency of Information Processing. ENTROPY (BASEL, SWITZERLAND) 2021; 23:454. [PMID: 33921298 PMCID: PMC8069154 DOI: 10.3390/e23040454] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/16/2021] [Accepted: 04/06/2021] [Indexed: 02/07/2023]
Abstract
Active inference is a normative framework for explaining behaviour under the free energy principle-a theory of self-organisation originating in neuroscience. It specifies neuronal dynamics for state-estimation in terms of a descent on (variational) free energy-a measure of the fit between an internal (generative) model and sensory observations. The free energy gradient is a prediction error-plausibly encoded in the average membrane potentials of neuronal populations. Conversely, the expected probability of a state can be expressed in terms of neuronal firing rates. We show that this is consistent with current models of neuronal dynamics and establish face validity by synthesising plausible electrophysiological responses. We then show that these neuronal dynamics approximate natural gradient descent, a well-known optimisation algorithm from information geometry that follows the steepest descent of the objective in information space. We compare the information length of belief updating in both schemes, a measure of the distance travelled in information space that has a direct interpretation in terms of metabolic cost. We show that neural dynamics under active inference are metabolically efficient and suggest that neural representations in biological agents may evolve by approximating steepest descent in information space towards the point of optimal inference.
Collapse
Affiliation(s)
- Lancelot Da Costa
- Department of Mathematics, Imperial College London, London SW7 2AZ, UK
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK; (T.P.); (B.S.); (K.F.)
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK; (T.P.); (B.S.); (K.F.)
| | - Biswa Sengupta
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK; (T.P.); (B.S.); (K.F.)
- Core Machine Learning Group, Zebra AI, London WC2H 8TJ, UK
- Department of Bioengineering, Imperial College London, London SW7 2AZ, UK
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK; (T.P.); (B.S.); (K.F.)
| |
Collapse
|
26
|
Van de Maele T, Verbelen T, Çatal O, De Boom C, Dhoedt B. Active Vision for Robot Manipulators Using the Free Energy Principle. Front Neurorobot 2021; 15:642780. [PMID: 33746730 PMCID: PMC7973267 DOI: 10.3389/fnbot.2021.642780] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2020] [Accepted: 02/03/2021] [Indexed: 11/14/2022] Open
Abstract
Occlusions, restricted field of view and limited resolution all constrain a robot's ability to sense its environment from a single observation. In these cases, the robot first needs to actively query multiple observations and accumulate information before it can complete a task. In this paper, we cast this problem of active vision as active inference, which states that an intelligent agent maintains a generative model of its environment and acts in order to minimize its surprise, or expected free energy according to this model. We apply this to an object-reaching task for a 7-DOF robotic manipulator with an in-hand camera to scan the workspace. A novel generative model using deep neural networks is proposed that is able to fuse multiple views into an abstract representation and is trained from data by minimizing variational free energy. We validate our approach experimentally for a reaching task in simulation in which a robotic agent starts without any knowledge about its workspace. Each step, the next view pose is chosen by evaluating the expected free energy. We find that by minimizing the expected free energy, exploratory behavior emerges when the target object to reach is not in view, and the end effector is moved to the correct reach position once the target is located. Similar to an owl scavenging for prey, the robot naturally prefers higher ground for exploring, approaching its target once located.
Collapse
Affiliation(s)
- Toon Van de Maele
- IDLab, Department of Information Technology, Ghent University—imec, Ghent, Belgium
| | | | | | | | | |
Collapse
|
27
|
Abstract
The concept of free energy has its origins in 19th century thermodynamics, but has recently found its way into the behavioral and neural sciences, where it has been promoted for its wide applicability and has even been suggested as a fundamental principle of understanding intelligent behavior and brain function. We argue that there are essentially two different notions of free energy in current models of intelligent agency, that can both be considered as applications of Bayesian inference to the problem of action selection: one that appears when trading off accuracy and uncertainty based on a general maximum entropy principle, and one that formulates action selection in terms of minimizing an error measure that quantifies deviations of beliefs and policies from given reference models. The first approach provides a normative rule for action selection in the face of model uncertainty or when information processing capabilities are limited. The second approach directly aims to formulate the action selection problem as an inference problem in the context of Bayesian brain theories, also known as Active Inference in the literature. We elucidate the main ideas and discuss critical technical and conceptual issues revolving around these two notions of free energy that both claim to apply at all levels of decision-making, from the high-level deliberation of reasoning down to the low-level information processing of perception.
Collapse
Affiliation(s)
- Sebastian Gottwald
- Institute of Neural Information Processing, Ulm University, Ulm, Germany
| | - Daniel A. Braun
- Institute of Neural Information Processing, Ulm University, Ulm, Germany
| |
Collapse
|
28
|
Da Costa L, Parr T, Sajid N, Veselic S, Neacsu V, Friston K. Active inference on discrete state-spaces: A synthesis. JOURNAL OF MATHEMATICAL PSYCHOLOGY 2020; 99:102447. [PMID: 33343039 PMCID: PMC7732703 DOI: 10.1016/j.jmp.2020.102447] [Citation(s) in RCA: 73] [Impact Index Per Article: 18.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/17/2020] [Revised: 07/23/2020] [Accepted: 09/03/2020] [Indexed: 05/05/2023]
Abstract
Active inference is a normative principle underwriting perception, action, planning, decision-making and learning in biological or artificial agents. From its inception, its associated process theory has grown to incorporate complex generative models, enabling simulation of a wide range of complex behaviours. Due to successive developments in active inference, it is often difficult to see how its underlying principle relates to process theories and practical implementation. In this paper, we try to bridge this gap by providing a complete mathematical synthesis of active inference on discrete state-space models. This technical summary provides an overview of the theory, derives neuronal dynamics from first principles and relates this dynamics to biological processes. Furthermore, this paper provides a fundamental building block needed to understand active inference for mixed generative models; allowing continuous sensations to inform discrete representations. This paper may be used as follows: to guide research towards outstanding challenges, a practical guide on how to implement active inference to simulate experimental behaviour, or a pointer towards various in-silico neurophysiological responses that may be used to make empirical predictions.
Collapse
Affiliation(s)
- Lancelot Da Costa
- Department of Mathematics, Imperial College London, London, SW7 2RH, United Kingdom
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom
| | - Noor Sajid
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom
| | - Sebastijan Veselic
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom
| | - Victorita Neacsu
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom
| |
Collapse
|
29
|
Heins RC, Mirza MB, Parr T, Friston K, Kagan I, Pooresmaeili A. Deep Active Inference and Scene Construction. Front Artif Intell 2020; 3:509354. [PMID: 33733195 PMCID: PMC7861336 DOI: 10.3389/frai.2020.509354] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2019] [Accepted: 09/10/2020] [Indexed: 11/17/2022] Open
Abstract
Adaptive agents must act in intrinsically uncertain environments with complex latent structure. Here, we elaborate a model of visual foraging-in a hierarchical context-wherein agents infer a higher-order visual pattern (a "scene") by sequentially sampling ambiguous cues. Inspired by previous models of scene construction-that cast perception and action as consequences of approximate Bayesian inference-we use active inference to simulate decisions of agents categorizing a scene in a hierarchically-structured setting. Under active inference, agents develop probabilistic beliefs about their environment, while actively sampling it to maximize the evidence for their internal generative model. This approximate evidence maximization (i.e., self-evidencing) comprises drives to both maximize rewards and resolve uncertainty about hidden states. This is realized via minimization of a free energy functional of posterior beliefs about both the world as well as the actions used to sample or perturb it, corresponding to perception and action, respectively. We show that active inference, in the context of hierarchical scene construction, gives rise to many empirical evidence accumulation phenomena, such as noise-sensitive reaction times and epistemic saccades. We explain these behaviors in terms of the principled drives that constitute the expected free energy, the key quantity for evaluating policies under active inference. In addition, we report novel behaviors exhibited by these active inference agents that furnish new predictions for research on evidence accumulation and perceptual decision-making. We discuss the implications of this hierarchical active inference scheme for tasks that require planned sequences of information-gathering actions to infer compositional latent structure (such as visual scene construction and sentence comprehension). This work sets the stage for future experiments to investigate active inference in relation to other formulations of evidence accumulation (e.g., drift-diffusion models) in tasks that require planning in uncertain environments with higher-order structure.
Collapse
Affiliation(s)
- R. Conor Heins
- Department of Collective Behaviour, Max Planck Institute for Animal Behavior, Konstanz, Germany
- Perception and Cognition Group, European Neuroscience Institute, A Joint Initiative of the University Medical Centre Göttingen and the Max-Planck-Society, Göttingen, Germany
- Leibniz Science Campus “Primate Cognition”, Göttingen, Germany
| | - M. Berk Mirza
- Department of Neuroimaging, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom
- The National Institute for Health Research (NIHR) Maudsley Biomedical Research Centre (BRC) at South London and Maudsley National Health Service (NHS) Foundation Trust and The Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
| | - Igor Kagan
- Leibniz Science Campus “Primate Cognition”, Göttingen, Germany
- Decision and Awareness Group, Cognitive Neuroscience Laboratory, German Primate Centre (DPZ), Göttingen, Germany
| | - Arezoo Pooresmaeili
- Perception and Cognition Group, European Neuroscience Institute, A Joint Initiative of the University Medical Centre Göttingen and the Max-Planck-Society, Göttingen, Germany
- Leibniz Science Campus “Primate Cognition”, Göttingen, Germany
| |
Collapse
|
30
|
Seth AK, Millidge B, Buckley CL, Tschantz A. Curious Inferences: Reply to Sun and Firestone on the Dark Room Problem. Trends Cogn Sci 2020; 24:681-683. [DOI: 10.1016/j.tics.2020.05.011] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2020] [Accepted: 05/22/2020] [Indexed: 10/23/2022]
|
31
|
Zhuang X, Yang Z, Cordes D. A technical review of canonical correlation analysis for neuroscience applications. Hum Brain Mapp 2020; 41:3807-3833. [PMID: 32592530 PMCID: PMC7416047 DOI: 10.1002/hbm.25090] [Citation(s) in RCA: 70] [Impact Index Per Article: 17.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2020] [Accepted: 05/23/2020] [Indexed: 12/11/2022] Open
Abstract
Collecting comprehensive data sets of the same subject has become a standard in neuroscience research and uncovering multivariate relationships among collected data sets have gained significant attentions in recent years. Canonical correlation analysis (CCA) is one of the powerful multivariate tools to jointly investigate relationships among multiple data sets, which can uncover disease or environmental effects in various modalities simultaneously and characterize changes during development, aging, and disease progressions comprehensively. In the past 10 years, despite an increasing number of studies have utilized CCA in multivariate analysis, simple conventional CCA dominates these applications. Multiple CCA-variant techniques have been proposed to improve the model performance; however, the complicated multivariate formulations and not well-known capabilities have delayed their wide applications. Therefore, in this study, a comprehensive review of CCA and its variant techniques is provided. Detailed technical formulation with analytical and numerical solutions, current applications in neuroscience research, and advantages and limitations of each CCA-related technique are discussed. Finally, a general guideline in how to select the most appropriate CCA-related technique based on the properties of available data sets and particularly targeted neuroscience questions is provided.
Collapse
Affiliation(s)
- Xiaowei Zhuang
- Cleveland Clinic Lou Ruvo Center for Brain HealthLas VegasNevadaUSA
| | - Zhengshi Yang
- Cleveland Clinic Lou Ruvo Center for Brain HealthLas VegasNevadaUSA
| | - Dietmar Cordes
- Cleveland Clinic Lou Ruvo Center for Brain HealthLas VegasNevadaUSA
- University of ColoradoBoulderColoradoUSA
- Department of Brain HealthUniversity of NevadaLas VegasNevadaUSA
| |
Collapse
|
32
|
Daucé E, Albiges P, Perrinet LU. A dual foveal-peripheral visual processing model implements efficient saccade selection. J Vis 2020; 20:22. [PMID: 38755789 PMCID: PMC7443118 DOI: 10.1167/jov.20.8.22] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2020] [Accepted: 05/07/2020] [Indexed: 11/24/2022] Open
Abstract
We develop a visuomotor model that implements visual search as a focal accuracy-seeking policy, with the target's position and category drawn independently from a common generative process. Consistently with the anatomical separation between the ventral versus dorsal pathways, the model is composed of two pathways that respectively infer what to see and where to look. The "What" network is a classical deep learning classifier that only processes a small region around the center of fixation, providing a "foveal" accuracy. In contrast, the "Where" network processes the full visual field in a biomimetic fashion, using a log-polar retinotopic encoding, which is preserved up to the action selection level. In our model, the foveal accuracy is used as a monitoring signal to train the "Where" network, much like in the "actor/critic" framework. After training, the "Where" network provides an "accuracy map" that serves to guide the eye toward peripheral objects. Finally, the comparison of both networks' accuracies amounts to either selecting a saccade or keeping the eye focused at the center to identify the target. We test this setup on a simple task of finding a digit in a large, cluttered image. Our simulation results demonstrate the effectiveness of this approach, increasing by one order of magnitude the radius of the visual field toward which the agent can detect and recognize a target, either through a single saccade or with multiple ones. Importantly, our log-polar treatment of the visual information exploits the strong compression rate performed at the sensory level, providing ways to implement visual search in a sublinear fashion, in contrast with mainstream computer vision.
Collapse
Affiliation(s)
- Emmanuel Daucé
- Institut de Neurosciences de la Timone (UMR 7289), Aix Marseille University, CNRS, Marseille, France
| | - Pierre Albiges
- Institut de Neurosciences de la Timone (UMR 7289), Aix Marseille University, CNRS, Marseille, France
| | - Laurent U Perrinet
- Institut de Neurosciences de la Timone (UMR 7289), Aix Marseille University, CNRS, Marseille, France
- https://laurentperrinet.github.io/
| |
Collapse
|
33
|
Smith R, Schwartenbeck P, Parr T, Friston KJ. An Active Inference Approach to Modeling Structure Learning: Concept Learning as an Example Case. Front Comput Neurosci 2020; 14:41. [PMID: 32508611 PMCID: PMC7250191 DOI: 10.3389/fncom.2020.00041] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2019] [Accepted: 04/17/2020] [Indexed: 11/13/2022] Open
Abstract
Within computational neuroscience, the algorithmic and neural basis of structure learning remains poorly understood. Concept learning is one primary example, which requires both a type of internal model expansion process (adding novel hidden states that explain new observations), and a model reduction process (merging different states into one underlying cause and thus reducing model complexity via meta-learning). Although various algorithmic models of concept learning have been proposed within machine learning and cognitive science, many are limited to various degrees by an inability to generalize, the need for very large amounts of training data, and/or insufficiently established biological plausibility. Using concept learning as an example case, we introduce a novel approach for modeling structure learning-and specifically state-space expansion and reduction-within the active inference framework and its accompanying neural process theory. Our aim is to demonstrate its potential to facilitate a novel line of active inference research in this area. The approach we lay out is based on the idea that a generative model can be equipped with extra (hidden state or cause) "slots" that can be engaged when an agent learns about novel concepts. This can be combined with a Bayesian model reduction process, in which any concept learning-associated with these slots-can be reset in favor of a simpler model with higher model evidence. We use simulations to illustrate this model's ability to add new concepts to its state space (with relatively few observations) and increase the granularity of the concepts it currently possesses. We also simulate the predicted neural basis of these processes. We further show that it can accomplish a simple form of "one-shot" generalization to new stimuli. Although deliberately simple, these simulation results highlight ways in which active inference could offer useful resources in developing neurocomputational models of structure learning. They provide a template for how future active inference research could apply this approach to real-world structure learning problems and assess the added utility it may offer.
Collapse
Affiliation(s)
- Ryan Smith
- Laureate Institute for Brain Research, Tulsa, OK, United States
| | - Philipp Schwartenbeck
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, United Kingdom
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, United Kingdom
| | - Karl J. Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, United Kingdom
| |
Collapse
|
34
|
Adams RA, Moutoussis M, Nour MM, Dahoun T, Lewis D, Illingworth B, Veronese M, Mathys C, de Boer L, Guitart-Masip M, Friston KJ, Howes OD, Roiser JP. Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models. Cereb Cortex 2020; 30:3573-3589. [PMID: 32083297 PMCID: PMC7233027 DOI: 10.1093/cercor/bhz327] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 11/18/2019] [Accepted: 12/05/2019] [Indexed: 12/17/2022] Open
Abstract
Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans.
Collapse
Affiliation(s)
- Rick A Adams
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
- Division of Psychiatry, University College London, London W1T 7NF, UK
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
| | - Michael Moutoussis
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
| | - Matthew M Nour
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King’s College London, London SE5 8AF, UK
| | - Tarik Dahoun
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford OX3 7JX, UK
| | - Declan Lewis
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| | - Benjamin Illingworth
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| | - Mattia Veronese
- Centre for Neuroimaging Sciences, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King's College London, London SE5 8AF, UK
| | - Christoph Mathys
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
- Scuola Internazionale Superiore di Studi Avanzati (SISSA), 34136 Trieste, Italy
- Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich and ETH Zurich, 8032 Zurich, Switzerland
| | - Lieke de Boer
- Aging Research Center, Karolinska Institute, 171 65 Stockholm, Sweden
| | - Marc Guitart-Masip
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
- Aging Research Center, Karolinska Institute, 171 65 Stockholm, Sweden
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK
| | - Oliver D Howes
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King’s College London, London SE5 8AF, UK
| | - Jonathan P Roiser
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| |
Collapse
|
35
|
Parr T. Inferring What to Do (And What Not to). ENTROPY (BASEL, SWITZERLAND) 2020; 22:E536. [PMID: 33286308 PMCID: PMC7517030 DOI: 10.3390/e22050536] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/17/2020] [Revised: 05/04/2020] [Accepted: 05/07/2020] [Indexed: 12/18/2022]
Abstract
In recent years, the "planning as inference" paradigm has become central to the study of behaviour. The advance offered by this is the formalisation of motivation as a prior belief about "how I am going to act". This paper provides an overview of the factors that contribute to this prior. These are rooted in optimal experimental design, information theory, and statistical decision making. We unpack how these factors imply a functional architecture for motivated behaviour. This raises an important question: how can we put this architecture to work in the service of understanding observed neurobiological structure? To answer this question, we draw from established techniques in experimental studies of behaviour. Typically, these examine the influence of perturbations of the nervous system-which include pathological insults or optogenetic manipulations-to see their influence on behaviour. Here, we argue that the message passing that emerges from inferring what to do can be similarly perturbed. If a given perturbation elicits the same behaviours as a focal brain lesion, this provides a functional interpretation of empirical findings and an anatomical grounding for theoretical results. We highlight examples of this approach that influence different sorts of goal-directed behaviour, active learning, and decision making. Finally, we summarise their implications for the neuroanatomy of inferring what to do (and what not to).
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, University College London, 12 Queen Square, London WC1N 3BG, UK
| |
Collapse
|
36
|
Linson A, Parr T, Friston KJ. Active inference, stressors, and psychological trauma: A neuroethological model of (mal)adaptive explore-exploit dynamics in ecological context. Behav Brain Res 2020; 380:112421. [PMID: 31830495 PMCID: PMC6961115 DOI: 10.1016/j.bbr.2019.112421] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2019] [Revised: 12/06/2019] [Accepted: 12/07/2019] [Indexed: 12/28/2022]
Abstract
This paper offers a formal account of emotional inference and stress-related behaviour, using the notion of active inference. We formulate responses to stressful scenarios in terms of Bayesian belief-updating and subsequent policy selection; namely, planning as (active) inference. Using a minimal model of how creatures or subjects account for their sensations (and subsequent action), we deconstruct the sequences of belief updating and behaviour that underwrite stress-related responses - and simulate the aberrant responses of the sort seen in post-traumatic stress disorder (PTSD). Crucially, the model used for belief-updating generates predictions in multiple (exteroceptive, proprioceptive and interoceptive) modalities, to provide an integrated account of evidence accumulation and multimodal integration that has consequences for both motor and autonomic responses. The ensuing phenomenology speaks to many constructs in the ecological and clinical literature on stress, which we unpack with reference to simulated inference processes and accompanying neuronal responses. A key insight afforded by this formal approach rests on the trade-off between the epistemic affordance of certain cues (that resolve uncertainty about states of affairs in the environment) and the consequences of epistemic foraging (that may be in conflict with the instrumental or pragmatic value of 'fleeing' or 'freezing'). Starting from first principles, we show how this trade-off is nuanced by prior (subpersonal) beliefs about the outcomes of behaviour - beliefs that, when held with unduly high precision, can lead to (Bayes optimal) responses that closely resemble PTSD.
Collapse
Affiliation(s)
- Adam Linson
- Faculty of Natural Sciences, University of Stirling, Stirling, UK; Faculty of Arts and Humanities, University of Stirling, Stirling, UK.
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, UK
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, UK
| |
Collapse
|
37
|
Abstract
Eye Movement Desensitization and Reprocessing Therapy (EMDR) is an effective treatment for Post-traumatic Stress Disorder (PTSD). The Adaptive Information Processing Model (AIP) guides the development and practice of EMDR. The AIP postulates inadequately processed memory as the foundation of PTSD pathology. Predictive Processing postulates that the primary function of the brain is prediction that serves to anticipate the next moment of experience in order to resist the dissipative force of entropy thus facilitating continued survival. Memory is the primary substrate of prediction, and is optimized by an ongoing process of precision weighted prediction error minimization that refines prediction by updating the memories on which it is based. The Predictive Processing model of EMDR postulates that EMDR facilitates the predictive processing of traumatic memory by overcoming the bias against exploration and evidence accumulation. The EMDR protocol brings the traumatic memory into an active state of re-experiencing. Defensive responding and/or low sensory precision preclude evidence accumulation to test the predictions of the traumatic memory in the present. Sets of therapist guided eye movements repeatedly challenge the bias against evidence accumulation and compel sensory sampling of the benign present. Eye movements reset the theta rhythm organizing the flow of information through the brain, facilitating the deployment of both overt and covert attention, and the mnemonic search for associations. Sampling of sensation does not support the predictions of the traumatic memory resulting in prediction error that the brain then attempts to minimize. The net result is a restoration of the integrity of the rhythmic deployment of attention, a recalibration of sensory precision, and the updating (reconsolidation) of the traumatic memory. Thus one prediction of the model is a decrease in Attention Bias Variability, a core dysfunction in PTSD, following successful treatment with EMDR.
Collapse
|
38
|
Mirza MB, Adams RA, Friston K, Parr T. Introducing a Bayesian model of selective attention based on active inference. Sci Rep 2019; 9:13915. [PMID: 31558746 PMCID: PMC6763492 DOI: 10.1038/s41598-019-50138-8] [Citation(s) in RCA: 32] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2019] [Accepted: 08/28/2019] [Indexed: 11/26/2022] Open
Abstract
Information gathering comprises actions whose (sensory) consequences resolve uncertainty (i.e., are salient). In other words, actions that solicit salient information cause the greatest shift in beliefs (i.e., information gain) about the causes of our sensations. However, not all information is relevant to the task at hand: this is especially the case in complex, naturalistic scenes. This paper introduces a formal model of selective attention based on active inference and contextual epistemic foraging. We consider a visual search task with a special emphasis on goal-directed and task-relevant exploration. In this scheme, attention modulates the expected fidelity (precision) of the mapping between observations and hidden states in a state-dependent or context-sensitive manner. This ensures task-irrelevant observations have little expected information gain, and so the agent - driven to reduce expected surprise (i.e., uncertainty) - does not actively seek them out. Instead, it selectively samples task-relevant observations, which inform (task-relevant) hidden states. We further show, through simulations, that the atypical exploratory behaviours in conditions such as autism and anxiety may be due to a failure to appropriately modulate sensory precision in a context-specific way.
Collapse
Affiliation(s)
- M Berk Mirza
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, UK.
- Department of Neuroimaging, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK.
- The NIHR Maudsley Biomedical Research Centre (BRC) at South London and Maudsley NHS Foundation Trust and the Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK.
| | - Rick A Adams
- Institute of Cognitive Neuroscience, 17 Queen Square, University College London, London, UK
- Division of Psychiatry, 149 Tottenham Court Road, University College London, London, UK
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, 10-12 Russell Square, London, WC1B 5EH, UK
- Department of Computer Science, University College London, Malet Place, London, WC1E 7JE, UK
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, UK
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, UK
| |
Collapse
|
39
|
Parr T, Corcoran AW, Friston KJ, Hohwy J. Perceptual awareness and active inference. Neurosci Conscious 2019; 2019:niz012. [PMID: 31528360 PMCID: PMC6734140 DOI: 10.1093/nc/niz012] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2019] [Revised: 05/24/2019] [Accepted: 07/28/2019] [Indexed: 12/16/2022] Open
Abstract
Perceptual awareness depends upon the way in which we engage with our sensorium. This notion is central to active inference, a theoretical framework that treats perception and action as inferential processes. This variational perspective on cognition formalizes the notion of perception as hypothesis testing and treats actions as experiments that are designed (in part) to gather evidence for or against alternative hypotheses. The common treatment of perception and action affords a useful interpretation of certain perceptual phenomena whose active component is often not acknowledged. In this article, we start by considering Troxler fading - the dissipation of a peripheral percept during maintenance of fixation, and its recovery during free (saccadic) exploration. This offers an important example of the failure to maintain a percept without actively interrogating a visual scene. We argue that this may be understood in terms of the accumulation of uncertainty about a hypothesized stimulus when free exploration is disrupted by experimental instructions or pathology. Once we take this view, we can generalize the idea of using bodily (oculomotor) action to resolve uncertainty to include the use of mental (attentional) actions for the same purpose. This affords a useful way to think about binocular rivalry paradigms, in which perceptual changes need not be associated with an overt movement.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, Institute of Neurology, 12 Queen Square, London, UK
| | - Andrew W Corcoran
- Cognition & Philosophy Laboratory, Department of Philosophy, Monash University, Melbourne, Australia
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, Institute of Neurology, 12 Queen Square, London, UK
| | - Jakob Hohwy
- Cognition & Philosophy Laboratory, Department of Philosophy, Monash University, Melbourne, Australia
| |
Collapse
|
40
|
Abstract
Many physiological and pathological changes in brain function manifest in eye-movement control. As such, assessment of oculomotion is an invaluable part of a clinical examination and affords a non-invasive window on several key aspects of neuronal computation. While oculomotion is often used to detect deficits of the sort associated with vascular or neoplastic events; subtler (e.g. pharmacological) effects on neuronal processing also induce oculomotor changes. We have previously framed oculomotor control as part of active vision, namely, a process of inference comprising two distinct but related challenges. The first is inferring where to look, and the second is inferring how to implement the selected action. In this paper, we draw from recent theoretical work on the neuromodulatory control of active inference. This allows us to simulate the sort of changes we would expect in oculomotor behaviour, following pharmacological enhancement or suppression of key neuromodulators-in terms of deciding where to look and the ensuing trajectory of the eye movement itself. We focus upon the influence of cholinergic and GABAergic agents on the speed of saccades, and consider dopaminergic and noradrenergic effects on more complex, memory-guided, behaviour. In principle, a computational approach to understanding the relationship between pharmacology and oculomotor behaviour affords the opportunity to estimate the influence of a given pharmaceutical upon neuronal function, and to use this to optimise therapeutic interventions on an individual basis.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, 12 Queen Square, London, WC1N 3BG UK
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, 12 Queen Square, London, WC1N 3BG UK
| |
Collapse
|
41
|
Dynamic Causal Modelling of Active Vision. J Neurosci 2019; 39:6265-6275. [PMID: 31182633 PMCID: PMC6687902 DOI: 10.1523/jneurosci.2459-18.2019] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2018] [Revised: 03/08/2019] [Accepted: 03/11/2019] [Indexed: 11/27/2022] Open
Abstract
In this paper, we draw from recent theoretical work on active perception, which suggests that the brain makes use of an internal (i.e., generative) model to make inferences about the causes of sensations. This view treats visual sensations as consequent on action (i.e., saccades) and implies that visual percepts must be actively constructed via a sequence of eye movements. Oculomotor control calls on a distributed set of brain sources that includes the dorsal and ventral frontoparietal (attention) networks. We argue that connections from the frontal eye fields to ventral parietal sources represent the mapping from “where”, fixation location to information derived from “what” representations in the ventral visual stream. During scene construction, this mapping must be learned, putatively through changes in the effective connectivity of these synapses. Here, we test the hypothesis that the coupling between the dorsal frontal cortex and the right temporoparietal cortex is modulated during saccadic interrogation of a simple visual scene. Using dynamic causal modeling for magnetoencephalography with (male and female) human participants, we assess the evidence for changes in effective connectivity by comparing models that allow for this modulation with models that do not. We find strong evidence for modulation of connections between the two attention networks; namely, a disinhibition of the ventral network by its dorsal counterpart. SIGNIFICANCE STATEMENT This work draws from recent theoretical accounts of active vision and provides empirical evidence for changes in synaptic efficacy consistent with these computational models. In brief, we used magnetoencephalography in combination with eye-tracking to assess the neural correlates of a form of short-term memory during a dot cancellation task. Using dynamic causal modeling to quantify changes in effective connectivity, we found evidence that the coupling between the dorsal and ventral attention networks changed during the saccadic interrogation of a simple visual scene. Intuitively, this is consistent with the idea that these neuronal connections may encode beliefs about “what I would see if I looked there”, and that this mapping is optimized as new data are obtained with each fixation.
Collapse
|
42
|
Parr T, Markovic D, Kiebel SJ, Friston KJ. Neuronal message passing using Mean-field, Bethe, and Marginal approximations. Sci Rep 2019; 9:1889. [PMID: 30760782 PMCID: PMC6374414 DOI: 10.1038/s41598-018-38246-3] [Citation(s) in RCA: 55] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2018] [Accepted: 12/19/2018] [Indexed: 01/08/2023] Open
Abstract
Neuronal computations rely upon local interactions across synapses. For a neuronal network to perform inference, it must integrate information from locally computed messages that are propagated among elements of that network. We review the form of two popular (Bayesian) message passing schemes and consider their plausibility as descriptions of inference in biological networks. These are variational message passing and belief propagation - each of which is derived from a free energy functional that relies upon different approximations (mean-field and Bethe respectively). We begin with an overview of these schemes and illustrate the form of the messages required to perform inference using Hidden Markov Models as generative models. Throughout, we use factor graphs to show the form of the generative models and of the messages they entail. We consider how these messages might manifest neuronally and simulate the inferences they perform. While variational message passing offers a simple and neuronally plausible architecture, it falls short of the inferential performance of belief propagation. In contrast, belief propagation allows exact computation of marginal posteriors at the expense of the architectural simplicity of variational message passing. As a compromise between these two extremes, we offer a third approach - marginal message passing - that features a simple architecture, while approximating the performance of belief propagation. Finally, we link formal considerations to accounts of neurological and psychiatric syndromes in terms of aberrant message passing.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, WC1N 3BG, UK.
| | - Dimitrije Markovic
- Chair of Neuroimaging, Psychology Department, Technische Universität Dresden, Dresden, Germany
| | - Stefan J Kiebel
- Chair of Neuroimaging, Psychology Department, Technische Universität Dresden, Dresden, Germany
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, WC1N 3BG, UK
| |
Collapse
|
43
|
Parr T, Friston KJ. The Anatomy of Inference: Generative Models and Brain Structure. Front Comput Neurosci 2018; 12:90. [PMID: 30483088 PMCID: PMC6243103 DOI: 10.3389/fncom.2018.00090] [Citation(s) in RCA: 81] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2018] [Accepted: 10/25/2018] [Indexed: 01/02/2023] Open
Abstract
To infer the causes of its sensations, the brain must call on a generative (predictive) model. This necessitates passing local messages between populations of neurons to update beliefs about hidden variables in the world beyond its sensory samples. It also entails inferences about how we will act. Active inference is a principled framework that frames perception and action as approximate Bayesian inference. This has been successful in accounting for a wide range of physiological and behavioral phenomena. Recently, a process theory has emerged that attempts to relate inferences to their neurobiological substrates. In this paper, we review and develop the anatomical aspects of this process theory. We argue that the form of the generative models required for inference constrains the way in which brain regions connect to one another. Specifically, neuronal populations representing beliefs about a variable must receive input from populations representing the Markov blanket of that variable. We illustrate this idea in four different domains: perception, planning, attention, and movement. In doing so, we attempt to show how appealing to generative models enables us to account for anatomical brain architectures. Ultimately, committing to an anatomical theory of inference ensures we can form empirical hypotheses that can be tested using neuroimaging, neuropsychological, and electrophysiological experiments.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London, United Kingdom
| | | |
Collapse
|
44
|
Abstract
This paper characterizes impulsive behavior using a patch-leaving paradigm and active inference-a framework for describing Bayes optimal behavior. This paradigm comprises different environments (patches) with limited resources that decline over time at different rates. The challenge is to decide when to leave the current patch for another to maximize reward. We chose this task because it offers an operational characterization of impulsive behavior, namely, maximizing proximal reward at the expense of future gain. We use a Markov decision process formulation of active inference to simulate behavioral and electrophysiological responses under different models and prior beliefs. Our main finding is that there are at least three distinct causes of impulsive behavior, which we demonstrate by manipulating three different components of the Markov decision process model. These components comprise (i) the depth of planning, (ii) the capacity to maintain and process information, and (iii) the perceived value of immediate (relative to delayed) rewards. We show how these manipulations change beliefs and subsequent choices through variational message passing. Furthermore, we appeal to the process theories associated with this message passing to simulate neuronal correlates. In future work, we will use this scheme to identify the prior beliefs that underlie different sorts of impulsive behavior-and ask whether different causes of impulsivity can be inferred from the electrophysiological correlates of choice behavior.
Collapse
|
45
|
Parr T, Benrimoh DA, Vincent P, Friston KJ. Precision and False Perceptual Inference. Front Integr Neurosci 2018; 12:39. [PMID: 30294264 PMCID: PMC6158318 DOI: 10.3389/fnint.2018.00039] [Citation(s) in RCA: 40] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2018] [Accepted: 08/30/2018] [Indexed: 12/24/2022] Open
Abstract
Accurate perceptual inference fundamentally depends upon accurate beliefs about the reliability of sensory data. In this paper, we describe a Bayes optimal and biologically plausible scheme that refines these beliefs through a gradient descent on variational free energy. To illustrate this, we simulate belief updating during visual foraging and show that changes in estimated sensory precision (i.e., confidence in visual data) are highly sensitive to prior beliefs about the contents of a visual scene. In brief, confident prior beliefs induce an increase in estimated precision when consistent with sensory evidence, but a decrease when they conflict. Prior beliefs held with low confidence are rapidly updated to posterior beliefs, determined by sensory data. These induce much smaller changes in beliefs about sensory precision. We argue that pathologies of scene construction may be due to abnormal priors, and show that these can induce a reduction in estimated sensory precision. Having previously associated this precision with cholinergic signaling, we note that several neurodegenerative conditions are associated with visual disturbances and cholinergic deficits; notably, the synucleinopathies. On relating the message passing in our model to the functional anatomy of the ventral visual stream, we find that simulated neuronal loss in temporal lobe regions induces confident, inaccurate, empirical prior beliefs at lower levels in the visual hierarchy. This provides a plausible, if speculative, computational mechanism for the loss of cholinergic signaling and the visual disturbances associated with temporal lobe Lewy body pathology. This may be seen as an illustration of the sorts of hypotheses that may be expressed within this computational framework.
Collapse
Affiliation(s)
- Thomas Parr
- Institute of Neurology, Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
| | - David A Benrimoh
- Institute of Neurology, Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
| | - Peter Vincent
- Institute of Neurology, Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
| | - Karl J Friston
- Institute of Neurology, Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
| |
Collapse
|
46
|
Parr T, Friston KJ. The Discrete and Continuous Brain: From Decisions to Movement-And Back Again. Neural Comput 2018. [PMID: 29894658 DOI: 10.1162/neco˙a˙01102] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
To act upon the world, creatures must change continuous variables such as muscle length or chemical concentration. In contrast, decision making is an inherently discrete process, involving the selection among alternative courses of action. In this article, we consider the interface between the discrete and continuous processes that translate our decisions into movement in a Newtonian world-and how movement informs our decisions. We do so by appealing to active inference, with a special focus on the oculomotor system. Within this exemplar system, we argue that the superior colliculus is well placed to act as a discrete-continuous interface. Interestingly, when the neuronal computations within the superior colliculus are formulated in terms of active inference, we find that many aspects of its neuroanatomy emerge from the computations it must perform in this role.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, WC1N 3BG, U.K.
| | - Karl J Friston
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, WC1N 3BG, U.K.
| |
Collapse
|
47
|
Parr T, Friston KJ. The Discrete and Continuous Brain: From Decisions to Movement-And Back Again. Neural Comput 2018; 30:2319-2347. [PMID: 29894658 PMCID: PMC6115199 DOI: 10.1162/neco_a_01102] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
To act upon the world, creatures must change continuous variables such as muscle length or chemical concentration. In contrast, decision making is an inherently discrete process, involving the selection among alternative courses of action. In this article, we consider the interface between the discrete and continuous processes that translate our decisions into movement in a Newtonian world—and how movement informs our decisions. We do so by appealing to active inference, with a special focus on the oculomotor system. Within this exemplar system, we argue that the superior colliculus is well placed to act as a discrete-continuous interface. Interestingly, when the neuronal computations within the superior colliculus are formulated in terms of active inference, we find that many aspects of its neuroanatomy emerge from the computations it must perform in this role.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, WC1N 3BG, U.K.
| | - Karl J Friston
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, WC1N 3BG, U.K.
| |
Collapse
|
48
|
Franklin NT, Frank MJ. Compositional clustering in task structure learning. PLoS Comput Biol 2018; 14:e1006116. [PMID: 29672581 PMCID: PMC5929577 DOI: 10.1371/journal.pcbi.1006116] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2017] [Revised: 05/01/2018] [Accepted: 04/03/2018] [Indexed: 11/18/2022] Open
Abstract
Humans are remarkably adept at generalizing knowledge between experiences in a way that can be difficult for computers. Often, this entails generalizing constituent pieces of experiences that do not fully overlap, but nonetheless share useful similarities with, previously acquired knowledge. However, it is often unclear how knowledge gained in one context should generalize to another. Previous computational models and data suggest that rather than learning about each individual context, humans build latent abstract structures and learn to link these structures to arbitrary contexts, facilitating generalization. In these models, task structures that are more popular across contexts are more likely to be revisited in new contexts. However, these models can only re-use policies as a whole and are unable to transfer knowledge about the transition structure of the environment even if only the goal has changed (or vice-versa). This contrasts with ecological settings, where some aspects of task structure, such as the transition function, will be shared between context separately from other aspects, such as the reward function. Here, we develop a novel non-parametric Bayesian agent that forms independent latent clusters for transition and reward functions, affording separable transfer of their constituent parts across contexts. We show that the relative performance of this agent compared to an agent that jointly clusters reward and transition functions depends environmental task statistics: the mutual information between transition and reward functions and the stochasticity of the observations. We formalize our analysis through an information theoretic account of the priors, and propose a meta learning agent that dynamically arbitrates between strategies across task domains to optimize a statistical tradeoff.
Collapse
Affiliation(s)
- Nicholas T. Franklin
- Department of Psychology, Harvard University, Cambridge, Massachusetts, United States of America
- Department of Cognitive, Linguistic & Psychological Sciences, Brown University, Providence, Rhode Island, United States of America
| | - Michael J. Frank
- Department of Cognitive, Linguistic & Psychological Sciences, Brown University, Providence, Rhode Island, United States of America
- Brown Institute for Brain Science, Brown University, Providence, Rhode Island, United States of America
| |
Collapse
|
49
|
Abstract
Computational theories of brain function have become very influential in neuroscience. They have facilitated the growth of formal approaches to disease, particularly in psychiatric research. In this paper, we provide a narrative review of the body of computational research addressing neuropsychological syndromes, and focus on those that employ Bayesian frameworks. Bayesian approaches to understanding brain function formulate perception and action as inferential processes. These inferences combine ‘prior’ beliefs with a generative (predictive) model to explain the causes of sensations. Under this view, neuropsychological deficits can be thought of as false inferences that arise due to aberrant prior beliefs (that are poor fits to the real world). This draws upon the notion of a Bayes optimal pathology – optimal inference with suboptimal priors – and provides a means for computational phenotyping. In principle, any given neuropsychological disorder could be characterized by the set of prior beliefs that would make a patient’s behavior appear Bayes optimal. We start with an overview of some key theoretical constructs and use these to motivate a form of computational neuropsychology that relates anatomical structures in the brain to the computations they perform. Throughout, we draw upon computational accounts of neuropsychological syndromes. These are selected to emphasize the key features of a Bayesian approach, and the possible types of pathological prior that may be present. They range from visual neglect through hallucinations to autism. Through these illustrative examples, we review the use of Bayesian approaches to understand the link between biology and computation that is at the heart of neuropsychology.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, United Kingdom
| | - Geraint Rees
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, United Kingdom.,Institute of Cognitive Neuroscience, University College London, London, United Kingdom
| | - Karl J Friston
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, United Kingdom
| |
Collapse
|
50
|
Goh JOS, Hung HY, Su YS. A conceptual consideration of the free energy principle in cognitive maps: How cognitive maps help reduce surprise. PSYCHOLOGY OF LEARNING AND MOTIVATION 2018. [DOI: 10.1016/bs.plm.2018.09.005] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|