1
|
Zhang Z, Xu F. An Overview of the Free Energy Principle and Related Research. Neural Comput 2024; 36:963-1021. [PMID: 38457757 DOI: 10.1162/neco_a_01642] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Accepted: 11/20/2023] [Indexed: 03/10/2024]
Abstract
The free energy principle and its corollary, the active inference framework, serve as theoretical foundations in the domain of neuroscience, explaining the genesis of intelligent behavior. This principle states that the processes of perception, learning, and decision making-within an agent-are all driven by the objective of "minimizing free energy," evincing the following behaviors: learning and employing a generative model of the environment to interpret observations, thereby achieving perception, and selecting actions to maintain a stable preferred state and minimize the uncertainty about the environment, thereby achieving decision making. This fundamental principle can be used to explain how the brain processes perceptual information, learns about the environment, and selects actions. Two pivotal tenets are that the agent employs a generative model for perception and planning and that interaction with the world (and other agents) enhances the performance of the generative model and augments perception. With the evolution of control theory and deep learning tools, agents based on the FEP have been instantiated in various ways across different domains, guiding the design of a multitude of generative models and decision-making algorithms. This letter first introduces the basic concepts of the FEP, followed by its historical development and connections with other theories of intelligence, and then delves into the specific application of the FEP to perception and decision making, encompassing both low-dimensional simple situations and high-dimensional complex situations. It compares the FEP with model-based reinforcement learning to show that the FEP provides a better objective function. We illustrate this using numerical studies of Dreamer3 by adding expected information gain into the standard objective function. In a complementary fashion, existing reinforcement learning, and deep learning algorithms can also help implement the FEP-based agents. Finally, we discuss the various capabilities that agents need to possess in complex environments and state that the FEP can aid agents in acquiring these capabilities.
Collapse
Affiliation(s)
- Zhengquan Zhang
- Key Laboratory of Information Science of Electromagnetic Waves, Fudan University, Shanghai, P.R.C.
| | - Feng Xu
- Key Laboratory of Information Science of Electromagnetic Waves, Fudan University, Shanghai, P.R.C.
| |
Collapse
|
2
|
Liu XQ, Ji XY, Weng X, Zhang YF. Artificial intelligence ecosystem for computational psychiatry: Ideas to practice. World J Meta-Anal 2023; 11:79-91. [DOI: 10.13105/wjma.v11.i4.79] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/26/2022] [Revised: 03/18/2023] [Accepted: 04/04/2023] [Indexed: 04/14/2023] Open
Abstract
Computational psychiatry is an emerging field that not only explores the biological basis of mental illness but also considers the diagnoses and identifies the underlying mechanisms. One of the key strengths of computational psychiatry is that it may identify patterns in large datasets that are not easily identifiable. This may help researchers develop more effective treatments and interventions for mental health problems. This paper is a narrative review that reviews the literature and produces an artificial intelligence ecosystem for computational psychiatry. The artificial intelligence ecosystem for computational psychiatry includes data acquisition, preparation, modeling, application, and evaluation. This approach allows researchers to integrate data from a variety of sources, such as brain imaging, genetics, and behavioral experiments, to obtain a more complete understanding of mental health conditions. Through the process of data preprocessing, training, and testing, the data that are required for model building can be prepared. By using machine learning, neural networks, artificial intelligence, and other methods, researchers have been able to develop diagnostic tools that can accurately identify mental health conditions based on a patient’s symptoms and other factors. Despite the continuous development and breakthrough of computational psychiatry, it has not yet influenced routine clinical practice and still faces many challenges, such as data availability and quality, biological risks, equity, and data protection. As we move progress in this field, it is vital to ensure that computational psychiatry remains accessible and inclusive so that all researchers may contribute to this significant and exciting field.
Collapse
Affiliation(s)
- Xin-Qiao Liu
- School of Education, Tianjin University, Tianjin 300350, China
| | - Xin-Yu Ji
- School of Education, Tianjin University, Tianjin 300350, China
| | - Xing Weng
- Huzhou Educational Science & Research Center, Huzhou 313000, Zhejiang Province, China
| | - Yi-Fan Zhang
- School of Education, Tianjin University, Tianjin 300350, China
| |
Collapse
|
3
|
Champion T, Grześ M, Bowman H. Branching Time Active Inference with Bayesian Filtering. Neural Comput 2022; 34:2132-2144. [PMID: 36027722 DOI: 10.1162/neco_a_01529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Accepted: 05/26/2022] [Indexed: 11/04/2022]
Abstract
Branching time active inference is a framework proposing to look at planning as a form of Bayesian model expansion. Its root can be found in active inference, a neuroscientific framework widely used for brain modeling, as well as in Monte Carlo tree search, a method broadly applied in the reinforcement learning literature. Up to now, the inference of the latent variables was carried out by taking advantage of the flexibility offered by variational message passing, an iterative process that can be understood as sending messages along the edges of a factor graph. In this letter, we harness the efficiency of an alternative method for inference, Bayesian filtering, which does not require the iteration of the update equations until convergence of the variational free energy. Instead, this scheme alternates between two phases: integration of evidence and prediction of future states. Both phases can be performed efficiently, and this provides a forty times speedup over the state of the art.
Collapse
Affiliation(s)
| | - Marek Grześ
- University of Kent, School of Computing, Canterbury CT2 7NZ, U.K.
| | - Howard Bowman
- University of Birmingham, School of Psychology, Birmingham B15 2TT, U.K.,University of Kent, School of Computing, Canterbury CT2 7NZ, U.K.
| |
Collapse
|
4
|
Constant A, Clark A, Kirchhoff M, Friston KJ. Extended active inference: Constructing predictive cognition beyond skulls. MIND & LANGUAGE 2022; 37:373-394. [PMID: 35875359 PMCID: PMC9292365 DOI: 10.1111/mila.12330] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/10/2019] [Revised: 10/07/2019] [Accepted: 11/19/2019] [Indexed: 05/17/2023]
Abstract
Cognitive niche construction is the process whereby organisms create and maintain cause-effect models of their niche as guides for fitness influencing behavior. Extended mind theory claims that cognitive processes extend beyond the brain to include predictable states of the world. Active inference and predictive processing in cognitive science assume that organisms embody predictive (i.e., generative) models of the world optimized by standard cognitive functions (e.g., perception, action, learning). This paper presents an active inference formulation that views cognitive niche construction as a cognitive function aimed at optimizing organisms' generative models. We call that process of optimization extended active inference.
Collapse
Affiliation(s)
- Axel Constant
- Charles Perkins CentreThe University of SydneySydneyNew South WalesAustralia
- Culture, Mind, and Brain ProgramMcGill UniversityMontrealQuebecCanada
- Wellcome Centre for Human NeuroimagingUniversity College LondonLondonUK
| | - Andy Clark
- Department of PhilosophyThe University of SussexBrightonUK
- Department of InformaticsThe University of SussexBrightonUK
- Department of PhilosophyMacquarie UniversitySydneyNew South WalesAustralia
| | - Michael Kirchhoff
- Department of PhilosophyUniversity of WollongongWollongongNew South WalesAustralia
| | - Karl J. Friston
- Culture, Mind, and Brain ProgramMcGill UniversityMontrealQuebecCanada
| |
Collapse
|
5
|
Da Costa L, Lanillos P, Sajid N, Friston K, Khan S. How Active Inference Could Help Revolutionise Robotics. ENTROPY (BASEL, SWITZERLAND) 2022; 24:361. [PMID: 35327872 PMCID: PMC8946999 DOI: 10.3390/e24030361] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Revised: 02/24/2022] [Accepted: 02/28/2022] [Indexed: 02/05/2023]
Abstract
Recent advances in neuroscience have characterised brain function using mathematical formalisms and first principles that may be usefully applied elsewhere. In this paper, we explain how active inference-a well-known description of sentient behaviour from neuroscience-can be exploited in robotics. In short, active inference leverages the processes thought to underwrite human behaviour to build effective autonomous systems. These systems show state-of-the-art performance in several robotics settings; we highlight these and explain how this framework may be used to advance robotics.
Collapse
Affiliation(s)
- Lancelot Da Costa
- Department of Mathematics, Imperial College London, London SW7 2AZ, UK
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3AR, UK; (N.S.); (K.F.)
| | - Pablo Lanillos
- Department of Artificial Intelligence, Donders Institute for Brain, Cognition and Behavior, Radboud University, 6525 XZ Nijmegen, The Netherlands;
| | - Noor Sajid
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3AR, UK; (N.S.); (K.F.)
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3AR, UK; (N.S.); (K.F.)
| | - Shujhat Khan
- Milton Keynes Hospital, Oxford Deanery, Milton Keynes MK6 5LD, UK;
| |
Collapse
|
6
|
Koudahl MT, Kouw WM, de Vries B. On Epistemics in Expected Free Energy for Linear Gaussian State Space Models. ENTROPY 2021; 23:e23121565. [PMID: 34945871 PMCID: PMC8700494 DOI: 10.3390/e23121565] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Revised: 11/19/2021] [Accepted: 11/23/2021] [Indexed: 01/20/2023]
Abstract
Active Inference (AIF) is a framework that can be used both to describe information processing in naturally intelligent systems, such as the human brain, and to design synthetic intelligent systems (agents). In this paper we show that Expected Free Energy (EFE) minimisation, a core feature of the framework, does not lead to purposeful explorative behaviour in linear Gaussian dynamical systems. We provide a simple proof that, due to the specific construction used for the EFE, the terms responsible for the exploratory (epistemic) drive become constant in the case of linear Gaussian systems. This renders AIF equivalent to KL control. From a theoretical point of view this is an interesting result since it is generally assumed that EFE minimisation will always introduce an exploratory drive in AIF agents. While the full EFE objective does not lead to exploration in linear Gaussian dynamical systems, the principles of its construction can still be used to design objectives that include an epistemic drive. We provide an in-depth analysis of the mechanics behind the epistemic drive of AIF agents and show how to design objectives for linear Gaussian dynamical systems that do include an epistemic drive. Concretely, we show that focusing solely on epistemics and dispensing with goal-directed terms leads to a form of maximum entropy exploration that is heavily dependent on the type of control signals driving the system. Additive controls do not permit such exploration. From a practical point of view this is an important result since linear Gaussian dynamical systems with additive controls are an extensively used model class, encompassing for instance Linear Quadratic Gaussian controllers. On the other hand, linear Gaussian dynamical systems driven by multiplicative controls such as switching transition matrices do permit an exploratory drive.
Collapse
Affiliation(s)
- Magnus T. Koudahl
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands; (W.M.K.); (B.d.V.)
- Correspondence:
| | - Wouter M. Kouw
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands; (W.M.K.); (B.d.V.)
| | - Bert de Vries
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands; (W.M.K.); (B.d.V.)
- GN Hearing, JF Kennedylaan 2, 5612 AB Eindhoven, The Netherlands
| |
Collapse
|
7
|
Barros P, Bloem AC, Hootsmans IM, Opheij LM, Toebosch RHA, Barakova E, Sciutti A. You Were Always on My Mind: Introducing Chef's Hat and COPPER for Personalized Reinforcement Learning. Front Robot AI 2021; 8:669990. [PMID: 34336935 PMCID: PMC8323774 DOI: 10.3389/frobt.2021.669990] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2021] [Accepted: 06/10/2021] [Indexed: 11/13/2022] Open
Abstract
Reinforcement learning simulation environments pose an important experimental test bed and facilitate data collection for developing AI-based robot applications. Most of them, however, focus on single-agent tasks, which limits their application to the development of social agents. This study proposes the Chef's Hat simulation environment, which implements a multi-agent competitive card game that is a complete reproduction of the homonymous board game, designed to provoke competitive strategies in humans and emotional responses. The game was shown to be ideal for developing personalized reinforcement learning, in an online learning closed-loop scenario, as its state representation is extremely dynamic and directly related to each of the opponent's actions. To adapt current reinforcement learning agents to this scenario, we also developed the COmPetitive Prioritized Experience Replay (COPPER) algorithm. With the help of COPPER and the Chef's Hat simulation environment, we evaluated the following: (1) 12 experimental learning agents, trained via four different regimens (self-play, play against a naive baseline, PER, or COPPER) with three algorithms based on different state-of-the-art learning paradigms (PPO, DQN, and ACER), and two "dummy" baseline agents that take random actions, (2) the performance difference between COPPER and PER agents trained using the PPO algorithm and playing against different agents (PPO, DQN, and ACER) or all DQN agents, and (3) human performance when playing against two different collections of agents. Our experiments demonstrate that COPPER helps agents learn to adapt to different types of opponents, improving the performance when compared to off-line learning models. An additional contribution of the study is the formalization of the Chef's Hat competitive game and the implementation of the Chef's Hat Player Club, a collection of trained and assessed agents as an enabler for embedding human competitive strategies in social continual and competitive reinforcement learning.
Collapse
Affiliation(s)
- Pablo Barros
- Cognitive Architecture for Collaborative Technologies (CONTACT) Unit Istituto Italiano di Tecnologia, Genova, Italy
| | - Anne C Bloem
- Department of Industrial Design, University of Technology Eindhoven, Eindhoven, Netherlands
| | - Inge M Hootsmans
- Department of Industrial Design, University of Technology Eindhoven, Eindhoven, Netherlands
| | - Lena M Opheij
- Department of Industrial Design, University of Technology Eindhoven, Eindhoven, Netherlands
| | - Romain H A Toebosch
- Department of Industrial Design, University of Technology Eindhoven, Eindhoven, Netherlands
| | - Emilia Barakova
- Department of Industrial Design, University of Technology Eindhoven, Eindhoven, Netherlands
| | - Alessandra Sciutti
- Cognitive Architecture for Collaborative Technologies (CONTACT) Unit Istituto Italiano di Tecnologia, Genova, Italy
| |
Collapse
|
8
|
Constant A, Hesp C, Davey CG, Friston KJ, Badcock PB. Why Depressed Mood is Adaptive: A Numerical Proof of Principle for an Evolutionary Systems Theory of Depression. COMPUTATIONAL PSYCHIATRY (CAMBRIDGE, MASS.) 2021; 5:60-80. [PMID: 34113717 PMCID: PMC7610949 DOI: 10.5334/cpsy.70] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
We provide a proof of principle for an evolutionary systems theory (EST) of depression. This theory suggests that normative depressive symptoms counter socioenvironmental volatility by increasing interpersonal support via social signalling and that this response depends upon the encoding of uncertainty about social contingencies, which can be targeted by neuromodulatory antidepressants. We simulated agents that committed to a series of decisions in a social two-arm bandit task before and after social adversity, which precipitated depressive symptoms. Responses to social adversity were modelled under various combinations of social support and pharmacotherapy. The normative depressive phenotype responded positively to social support and simulated treatments with antidepressants. Attracting social support and administering antidepressants also alleviated anhedonia and social withdrawal, speaking to improvements in interpersonal relationships. These results support the EST of depression by demonstrating that following adversity, normative depressed mood preserved social inclusion with appropriate interpersonal support or pharmacotherapy.
Collapse
Affiliation(s)
- Axel Constant
- Charles Perkins Centre, The University of Sydney, AU; Culture, Mind, and Brain Program, McGill University, CA; Wellcome Trust Centre for Human Neuroimaging, University College London, UK
| | - Casper Hesp
- Wellcome Trust Centre for Human Neuroimaging, University College London, UK; Department of Developmental Psychology, University of Amsterdam, NL; Amsterdam Brain and Cognition Center, University of Amsterdam, NL; Institute for Advanced Study, University of Amsterdam, NL
| | - Christopher G Davey
- Centre for Youth Mental Health, The University of Melbourne, AU; Department of Psychiatry, The University of Melbourne, AU
| | - Karl J Friston
- Wellcome Trust Centre for Human Neuroimaging, University College London, UK
| | - Paul B Badcock
- Centre for Youth Mental Health, The University of Melbourne, AU; Department of Psychiatry, The University of Melbourne, AU; Orygen, AU
| |
Collapse
|
9
|
Da Costa L, Parr T, Sengupta B, Friston K. Neural Dynamics under Active Inference: Plausibility and Efficiency of Information Processing. ENTROPY (BASEL, SWITZERLAND) 2021; 23:454. [PMID: 33921298 PMCID: PMC8069154 DOI: 10.3390/e23040454] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/16/2021] [Accepted: 04/06/2021] [Indexed: 02/07/2023]
Abstract
Active inference is a normative framework for explaining behaviour under the free energy principle-a theory of self-organisation originating in neuroscience. It specifies neuronal dynamics for state-estimation in terms of a descent on (variational) free energy-a measure of the fit between an internal (generative) model and sensory observations. The free energy gradient is a prediction error-plausibly encoded in the average membrane potentials of neuronal populations. Conversely, the expected probability of a state can be expressed in terms of neuronal firing rates. We show that this is consistent with current models of neuronal dynamics and establish face validity by synthesising plausible electrophysiological responses. We then show that these neuronal dynamics approximate natural gradient descent, a well-known optimisation algorithm from information geometry that follows the steepest descent of the objective in information space. We compare the information length of belief updating in both schemes, a measure of the distance travelled in information space that has a direct interpretation in terms of metabolic cost. We show that neural dynamics under active inference are metabolically efficient and suggest that neural representations in biological agents may evolve by approximating steepest descent in information space towards the point of optimal inference.
Collapse
Affiliation(s)
- Lancelot Da Costa
- Department of Mathematics, Imperial College London, London SW7 2AZ, UK
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK; (T.P.); (B.S.); (K.F.)
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK; (T.P.); (B.S.); (K.F.)
| | - Biswa Sengupta
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK; (T.P.); (B.S.); (K.F.)
- Core Machine Learning Group, Zebra AI, London WC2H 8TJ, UK
- Department of Bioengineering, Imperial College London, London SW7 2AZ, UK
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK; (T.P.); (B.S.); (K.F.)
| |
Collapse
|
10
|
Abstract
Active inference is a first principle account of how autonomous agents operate in dynamic, nonstationary environments. This problem is also considered in reinforcement learning, but limited work exists on comparing the two approaches on the same discrete-state environments. In this letter, we provide (1) an accessible overview of the discrete-state formulation of active inference, highlighting natural behaviors in active inference that are generally engineered in reinforcement learning, and (2) an explicit discrete-state comparison between active inference and reinforcement learning on an OpenAI gym baseline. We begin by providing a condensed overview of the active inference literature, in particular viewing the various natural behaviors of active inference agents through the lens of reinforcement learning. We show that by operating in a pure belief-based setting, active inference agents can carry out epistemic exploration-and account for uncertainty about their environment-in a Bayes-optimal fashion. Furthermore, we show that the reliance on an explicit reward signal in reinforcement learning is removed in active inference, where reward can simply be treated as another observation we have a preference over; even in the total absence of rewards, agent behaviors are learned through preference learning. We make these properties explicit by showing two scenarios in which active inference agents can infer behaviors in reward-free environments compared to both Q-learning and Bayesian model-based reinforcement learning agents and by placing zero prior preferences over rewards and learning the prior preferences over the observations corresponding to reward. We conclude by noting that this formalism can be applied to more complex settings (e.g., robotic arm movement, Atari games) if appropriate generative models can be formulated. In short, we aim to demystify the behavior of active inference agents by presenting an accessible discrete state-space and time formulation and demonstrate these behaviors in a OpenAI gym environment, alongside reinforcement learning agents.
Collapse
Affiliation(s)
- Noor Sajid
- Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, London, WC1N 3AR, U.K.
| | - Philip J Ball
- Machine Learning Research Group, Department of Engineering Science, University of Oxford, Oxford OX1 3PJ, U.K.
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, London, WC1N 3AR, U.K.
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, London, WC1N 3AR, U.K.
| |
Collapse
|
11
|
Da Costa L, Parr T, Sajid N, Veselic S, Neacsu V, Friston K. Active inference on discrete state-spaces: A synthesis. JOURNAL OF MATHEMATICAL PSYCHOLOGY 2020; 99:102447. [PMID: 33343039 PMCID: PMC7732703 DOI: 10.1016/j.jmp.2020.102447] [Citation(s) in RCA: 73] [Impact Index Per Article: 18.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/17/2020] [Revised: 07/23/2020] [Accepted: 09/03/2020] [Indexed: 05/05/2023]
Abstract
Active inference is a normative principle underwriting perception, action, planning, decision-making and learning in biological or artificial agents. From its inception, its associated process theory has grown to incorporate complex generative models, enabling simulation of a wide range of complex behaviours. Due to successive developments in active inference, it is often difficult to see how its underlying principle relates to process theories and practical implementation. In this paper, we try to bridge this gap by providing a complete mathematical synthesis of active inference on discrete state-space models. This technical summary provides an overview of the theory, derives neuronal dynamics from first principles and relates this dynamics to biological processes. Furthermore, this paper provides a fundamental building block needed to understand active inference for mixed generative models; allowing continuous sensations to inform discrete representations. This paper may be used as follows: to guide research towards outstanding challenges, a practical guide on how to implement active inference to simulate experimental behaviour, or a pointer towards various in-silico neurophysiological responses that may be used to make empirical predictions.
Collapse
Affiliation(s)
- Lancelot Da Costa
- Department of Mathematics, Imperial College London, London, SW7 2RH, United Kingdom
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom
| | - Noor Sajid
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom
| | - Sebastijan Veselic
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom
| | - Victorita Neacsu
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom
| |
Collapse
|
12
|
Demekas D, Parr T, Friston KJ. An Investigation of the Free Energy Principle for Emotion Recognition. Front Comput Neurosci 2020; 14:30. [PMID: 32390817 PMCID: PMC7189749 DOI: 10.3389/fncom.2020.00030] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2019] [Accepted: 03/23/2020] [Indexed: 01/23/2023] Open
Abstract
This paper offers a prospectus of what might be achievable in the development of emotional recognition devices. It provides a conceptual overview of the free energy principle; including Markov blankets, active inference, and-in particular-a discussion of selfhood and theory of mind, followed by a brief explanation of how these concepts can explain both neural and cultural models of emotional inference. The underlying hypothesis is that emotion recognition and inference devices will evolve from state-of-the-art deep learning models into active inference schemes that go beyond marketing applications and become adjunct to psychiatric practice. Specifically, this paper proposes that a second wave of emotion recognition devices will be equipped with an emotional lexicon (or the ability to epistemically search for one), allowing the device to resolve uncertainty about emotional states by actively eliciting responses from the user and learning from these responses. Following this, a third wave of emotional devices will converge upon the user's generative model, resulting in the machine and human engaging in a reciprocal, prosocial emotional interaction, i.e., sharing a generative model of emotional states.
Collapse
Affiliation(s)
- Daphne Demekas
- Department of Mathematics, University College London, London, United Kingdom
| | - Thomas Parr
- Department of Mathematics, University College London, London, United Kingdom
| | - Karl J. Friston
- Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
| |
Collapse
|
13
|
Deserno L, Boehme R, Mathys C, Katthagen T, Kaminski J, Stephan KE, Heinz A, Schlagenhauf F. Volatility Estimates Increase Choice Switching and Relate to Prefrontal Activity in Schizophrenia. BIOLOGICAL PSYCHIATRY: COGNITIVE NEUROSCIENCE AND NEUROIMAGING 2020; 5:173-183. [DOI: 10.1016/j.bpsc.2019.10.007] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/24/2019] [Revised: 09/11/2019] [Accepted: 10/06/2019] [Indexed: 12/28/2022]
|
14
|
van de Laar TW, de Vries B. Simulating Active Inference Processes by Message Passing. Front Robot AI 2019; 6:20. [PMID: 33501036 PMCID: PMC7805795 DOI: 10.3389/frobt.2019.00020] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2018] [Accepted: 03/05/2019] [Indexed: 01/28/2023] Open
Abstract
The free energy principle (FEP) offers a variational calculus-based description for how biological agents persevere through interactions with their environment. Active inference (AI) is a corollary of the FEP, which states that biological agents act to fulfill prior beliefs about preferred future observations (target priors). Purposeful behavior then results from variational free energy minimization with respect to a generative model of the environment with included target priors. However, manual derivations for free energy minimizing algorithms on custom dynamic models can become tedious and error-prone. While probabilistic programming (PP) techniques enable automatic derivation of inference algorithms on free-form models, full automation of AI requires specialized tools for inference on dynamic models, together with the description of an experimental protocol that governs the interaction between the agent and its simulated environment. The contributions of the present paper are two-fold. Firstly, we illustrate how AI can be automated with the use of ForneyLab, a recent PP toolbox that specializes in variational inference on flexibly definable dynamic models. More specifically, we describe AI agents in a dynamic environment as probabilistic state space models (SSM) and perform inference for perception and control in these agents by message passing on a factor graph representation of the SSM. Secondly, we propose a formal experimental protocol for simulated AI. We exemplify how this protocol leads to goal-directed behavior for flexibly definable AI agents in two classical RL examples, namely the Bayesian thermostat and the mountain car parking problems.
Collapse
Affiliation(s)
- Thijs W. van de Laar
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Bert de Vries
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
- GN Hearing Benelux BV, Eindhoven, Netherlands
| |
Collapse
|
15
|
Fornito A, Zalesky A. Computational Approaches to Understanding Mental Dysfunction: Progress, Challenges, and New Frontiers. BIOLOGICAL PSYCHIATRY: COGNITIVE NEUROSCIENCE AND NEUROIMAGING 2018; 3:728-730. [PMID: 30170710 DOI: 10.1016/j.bpsc.2018.07.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/18/2018] [Accepted: 07/20/2018] [Indexed: 10/28/2022]
Affiliation(s)
- Alex Fornito
- Brain and Mental Health Research Hub, Monash Institute of Cognitive and Clinical Neuroscience, School of Psychological Sciences, and Monash Biomedical Imaging, Monash University, Melbourne, Australia.
| | - Andrew Zalesky
- Departments of Biomedical Engineering and Psychiatry, University of Melbourne, Melbourne, Australia
| |
Collapse
|