Active Inference: A Process Theory

Branching Time Active Inference with Bayesian Filtering

Branching time active inference is a framework proposing to look at planning as a form of Bayesian model expansion. Its root can be found in active inference, a neuroscientific framework widely used for brain modeling, as well as in Monte Carlo tree search, a method broadly applied in the reinforcement learning literature. Up to now, the inference of the latent variables was carried out by taking advantage of the flexibility offered by variational message passing, an iterative process that can be understood as sending messages along the edges of a factor graph. In this letter, we harness the efficiency of an alternative method for inference, Bayesian filtering, which does not require the iteration of the update equations until convergence of the variational free energy. Instead, this scheme alternates between two phases: integration of evidence and prediction of future states. Both phases can be performed efficiently, and this provides a forty times speedup over the state of the art.

The why of the phenomenal aspect of consciousness: Its main functions and the mechanisms underpinning it

What distinguishes conscious information processing from other kinds of information processing is its phenomenal aspect (PAC), the-what-it-is-like for an agent to experience something. The PAC supplies the agent with a sense of self, and informs the agent on how its self is affected by the agent's own operations. The PAC originates from the activity that attention performs to detect the state of what I define "the self" (S). S is centered and develops on a hierarchy of innate and acquired values, and is primarily expressed via the central and peripheral nervous systems; it maps the agent's body and cognitive capacities, and its interactions with the environment. The detection of the state of S by attention modulates the energy level of the organ of attention (OA), i.e., the neural substrate that underpins attention. This modulation generates the PAC. The PAC can be qualified according to five dimensions: qualitative, quantitative, hedonic, temporal and spatial. Each dimension can be traced back to a specific feature of the modulation of the energy level of the OA.

Reclaiming saliency: Rhythmic precision-modulated action and perception

Computational models of visual attention in artificial intelligence and robotics have been inspired by the concept of a saliency map. These models account for the mutual information between the (current) visual information and its estimated causes. However, they fail to consider the circular causality between perception and action. In other words, they do not consider where to sample next, given current beliefs. Here, we reclaim salience as an active inference process that relies on two basic principles: uncertainty minimization and rhythmic scheduling. For this, we make a distinction between attention and salience. Briefly, we associate attention with precision control, i.e., the confidence with which beliefs can be updated given sampled sensory data, and salience with uncertainty minimization that underwrites the selection of future sensory data. Using this, we propose a new account of attention based on rhythmic precision-modulation and discuss its potential in robotics, providing numerical experiments that showcase its advantages for state and noise estimation, system identification and action selection for informative path planning.

Cognitive control is modulated by hierarchical complexity of task switching: An event-related potential study

The study aimed to explore the effect of hierarchical complexity on task switching. The participants (n = 36) were asked to perform a magnitude or parity judgement on digits (1-9) in the hierarchical simple or complex block. In the simple block, participants made a numerical judgement on the presented digit (1-9) in each trial, whereas in the complex block, they had to first identify whether the digit in the current trial belonged to a predefined category (e.g., whether it was an even number), then perform a numerical judgment or not respond. The behavioural results revealed a significant interaction between hierarchical complexity and transition type (repeat vs. switch), with greater switch cost in the complex than in the simple block. Event-related potentials (ERPs) locked in the cue stage did not reveal this interaction, whereas the ERPs locked in the target stage revealed this interaction during the N2 and P3 time windows, with a larger switch negativity (switch minus repeat) in the complex than in the simple block. These findings demonstrate that an increase in hierarchical complexity triggers increased reactive control in the inhibition of the old task-set and reconfiguration of the new task-set during task switching.

A Quantum Predictive Brain: Complementarity Between Top-Down Predictions and Bottom-Up Evidence

Predictive brain theory challenges the general assumption of a brain extracting knowledge from sensations and considers the brain as an organ of inference, actively constructing explanations about reality beyond its sensory evidence. Predictive brain has been formalized through Bayesian updating, where top-down predictions are compared with bottom-up evidence. In this article, we propose a different approach to predictive brain based on quantum probability-we call it Quantum Predictive Brain (QPB). QPB is consistent with the Bayesian framework, but considers it as a special case. The tenet of QPB is that top-down predictions and bottom-up evidence are complementary, as they cannot be co-jointly determined to pursue a univocal model of brain functioning. QPB can account for several high-order cognitive phenomena (which are problematic in current predictive brain theories) and offers new insights into the mechanisms of neural reuse.

Everything is connected: Inference and attractors in delusions

Delusions are, by popular definition, false beliefs that are held with certainty and resistant to contradictory evidence. They seem at odds with the notion that the brain at least approximates Bayesian inference. This is especially the case in schizophrenia, a disorder thought to relate to decreased - rather than increased - certainty in the brain's model of the world. We use an active inference Markov decision process model (a Bayes-optimal decision-making agent) to perform a simple task involving social and non-social inferences. We show that even moderate changes in some model parameters - decreasing confidence in sensory input and increasing confidence in states implied by its own (especially habitual) actions - can lead to delusions as defined above. Incorporating affect in the model increases delusions, specifically in the social domain. The model also reproduces some classic psychological effects, including choice-induced preference change, and an optimism bias in inferences about oneself. A key observation is that no change in a single parameter is both necessary and sufficient for delusions; rather, delusions arise due to conditional dependencies that create 'basins of attraction' which trap Bayesian beliefs. Simulating the effects of antidopaminergic antipsychotics - by reducing the model's confidence in its actions - demonstrates that the model can escape from these attractors, through this synthetic pharmacotherapy.

Neuroscience and architecture: Modulating behavior through sensorimotor responses to the built environment

As we move through the world, natural and built environments implicitly guide behavior by appealing to certain sensory and motor dynamics. This process can be motivated by automatic attention to environmental features that resonate with specific sensorimotor responses. This review aims at providing a psychobiological framework describing how environmental features can lead to automated sensorimotor responses through defined neurophysiological mechanisms underlying attention. Through the use of automated processes in subsets of cortical structures, the goal of this framework is to describe on a neuronal level the functional link between the designed environment and sensorimotor responses. By distinguishing between environmental features and sensorimotor responses we elaborate on how automatic behavior employs the environment for sensorimotor adaptation. This is realized through a thalamo-cortical network integrating environmental features with motor aspects of behavior. We highlight the underlying transthalamic transmission from an Enactive and predictive perspective and review recent studies that effectively modulated behavior by systematically manipulating environmental features. We end by suggesting a promising combination of neuroimaging and computational analysis for future studies.

Neurocomputations of strategic behavior: From iterated to novel interactions

Strategic interactions, where an individual's payoff depends on the decisions of multiple intelligent agents, are ubiquitous among social animals. They span a variety of important social behaviors such as competition, cooperation, coordination, and communication, and often involve complex, intertwining cognitive operations ranging from basic reward processing to higher‐order mentalization. Here, we review the progress and challenges in probing the neural and cognitive mechanisms of strategic behavior of interacting individuals, drawing an analogy to recent developments in studies of reward‐seeking behavior, in particular, how research focuses in the field of strategic behavior have been expanded from adaptive behavior based on trial‐and‐error to flexible decisions based on limited prior experience. We highlight two important research questions in the field of strategic behavior: (i) How does the brain exploit past experience for learning to behave strategically? and (ii) How does the brain decide what to do in novel strategic situations in the absence of direct experience? For the former, we discuss the utility of learning models that have effectively connected various types of neural data with strategic learning behavior and helped elucidate the interplay among multiple learning processes. For the latter, we review the recent evidence and propose a neural generative mechanism by which the brain makes novel strategic choices through simulating others' goal‐directed actions according to rational or bounded‐rational principles obtained through indirect social knowledge.

This article is categorized under:

Economics > Interactive Decision‐Making

Psychology > Reasoning and Decision Making

Neuroscience > Cognition

[Perinatal beliefs: Neurocognitive mechanisms and cultural specificities]

Perinatal beliefs contribute to the experience of pregnancy and the process of parenthood. Many of these perinatal beliefs have been perpetuated and evolved over time and throughout the world, exerting their influence on the behavior of pregnant women in interaction with medical recommendations. These beliefs generally offer explanations for gravidic and puerperal phenomena, helping to reduce the uncertainty of parents faced with the biological, psychological and social transitions of pregnancy. But certain beliefs can also be harmful, and alter the maternal experience of pregnancy and postpartum. In this paper, we provide an overview of the beliefs associated with the perinatal period. We successively detail the beliefs concerning fertility, pregnancy, childbirth, and postpartum, specifying the cultural beliefs from other cultures interacting with medical recommendations. Finally, we propose a neurocognitive model of perinatal beliefs generation, and we show the need to know these beliefs to improve care in midwifery, obstetrics, and fetal medicine.

Rethinking delusions: A selective review of delusion research through a computational lens

Delusions are rigid beliefs held with high certainty despite contradictory evidence. Notwithstanding decades of research, we still have a limited understanding of the computational and neurobiological alterations giving rise to delusions. In this review, we highlight a selection of recent work in computational psychiatry aimed at developing quantitative models of inference and its alterations, with the goal of providing an explanatory account for the form of delusional beliefs in psychosis. First, we assess and evaluate the experimental paradigms most often used to study inferential alterations in delusions. Based on our review of the literature and theoretical considerations, we contend that classic draws-to-decision paradigms are not well-suited to isolate inferential processes, further arguing that the commonly cited 'jumping-to-conclusion' bias may reflect neither delusion-specific nor inferential alterations. Second, we discuss several enhancements to standard paradigms that show promise in more effectively isolating inferential processes and delusion-related alterations therein. We further draw on our recent work to build an argument for a specific failure mode for delusions consisting of prior overweighting in high-level causal inferences about partially observable hidden states. Finally, we assess plausible neurobiological implementations for this candidate failure mode of delusional beliefs and outline promising future directions in this area.

Resourceful Event-Predictive Inference: The Nature of Cognitive Effort

Pursuing a precise, focused train of thought requires cognitive effort. Even more effort is necessary when more alternatives need to be considered or when the imagined situation becomes more complex. Cognitive resources available to us limit the cognitive effort we can spend. In line with previous work, an information-theoretic, Bayesian brain approach to cognitive effort is pursued: to solve tasks in our environment, our brain needs to invest information, that is, negative entropy, to impose structure, or focus, away from a uniform structure or other task-incompatible, latent structures. To get a more complete formalization of cognitive effort, a resourceful event-predictive inference model (REPI) is introduced, which offers computational and algorithmic explanations about the latent structure of our generative models, the active inference dynamics that unfold within, and the cognitive effort required to steer the dynamics-to, for example, purposefully process sensory signals, decide on responses, and invoke their execution. REPI suggests that we invest cognitive resources to infer preparatory priors, activate responses, and anticipate action consequences. Due to our limited resources, though, the inference dynamics are prone to task-irrelevant distractions. For example, the task-irrelevant side of the imperative stimulus causes the Simon effect and, due to similar reasons, we fail to optimally switch between tasks. An actual model implementation simulates such task interactions and offers first estimates of the involved cognitive effort. The approach may be further studied and promises to offer deeper explanations about why we get quickly exhausted from multitasking, how we are influenced by irrelevant stimulus modalities, why we exhibit magnitude interference, and, during social interactions, why we often fail to take the perspective of others into account.

The Problem of Meaning: The Free Energy Principle and Artificial Agency

Biological agents can act in ways that express a sensitivity to context-dependent relevance. So far it has proven difficult to engineer this capacity for context-dependent sensitivity to relevance in artificial agents. We give this problem the label the "problem of meaning". The problem of meaning could be circumvented if artificial intelligence researchers were to design agents based on the assumption of the continuity of life and mind. In this paper, we focus on the proposal made by enactive cognitive scientists to design artificial agents that possess sensorimotor autonomy-stable, self-sustaining patterns of sensorimotor interaction that can ground values, norms and goals necessary for encountering a meaningful environment. More specifically, we consider whether the Free Energy Principle (FEP) can provide formal tools for modeling sensorimotor autonomy. There is currently no consensus on how to understand the relationship between enactive cognitive science and the FEP. However, a number of recent papers have argued that the two frameworks are fundamentally incompatible. Some argue that biological systems exhibit historical path-dependent learning that is absent from systems that minimize free energy. Others have argued that a free energy minimizing system would fail to satisfy a key condition for sensorimotor agency referred to as "interactional asymmetry". These critics question the claim we defend in this paper that the FEP can be used to formally model autonomy and adaptivity. We will argue it is too soon to conclude that the two frameworks are incompatible. There are undeniable conceptual differences between the two frameworks but in our view each has something important and necessary to offer. The FEP needs enactive cognitive science for the solution it provides to the problem of meaning. Enactive cognitive science needs the FEP to formally model the properties it argues to be constitutive of agency. Our conclusion will be that active inference models based on the FEP provides a way by which scientists can think about how to address the problems of engineering autonomy and adaptivity in artificial agents in formal terms. In the end engaging more closely with this formalism and its further developments will benefit those working within the enactive framework.

Electromagnetism's Bridge Across the Explanatory Gap: How a Neuroscience/Physics Collaboration Delivers Explanation Into All Theories of Consciousness

A productive, informative three decades of correlates of phenomenal consciousness (P-Consciousness) have delivered valuable knowledge while simultaneously locating us in a unique and unprecedented explanatory cul-de-sac. Observational correlates are demonstrated to be intrinsically very unlikely to explain or lead to a fundamental principle underlying the strongly emergent 1st-person-perspective (1PP) invisibly stowed away inside them. That lack is now solidly evidenced in practice. To escape our explanatory impasse, this article focuses on fundamental physics (the standard model of particle physics), which brings to light a foundational argument for how the brain is an essentially electromagnetic (EM) field object from the atomic level up. That is, our multitude of correlates of P-Consciousness are actually descriptions of specific EM field behaviors that are posed (hypothesized) as "the right" correlate by a particular theory of consciousness. Because of this, our 30 years of empirical progress can be reinterpreted as, in effect, the delivery of a large body of evidence that the standard model's EM quadrant can deliver a 1PP. That is, all theories of consciousness are, in the end, merely recipes that select a particular subset of the totality of EM field expression that is brain tissue. With a universal convergence on EM, the science of P-Consciousness becomes a collaborative effort between neuroscience and physics. The collaboration acts in pursuit of a unified explanation applicable to all theories of consciousness while remaining mindful that the process still contains no real explanation as to why or how EM fields deliver a 1PP. The apparent continued lack of explanation is, however, different: this time, the way forward is opened through its direct connection to fundamental physics. This is the first result (Part I). Part II posits, in general terms, a structural (epistemic) add-on/upgrade to the standard model that has the potential to deliver the missing route to an explanation of how subjectivity is delivered through EM fields. The revised standard model, under the neuroscience/physics collaboration, intimately integrates with the existing "correlates of-" paradigm, which acts as its source of empirical evidence. No existing theory of consciousness is lost or invalidated.

A Predictive Processing Model of Episodic Memory and Time Perception

Human perception and experience of time are strongly influenced by ongoing stimulation, memory of past experiences, and required task context. When paying attention to time, time experience seems to expand; when distracted, it seems to contract. When considering time based on memory, the experience may be different than what is in the moment, exemplified by sayings like "time flies when you're having fun." Experience of time also depends on the content of perceptual experience-rapidly changing or complex perceptual scenes seem longer in duration than less dynamic ones. The complexity of interactions among attention, memory, and perceptual stimulation is a likely reason that an overarching theory of time perception has been difficult to achieve. Here, we introduce a model of perceptual processing and episodic memory that makes use of hierarchical predictive coding, short-term plasticity, spatiotemporal attention, and episodic memory formation and recall, and apply this model to the problem of human time perception. In an experiment with approximately 13,000 human participants, we investigated the effects of memory, cognitive load, and stimulus content on duration reports of dynamic natural scenes up to about 1 minute long. Using our model to generate duration estimates, we compared human and model performance. Model-based estimates replicated key qualitative biases, including differences by cognitive load (attention), scene type (stimulation), and whether the judgment was made based on current or remembered experience (memory). Our work provides a comprehensive model of human time perception and a foundation for exploring the computational basis of episodic memory within a hierarchical predictive coding framework.

Grounding Context in Embodied Cognitive Robotics

Biological agents are context-dependent systems that exhibit behavioral flexibility. The internal and external information agents process, their actions, and emotions are all grounded in the context within which they are situated. However, in the field of cognitive robotics, the concept of context is far from being clear with most studies making little to no reference to it. The aim of this paper is to provide an interpretation of the notion of context and its core elements based on different studies in natural agents, and how these core contextual elements have been modeled in cognitive robotics, to introduce a new hypothesis about the interactions between these contextual elements. Here, global context is categorized as agent-related, environmental, and task-related context. The interaction of their core elements, allows agents to first select self-relevant tasks depending on their current needs, or for learning and mastering their environment through exploration. Second, to perform a task and continuously monitor its performance. Third, to abandon a task in case its execution is not going as expected. Here, the monitoring of prediction error, the difference between sensorimotor predictions and incoming sensory information, is at the core of behavioral flexibility during situated action cycles. Additionally, monitoring prediction error dynamics and its comparison with the expected reduction rate should indicate the agent its overall performance on executing the task. Sensitivity to performance evokes emotions that function as the driving element for autonomous behavior which, at the same time, depends on the processing of the interacting core elements. Taking all these into account, an interactionist model of contexts and their core elements is proposed. The model is embodied, affective, and situated, by means of the processing of the agent-related and environmental core contextual elements. Additionally, it is grounded in the processing of the task-related context and the associated situated action cycles during task execution. Finally, the model proposed here aims to guide how artificial agents should process the core contextual elements of the agent-related and environmental context to give rise to the task-related context, allowing agents to autonomously select a task, its planning, execution, and monitoring for behavioral flexibility.

Spatiotemporal Signatures of Surprise Captured by Magnetoencephalography

Surprise and social influence are linked through several neuropsychological mechanisms. By garnering attention, causing arousal, and motivating engagement, surprise provides a context for effective or durable social influence. Attention to a surprising event motivates the formation of an explanation or updating of models, while high arousal experiences due to surprise promote memory formation. They both encourage engagement with the surprising event through efforts aimed at understanding the situation. By affecting the behavior of the individual or a social group via setting an attractive engagement context, surprise plays an important role in shaping personal and social change. Surprise is an outcome of the brain’s function in constantly anticipating the future of sensory inputs based on past experiences. When new sensory data is different from the brain’s predictions shaped by recent trends, distinct neural signals are generated to report this surprise. As a quantitative approach to modeling the generation of brain surprise, input stimuli containing surprising elements are employed in experiments such as oddball tasks during which brain activity is recorded. Although surprise has been well characterized in many studies, an information-theoretical model to describe and predict the surprise level of an external stimulus in the recorded MEG data has not been reported to date, and setting forth such a model is the main objective of this paper. Through mining trial-by-trial MEG data in an oddball task according to theoretical definitions of surprise, the proposed surprise decoding model employs the entire epoch of the brain response to a stimulus to measure surprise and assesses which collection of temporal/spatial components in the recorded data can provide optimal power for describing the brain’s surprise. We considered three different theoretical formulations for surprise assuming the brain acts as an ideal observer that calculates transition probabilities to estimate the generative distribution of the input. We found that middle temporal components and the right and left fronto-central regions offer the strongest power for decoding surprise. Our findings provide a practical and rigorous method for measuring the brain’s surprise, which can be employed in conjunction with behavioral data to evaluate the interactive and social effects of surprising events.

Competency in Navigating Arbitrary Spaces as an Invariant for Analyzing Cognition in Diverse Embodiments

One of the most salient features of life is its capacity to handle novelty and namely to thrive and adapt to new circumstances and changes in both the environment and internal components. An understanding of this capacity is central to several fields: the evolution of form and function, the design of effective strategies for biomedicine, and the creation of novel life forms via chimeric and bioengineering technologies. Here, we review instructive examples of living organisms solving diverse problems and propose competent navigation in arbitrary spaces as an invariant for thinking about the scaling of cognition during evolution. We argue that our innate capacity to recognize agency and intelligence in unfamiliar guises lags far behind our ability to detect it in familiar behavioral contexts. The multi-scale competency of life is essential to adaptive function, potentiating evolution and providing strategies for top-down control (not micromanagement) to address complex disease and injury. We propose an observer-focused viewpoint that is agnostic about scale and implementation, illustrating how evolution pivoted similar strategies to explore and exploit metabolic, transcriptional, morphological, and finally 3D motion spaces. By generalizing the concept of behavior, we gain novel perspectives on evolution, strategies for system-level biomedical interventions, and the construction of bioengineered intelligences. This framework is a first step toward relating to intelligence in highly unfamiliar embodiments, which will be essential for progress in artificial intelligence and regenerative medicine and for thriving in a world increasingly populated by synthetic, bio-robotic, and hybrid beings.

Cognitive Neural Mechanism of Backward Inhibition and Deinhibition: A Review

Task switching is one of the typical paradigms to study cognitive control. When switching back to a recently inhibited task (e.g., “A” in an ABA sequence), the performance is often worse compared to a task without N-2 task repetitions (e.g., CBA). This difference is called the backward inhibitory effect (BI effect), which reflects the process of overcoming residual inhibition from a recently performed task (i.e., deinhibition). The neural mechanism of backward inhibition and deinhibition has received a lot of attention in the past decade. Multiple brain regions, including the frontal lobe, parietal, basal ganglia, and cerebellum, are activated during deinhibition. The event-related potentials (ERP) studies have shown that deinhibition process is reflected in the P1/N1 and P3 components, which might be related to early attention control, context updating, and response selection, respectively. Future research can use a variety of new paradigms to separate the neural mechanisms of BI and deinhibition.

Towards a cross-level understanding of Bayesian inference in the brain

Perception emerges from unconscious probabilistic inference, which guides behaviour in our ubiquitously uncertain environment. Bayesian decision theory is a prominent computational model that describes how people make rational decisions using noisy and ambiguous sensory observations. However, critical questions have been raised about the validity of the Bayesian framework in explaining the mental process of inference. Firstly, some natural behaviours deviate from Bayesian optimum. Secondly, the neural mechanisms that support Bayesian computations in the brain are yet to be understood. Taking Marr's cross level approach, we review the recent progress made in addressing these challenges. We first review studies that combined behavioural paradigms and modelling approaches to explain both optimal and suboptimal behaviours. Next, we evaluate the theoretical advances and the current evidence for ecologically feasible algorithms and neural implementations in the brain, which may enable probabilistic inference. We argue that this cross-level approach is necessary for the worthwhile pursuit to uncover mechanistic accounts of human behaviour.

EMDR Therapy and PTSD: A Goal-Directed Predictive Processing Perspective

Eye movement desensitization and reprocessing (EMDR) therapy is a widely used evidence-based treatment for posttraumatic stress disorder (PTSD). The mental processes underlying both PTSD and EMDR treatment effects are often explained by drawing on processes that involve the automatic formation and change of mental associations. Recent evidence that contrasts with these explanations is discussed and a new perspective to PTSD and EMDR treatment effects is proposed that draws on automatic inferential processes and can be readily integrated with the dominant (Adaptive Information Processing) model. This new perspective incorporates insights from cognitive theories that draw on predictive processing and goal-directed processes to elucidate (changes in) automatic inferences that underlie PTSD symptoms and EMDR treatment effects. Recommendations for clinical practice are provided based on this new perspective.

How Stress Can Change Our Deepest Preferences: Stress Habituation Explained Using the Free Energy Principle

People who habituate to stress show a repetition-induced response attenuation—neuroendocrine, cardiovascular, neuroenergetic, and emotional—when exposed to a threatening environment. But the exact dynamics underlying stress habituation remain obscure. The free energy principle offers a unifying account of self-organising systems such as the human brain. In this paper, we elaborate on how stress habituation can be explained and modelled using the free energy principle. We introduce habituation priors that encode the agent’s tendency for stress habituation and incorporate them in the agent’s decision-making process. Using differently shaped goal priors—that encode the agent’s goal preferences—we illustrate, in two examples, the optimising (and thus habituating) behaviour of agents. We show that habituation minimises free energy by reducing the precision (inverse variance) of goal preferences. Reducing the precision of goal priors means that the agent accepts adverse (previously unconscionable) states (e.g., lower social status and poverty). Acceptance or tolerance of adverse outcomes may explain why habituation causes people to exhibit an attenuation of the stress response. Given that stress habituation occurs in brain regions where goal priors are encoded, i.e., in the ventromedial prefrontal cortex and that these priors are encoded as sufficient statistics of probability distributions, our approach seems plausible from an anatomical-functional and neuro-statistical point of view. The ensuing formal and generalisable account—based on the free energy principle—further motivate our novel treatment of stress habituation. Our analysis suggests that stress habituation has far-reaching consequences, protecting against the harmful effects of toxic stress, but on the other hand making the acceptability of precarious living conditions and the development of the obese type 2 diabetes mellitus phenotype more likely.

An informal reconstruction of the free-energy framework, examining the conceptual problems that arise

Recent decades have seen increasing attention given to the free-energy framework. This proposes that many of the phenomena we associate with life, intelligence and adaptation can be explained in terms of a single, mathematical precept: the free-energy principle. This is claimed to apply to the adaptive behavior of primitive organisms as much as it does to the physiological structures of human brains. The proposal is potentially of interest to theorists from multiple disciplines. But as presentations often make intensive use of mathematical notation, it can be hard to understand, even for experts. The present article presents an informal reconstruction, using schematic illustrations. Mathematical notation is largely avoided, while detail and precision are retained as far as possible. The specifically conceptual problems that come to notice in the reconstruction are highlighted and discussed.

Ongoing Brain Activity and Its Role in Cognition: Dual versus Baseline Models

What is the role of the brain's ongoing activity for cognition? The predominant perspectives associate ongoing brain activity with resting state, the default-mode network (DMN), and internally oriented mentation. This triad is often contrasted with task states, non-DMN brain networks, and externally oriented mentation, together comprising a "dual model" of brain and cognition. In opposition to this duality, however, we propose that ongoing brain activity serves as a neuronal baseline; this builds upon Raichle's original search for the default mode of brain function that extended beyond the canonical default-mode brain regions. That entails what we refer to as the "baseline model." Akin to an internal biological clock for the rest of the organism, the ongoing brain activity may serve as an internal point of reference or standard by providing a shared neural code for the brain's rest as well as task states, including their associated cognition. Such shared neural code is manifest in the spatiotemporal organization of the brain's ongoing activity, including its global signal topography and dynamics like intrinsic neural timescales. We conclude that recent empirical evidence supports a baseline model over the dual model; the ongoing activity provides a global shared neural code that allows integrating the brain's rest and task states, its DMN and non-DMN, and internally and externally oriented cognition.

A free energy principle for generic quantum systems

The Free Energy Principle (FEP) states that under suitable conditions of weak coupling, random dynamical systems with sufficient degrees of freedom will behave so as to minimize an upper bound, formalized as a variational free energy, on surprisal (a.k.a., self-information). This upper bound can be read as a Bayesian prediction error. Equivalently, its negative is a lower bound on Bayesian model evidence (a.k.a., marginal likelihood). In short, certain random dynamical systems evince a kind of self-evidencing. Here, we reformulate the FEP in the formal setting of spacetime-background free, scale-free quantum information theory. We show how generic quantum systems can be regarded as observers, which with the standard freedom of choice assumption become agents capable of assigning semantics to observational outcomes. We show how such agents minimize Bayesian prediction error in environments characterized by uncertainty, insufficient learning, and quantum contextuality. We show that in its quantum-theoretic formulation, the FEP is asymptotically equivalent to the Principle of Unitarity. Based on these results, we suggest that biological systems employ quantum coherence as a computational resource and - implicitly - as a communication resource. We summarize a number of problems for future research, particularly involving the resources required for classical communication and for detecting and responding to quantum context switches.

Applying the Free Energy Principle to Complex Adaptive Systems

The free energy principle (FEP) is a formulation of the adaptive, belief-driven behaviour of self-organizing systems that gained prominence in the early 2000s as a unified model of the brain [...].

Emergence of associative learning in a neuromorphic inference network

OBJECTIVE

In the theoretical framework of predictive coding and active inference, the brain can be viewed as instantiating a rich generative model of the world that predicts incoming sensory data while continuously updating its parameters via minimization of prediction errors. While this theory has been successfully applied to cognitive processes - by modelling the activity of functional neural networks at a mesoscopic scale - the validity of the approach when modelling neurons as an ensemble of inferring agents, in a biologically plausible architecture, remained to be explored.

APPROACH

We modelled a simplified cerebellar circuit with individual neurons acting as Bayesian agents to simulate the classical delayed eyeblink conditioning protocol. Neurons and synapses adjusted their activity to minimize their prediction error, which was used as the network cost function. This cerebellar network was then implemented in hardware by replicating digital neuronal elements via a low-power microcontroller.

MAIN RESULTS

Persistent changes of synaptic strength - that mirrored neurophysiological observations - emerged via local (neurocentric) prediction error minimization, leading to the expression of associative learning. The same paradigm was effectively emulated in low-power hardware showing remarkably efficient performance compared to conventional neuromorphic architectures.

SIGNIFICANCE

These findings show that: i) an ensemble of free energy minimizing neurons - organized in a biological plausible architecture - can recapitulate functional self-organization observed in nature, such as associative plasticity, and ii) a neuromorphic network of inference units can learn unsupervised tasks without embedding predefined learning rules in the circuit, thus providing a potential avenue to a novel form of brain-inspired artificial intelligence.

Theories of consciousness

Recent years have seen a blossoming of theories about the biological and physical basis of consciousness. Good theories guide empirical research, allowing us to interpret data, develop new experimental techniques and expand our capacity to manipulate the phenomenon of interest. Indeed, it is only when couched in terms of a theory that empirical discoveries can ultimately deliver a satisfying understanding of a phenomenon. However, in the case of consciousness, it is unclear how current theories relate to each other, or whether they can be empirically distinguished. To clarify this complicated landscape, we review four prominent theoretical approaches to consciousness: higher-order theories, global workspace theories, re-entry and predictive processing theories and integrated information theory. We describe the key characteristics of each approach by identifying which aspects of consciousness they propose to explain, what their neurobiological commitments are and what empirical data are adduced in their support. We consider how some prominent empirical debates might distinguish among these theories, and we outline three ways in which theories need to be developed to deliver a mature regimen of theory-testing in the neuroscience of consciousness. There are good reasons to think that the iterative development, testing and comparison of theories of consciousness will lead to a deeper understanding of this most profound of mysteries.

I overthink-Therefore I am not: An active inference account of altered sense of self and agency in depersonalisation disorder

This paper considers the phenomenology of depersonalisation disorder, in relation to predictive processing and its associated pathophysiology. To do this, we first establish a few mechanistic tenets of predictive processing that are necessary to talk about phenomenal transparency, mental action, and self as subject. We briefly review the important role of 'predicting precision' and how this affords mental action and the loss of phenomenal transparency. We then turn to sensory attenuation and the phenomenal consequences of (pathophysiological) failures to attenuate or modulate sensory precision. We then consider this failure in the context of depersonalisation disorder. The key idea here is that depersonalisation disorder reflects the remarkable capacity to explain perceptual engagement with the world via the hypothesis that "I am an embodied perceiver, but I am not in control of my perception". We suggest that individuals with depersonalisation may believe that 'another agent' is controlling their thoughts, perceptions or actions, while maintaining full insight that the 'other agent' is 'me' (the self). Finally, we rehearse the predictions of this formal analysis, with a special focus on the psychophysical and physiological abnormalities that may underwrite the phenomenology of depersonalisation.

Noradrenergic deficits contribute to apathy in Parkinson's disease through the precision of expected outcomes

Apathy is a debilitating feature of many neuropsychiatric diseases, that is typically described as a reduction of goal-directed behaviour. Despite its prevalence and prognostic importance, the mechanisms underlying apathy remain controversial. Degeneration of the locus coeruleus-noradrenaline system is known to contribute to motivational deficits, including apathy. In healthy people, noradrenaline has been implicated in signalling the uncertainty of expectations about the environment. We proposed that noradrenergic deficits contribute to apathy by modulating the relative weighting of prior beliefs about action outcomes. We tested this hypothesis in the clinical context of Parkinson's disease, given its associations with apathy and noradrenergic dysfunction. Participants with mild-to-moderate Parkinson's disease (N = 17) completed a randomised double-blind, placebo-controlled, crossover study with 40 mg of the noradrenaline reuptake inhibitor atomoxetine. Prior weighting was inferred from psychophysical analysis of performance in an effort-based visuomotor task, and was confirmed as negatively correlated with apathy. Locus coeruleus integrity was assessed in vivo using magnetisation transfer imaging at ultra-high field 7T. The effect of atomoxetine depended on locus coeruleus integrity: participants with a more degenerate locus coeruleus showed a greater increase in prior weighting on atomoxetine versus placebo. The results indicate a contribution of the noradrenergic system to apathy and potential benefit from noradrenergic treatment of people with Parkinson's disease, subject to stratification according to locus coeruleus integrity. More broadly, these results reconcile emerging predictive processing accounts of the role of noradrenaline in goal-directed behaviour with the clinical symptom of apathy and its potential pharmacological treatment.

Late N1 and Post Imperative Negative Variation analysis depending on the previous trial history in paradigms of increasing auditory complexity

Predictive coding reflects the ability of the human brain to extract environmental patterns in order to reformulate previous expectations. The present report analyzes through the late N1 auditory component and the post imperative negative variation (PINV) the updating of predictions regarding the characteristics of a new trial, depending on the previous trial history, complexity, and type of trial (standard or deviant). Data were obtained from 31 healthy subjects recorded in a previous study, based on two paradigms composed of stimulus sequences of decreasing or increasing frequencies intermingled with the sporadic appearance of unexpected tone endings. Our results showed a higher amplitude for the most complex condition and deviant trials for both the late N1 and the PINV components. Additionally, the N1 and PINV presented a different amplitude response to the standard and deviant trials as a function of previous trial history, suggesting a continuous updating of trial categorization. The results suggest that the late N1 and PINV components are involved in the generation of an internal model about the rules of external auditory stimulation.

Integrating Evolutionary, Cultural, and Computational Psychiatry: A Multilevel Systemic Approach

This paper proposes an integrative perspective on evolutionary, cultural and computational approaches to psychiatry. These three approaches attempt to frame mental disorders as multiscale entities and offer modes of explanations and modeling strategies that can inform clinical practice. Although each of these perspectives involves systemic thinking, each is limited in its ability to address the complex developmental trajectories and larger social systemic interactions that lead to mental disorders. Inspired by computational modeling in theoretical biology, this paper aims to integrate the modes of explanation offered by evolutionary, cultural and computational psychiatry in a multilevel systemic perspective. We apply the resulting Evolutionary, Cultural and Computational (ECC) model to Major Depressive Disorder (MDD) to illustrate how this integrative approach can guide research and practice in psychiatry.

A step-by-step tutorial on active inference and its application to empirical data

The active inference framework, and in particular its recent formulation as a partially observable Markov decision process (POMDP), has gained increasing popularity in recent years as a useful approach for modeling neurocognitive processes. This framework is highly general and flexible in its ability to be customized to model any cognitive process, as well as simulate predicted neuronal responses based on its accompanying neural process theory. It also affords both simulation experiments for proof of principle and behavioral modeling for empirical studies. However, there are limited resources that explain how to build and run these models in practice, which limits their widespread use. Most introductions assume a technical background in programming, mathematics, and machine learning. In this paper we offer a step-by-step tutorial on how to build POMDPs, run simulations using standard MATLAB routines, and fit these models to empirical data. We assume a minimal background in programming and mathematics, thoroughly explain all equations, and provide exemplar scripts that can be customized for both theoretical and empirical studies. Our goal is to provide the reader with the requisite background knowledge and practical tools to apply active inference to their own research. We also provide optional technical sections and multiple appendices, which offer the interested reader additional technical details. This tutorial should provide the reader with all the tools necessary to use these models and to follow emerging advances in active inference research.

Epistemic Communities under Active Inference

The spread of ideas is a fundamental concern of today's news ecology. Understanding the dynamics of the spread of information and its co-option by interested parties is of critical importance. Research on this topic has shown that individuals tend to cluster in echo-chambers and are driven by confirmation bias. In this paper, we leverage the active inference framework to provide an in silico model of confirmation bias and its effect on echo-chamber formation. We build a model based on active inference, where agents tend to sample information in order to justify their own view of reality, which eventually leads to them to have a high degree of certainty about their own beliefs. We show that, once agents have reached a certain level of certainty about their beliefs, it becomes very difficult to get them to change their views. This system of self-confirming beliefs is upheld and reinforced by the evolving relationship between an agent's beliefs and observations, which over time will continue to provide evidence for their ingrained ideas about the world. The epistemic communities that are consolidated by these shared beliefs, in turn, tend to produce perceptions of reality that reinforce those shared beliefs. We provide an active inference account of this community formation mechanism. We postulate that agents are driven by the epistemic value that they obtain from sampling or observing the behaviours of other agents. Inspired by digital social networks like Twitter, we build a generative model in which agents generate observable social claims or posts (e.g., 'tweets') while reading the socially observable claims of other agents that lend support to one of two mutually exclusive abstract topics. Agents can choose which other agent they pay attention to at each timestep, and crucially who they attend to and what they choose to read influences their beliefs about the world. Agents also assess their local network's perspective, influencing which kinds of posts they expect to see other agents making. The model was built and simulated using the freely available Python package pymdp. The proposed active inference model can reproduce the formation of echo-chambers over social networks, and gives us insight into the cognitive processes that lead to this phenomenon.

Model Reduction Through Progressive Latent Space Pruning in Deep Active Inference

Although still not fully understood, sleep is known to play an important role in learning and in pruning synaptic connections. From the active inference perspective, this can be cast as learning parameters of a generative model and Bayesian model reduction, respectively. In this article, we show how to reduce dimensionality of the latent space of such a generative model, and hence model complexity, in deep active inference during training through a similar process. While deep active inference uses deep neural networks for state space construction, an issue remains in that the dimensionality of the latent space must be specified beforehand. We investigate two methods that are able to prune the latent space of deep active inference models. The first approach functions similar to sleep and performs model reduction post hoc. The second approach is a novel method which is more similar to reflection, operates during training and displays “aha” moments when the model is able to reduce latent space dimensionality. We show for two well-known simulated environments that model performance is retained in the first approach and only diminishes slightly in the second approach. We also show that reconstructions from a real world example are indistinguishable before and after reduction. We conclude that the most important difference constitutes a trade-off between training time and model performance in terms of accuracy and the ability to generalize, via minimization of model complexity.

How Active Inference Could Help Revolutionise Robotics

Recent advances in neuroscience have characterised brain function using mathematical formalisms and first principles that may be usefully applied elsewhere. In this paper, we explain how active inference-a well-known description of sentient behaviour from neuroscience-can be exploited in robotics. In short, active inference leverages the processes thought to underwrite human behaviour to build effective autonomous systems. These systems show state-of-the-art performance in several robotics settings; we highlight these and explain how this framework may be used to advance robotics.

A target-driven visual navigation method based on intrinsic motivation exploration and space topological cognition

Target-driven visual navigation is essential for many applications in robotics, and it has gained increasing interest in recent years. In this work, inspired by animal cognitive mechanisms, we propose a novel navigation architecture that simultaneously learns exploration policy and encodes environmental structure. First, to learn exploration policy directly from raw visual input, we use deep reinforcement learning as the basic framework and allow agents to create rewards for themselves as learning signals. In our approach, the reward for the current observation is driven by curiosity and calculated by a count-based approach and temporal distance. While agents learn exploration policy, we use temporal distance to find waypoints in observation sequences and incrementally describe the structure of the environment in a way that integrates episodic memory. Finally, space topological cognition is integrated into the model as a path planning module and combined with a locomotion network to obtain a more generalized approach to navigation. We test our approach in the DMlab, a visually rich 3D environment, and validate its exploration efficiency and navigation performance through extensive experiments. The experimental results show that our approach can explore and encode the environment more efficiently and has better capability in dealing with stochastic objects. In navigation tasks, agents can use space topological cognition to effectively reach the target and guide detour behaviour when a path is unavailable, exhibiting good environmental adaptability.

How particular is the physics of the free energy principle?

The free energy principle (FEP) states that any dynamical system can be interpreted as performing Bayesian inference upon its surrounding environment. Although, in theory, the FEP applies to a wide variety of systems, there has been almost no direct exploration or demonstration of the principle in concrete systems. In this work, we examine in depth the assumptions required to derive the FEP in the simplest possible set of systems - weakly-coupled non-equilibrium linear stochastic systems. Specifically, we explore (i) how general the requirements imposed on the statistical structure of a system are and (ii) how informative the FEP is about the behaviour of such systems. We discover that two requirements of the FEP - the Markov blanket condition (i.e. a statistical boundary precluding direct coupling between internal and external states) and stringent restrictions on its solenoidal flows (i.e. tendencies driving a system out of equilibrium) - are only valid for a very narrow space of parameters. Suitable systems require an absence of perception-action asymmetries that is highly unusual for living systems interacting with an environment. More importantly, we observe that a mathematically central step in the argument, connecting the behaviour of a system to variational inference, relies on an implicit equivalence between the dynamics of the average states of a system with the average of the dynamics of those states. This equivalence does not hold in general even for linear stochastic systems, since it requires an effective decoupling from the system's history of interactions. These observations are critical for evaluating the generality and applicability of the FEP and indicate the existence of significant problems of the theory in its current form. These issues make the FEP, as it stands, not straightforwardly applicable to the simple linear systems studied here and suggest that more development is needed before the theory could be applied to the kind of complex systems that describe living and cognitive processes.

Therapeutic Alliance as Active Inference: The Role of Therapeutic Touch and Synchrony

Recognizing and aligning individuals' unique adaptive beliefs or "priors" through cooperative communication is critical to establishing a therapeutic relationship and alliance. Using active inference, we present an empirical integrative account of the biobehavioral mechanisms that underwrite therapeutic relationships. A significant mode of establishing cooperative alliances-and potential synchrony relationships-is through ostensive cues generated by repetitive coupling during dynamic touch. Established models speak to the unique role of affectionate touch in developing communication, interpersonal interactions, and a wide variety of therapeutic benefits for patients of all ages; both neurophysiologically and behaviorally. The purpose of this article is to argue for the importance of therapeutic touch in establishing a therapeutic alliance and, ultimately, synchrony between practitioner and patient. We briefly overview the importance and role of therapeutic alliance in prosocial and clinical interactions. We then discuss how cooperative communication and mental state alignment-in intentional communication-are accomplished using active inference. We argue that alignment through active inference facilitates synchrony and communication. The ensuing account is extended to include the role of (C-) tactile afferents in realizing the beneficial effect of therapeutic synchrony. We conclude by proposing a method for synchronizing the effects of touch using the concept of active inference.

The evolution of brain architectures for predictive coding and active inference

This article considers the evolution of brain architectures for predictive processing. We argue that brain mechanisms for predictive perception and action are not late evolutionary additions of advanced creatures like us. Rather, they emerged gradually from simpler predictive loops (e.g. autonomic and motor reflexes) that were a legacy from our earlier evolutionary ancestors-and were key to solving their fundamental problems of adaptive regulation. We characterize simpler-to-more-complex brains formally, in terms of generative models that include predictive loops of increasing hierarchical breadth and depth. These may start from a simple homeostatic motif and be elaborated during evolution in four main ways: these include the multimodal expansion of predictive control into an allostatic loop; its duplication to form multiple sensorimotor loops that expand an animal's behavioural repertoire; and the gradual endowment of generative models with hierarchical depth (to deal with aspects of the world that unfold at different spatial scales) and temporal depth (to select plans in a future-oriented manner). In turn, these elaborations underwrite the solution to biological regulation problems faced by increasingly sophisticated animals. Our proposal aligns neuroscientific theorising-about predictive processing-with evolutionary and comparative data on brain architectures in different animal species. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Synthetic Spatial Foraging With Active Inference in a Geocaching Task

Humans are highly proficient in learning about the environments in which they operate. They form flexible spatial representations of their surroundings that can be leveraged with ease during spatial foraging and navigation. To capture these abilities, we present a deep Active Inference model of goal-directed behavior, and the accompanying belief updating. Active Inference rests upon optimizing Bayesian beliefs to maximize model evidence or marginal likelihood. Bayesian beliefs are probability distributions over the causes of observable outcomes. These causes include an agent's actions, which enables one to treat planning as inference. We use simulations of a geocaching task to elucidate the belief updating-that underwrites spatial foraging-and the associated behavioral and neurophysiological responses. In a geocaching task, the aim is to find hidden objects in the environment using spatial coordinates. Here, synthetic agents learn about the environment via inference and learning (e.g., learning about the likelihoods of outcomes given latent states) to reach a target location, and then forage locally to discover the hidden object that offers clues for the next location.

High-value decisions are fast and accurate, inconsistent with diminishing value sensitivity

It is a widely held belief that people's choices are less sensitive to changes in value as value increases. For example, the subjective difference between $11 and $12 is believed to be smaller than between $1 and $2. This idea is consistent with applications of the Weber-Fechner Law and divisive normalization to value-based choice and with psychological interpretations of diminishing marginal utility. According to random utility theory in economics, smaller subjective differences predict less accurate choices. Meanwhile, in the context of sequential sampling models in psychology, smaller subjective differences also predict longer response times. Based on these models, we would predict decisions between high-value options to be slower and less accurate. In contrast, some have argued on normative grounds that choices between high-value options should be made with less caution, leading to faster and less accurate choices. Here, we model the dynamics of the choice process across three different choice domains, accounting for both discriminability and response caution. Contrary to predictions, we mostly observe faster and more accurate decisions (i.e., higher drift rates) between high-value options. We also observe that when participants are alerted about incoming high-value decisions, they exert more caution and not less. We rule out several explanations for these results, using tasks with both subjective and objective values. These results cast doubt on the notion that increasing value reduces discriminability.

Artificial Interactionism: Avoiding Isolating Perception From Cognition in AI

We discuss the influence upon the fields of robotics and AI of the manner one conceives the relationships between artificial agents' perception, cognition, and action. We shed some light upon a widespread paradigm we call the isolated perception paradigm that addresses perception as isolated from cognition and action. By mobilizing the resources of philosophy (phenomenology and epistemology) and cognitive sciences, and by drawing on recent approaches in AI, we explore what it could mean for robotics and AI to take distance from the isolated perception paradigm. We argue that such a renouncement opens interesting ways to explore the possibilities for designing artificial agents with intrinsic motivations and constitutive autonomy. We then propose Artificial Interactionism, our approach that escapes the isolated perception paradigm by drawing on the inversion of the interaction cycle. When the interaction cycle is inverted, input data are not percepts directly received from the environment, but outcomes of control loops. Perception is not received from sensors in isolation from cognition but is actively constructed by the cognitive architecture through interaction. We give an example implementation of artificial interactionism that demonstrates basic intrinsically motivated learning behavior in a dynamic simulated environment.