On Bayesian mechanics: a physics of and by beliefs

An Active-Inference Approach to Second-Person Neuroscience

Social neuroscience has often been criticized for approaching the investigation of the neural processes that enable social interaction and cognition from a passive, detached, third-person perspective, without involving any real-time social interaction. With the emergence of second-person neuroscience, investigators have uncovered the unique complexity of neural-activation patterns in actual, real-time interaction. Social cognition that occurs during social interaction is fundamentally different from that unfolding during social observation. However, it remains unclear how the neural correlates of social interaction are to be interpreted. Here, we leverage the active-inference framework to shed light on the mechanisms at play during social interaction in second-person neuroscience studies. Specifically, we show how counterfactually rich mutual predictions, real-time bodily adaptation, and policy selection explain activation in components of the default mode, salience, and frontoparietal networks of the brain, as well as in the basal ganglia. We further argue that these processes constitute the crucial neural processes that underwrite bona fide social interaction. By placing the experimental approach of second-person neuroscience on the theoretical foundation of the active-inference framework, we inform the field of social neuroscience about the mechanisms of real-life interactions. We thereby contribute to the theoretical foundations of empirical second-person neuroscience.

Supervised structure learning

This paper concerns structure learning or discovery of discrete generative models. It focuses on Bayesian model selection and the assimilation of training data or content, with a special emphasis on the order in which data are ingested. A key move-in the ensuing schemes-is to place priors on the selection of models, based upon expected free energy. In this setting, expected free energy reduces to a constrained mutual information, where the constraints inherit from priors over outcomes (i.e., preferred outcomes). The resulting scheme is first used to perform image classification on the MNIST dataset to illustrate the basic idea, and then tested on a more challenging problem of discovering models with dynamics, using a simple sprite-based visual disentanglement paradigm and the Tower of Hanoi (cf., blocks world) problem. In these examples, generative models are constructed autodidactically to recover (i.e., disentangle) the factorial structure of latent states-and their characteristic paths or dynamics.

Cortical development in the structural model and free energy minimization

A model of neocortical development invoking Friston's Free Energy Principle is applied within the Structural Model of Barbas et al. and the associated functional interpretation advanced by Tucker and Luu. Evolution of a neural field with Hebbian and anti-Hebbian plasticity, maximizing synchrony and minimizing axonal length by apoptotic selection, leads to paired connection systems with mirror symmetry, interacting via Markov blankets along their line of reflection. Applied to development along the radial lines of development in the Structural Model, a primary Markov blanket emerges between the centrifugal synaptic flux in layers 2,3 and 5,6, versus the centripetal flow in layer 4, and axonal orientations in layer 4 give rise to the differing shape and movement sensitivities characteristic of neurons of dorsal and ventral neocortex. Prediction error minimization along the primary blanket integrates limbic and subcortical networks with the neocortex. Synaptic flux bypassing the blanket triggers the arousal response to surprising stimuli, enabling subsequent adaptation. As development progresses ubiquitous mirror systems separated by Markov blankets and enclosed blankets-within-blankets arise throughout neocortex, creating the typical order and response characteristics of columnar and noncolumnar cortex.

Artificial consciousness: a perspective from the free energy principle

Does the assumption of a weak form of computational functionalism, according to which the right form of neural computation is sufficient for consciousness, entail that a digital computational simulation of such neural computations is conscious? Or must this computational simulation be implemented in the right way, in order to replicate consciousness?

From the perspective of Karl Friston’s free energy principle, self-organising systems (such as living organisms) share a set of properties that could be realised in artificial systems, but are not instantiated by computers with a classical (von Neumann) architecture. I argue that at least one of these properties, viz. a certain kind of causal flow, can be used to draw a distinction between systems that merely simulate, and those that actually replicate consciousness.

When the interoceptive and conceptual clash: The case of oppositional phenomenal self-modelling in Tourette syndrome

Tourette syndrome (TS) has been associated with a rich set of symptoms that are said to be uncomfortable, unwilled, and effortful to manage. Furthermore, tics, the canonical characteristic of TS, are multifaceted, and their onset and maintenance is complex. A formal account that integrates these features of TS symptomatology within a plausible theoretical framework is currently absent from the field. In this paper, we assess the explanatory power of hierarchical generative modelling in accounting for TS symptomatology from the perspective of active inference. We propose a fourfold analysis of sensory, motor, and cognitive phenomena associated with TS. In Section 1, we characterise tics as a form of action aimed at sensory attenuation. In Section 2, we introduce the notion of epistemic ticcing and describe such behaviour as the search for evidence that there is an agent (i.e., self) at the heart of the generative hierarchy. In Section 3, we characterise both epistemic (sensation-free) and nonepistemic (sensational) tics as habitual behaviour. Finally, in Section 4, we propose that ticcing behaviour involves an inevitable conflict between distinguishable aspects of selfhood; namely, between the minimal phenomenal sense of self-which is putatively underwritten by interoceptive inference-and the explicit preferences that constitute the individual's conceptual sense of self. In sum, we aim to provide an empirically informed analysis of TS symptomatology under active inference, revealing a continuity between covert and overt features of the condition.

Making the Thermodynamic Cost of Active Inference Explicit

When describing Active Inference Agents (AIAs), the term "energy" can have two distinct meanings. One is the energy that is utilized by the AIA (e.g., electrical energy or chemical energy). The second meaning is so-called Variational Free Energy (VFE), a statistical quantity which provides an upper bound on surprisal. In this paper, we develop an account of the former quantity-the Thermodynamic Free Energy (TFE)-and its relationship with the latter. We highlight the necessary tradeoffs between these two in a generic, quantum information-theoretic formulation, and the macroscopic consequences of those tradeoffs for the ways that organisms approach their environments. By making this tradeoff explicit, we provide a theoretical basis for the different metabolic strategies that organisms from plants to predators use to survive.

Forgetting ourselves in flow: an active inference account of flow states and how we experience ourselves within them

Flow has been described as a state of optimal performance, experienced universally across a broad range of domains: from art to athletics, gaming to writing. However, its phenomenal characteristics can, at first glance, be puzzling. Firstly, individuals in flow supposedly report a loss of self-awareness, even though they perform in a manner which seems to evince their agency and skill. Secondly, flow states are felt to be effortless, despite the prerequisite complexity of the tasks that engender them. In this paper, we unpick these features of flow, as well as others, through the active inference framework, which posits that action and perception are forms of active Bayesian inference directed at sustained self-organisation; i.e., the minimisation of variational free energy. We propose that the phenomenology of flow is rooted in the deployment of high precision weight over (i) the expected sensory consequences of action and (ii) beliefs about how action will sequentially unfold. This computational mechanism thus draws the embodied cognitive system to minimise the ensuing (i.e., expected) free energy through the exploitation of the pragmatic affordances at hand. Furthermore, given the challenging dynamics the flow-inducing situation presents, attention must be wholly focussed on the unfolding task whilst counterfactual planning is restricted, leading to the attested loss of the sense of self-as-object. This involves the inhibition of both the sense of self as a temporally extended object and higher-order, meta-cognitive forms of self-conceptualisation. Nevertheless, we stress that self-awareness is not entirely lost in flow. Rather, it is pre-reflective and bodily. Our approach to bodily-action-centred phenomenology can be applied to similar facets of seemingly agentive experience beyond canonical flow states, providing insights into the mechanisms of so-called selfless experiences, embodied expertise and wellbeing.

Self-Improvising Memory: A Perspective on Memories as Agential, Dynamically Reinterpreting Cognitive Glue

Many studies on memory emphasize the material substrate and mechanisms by which data can be stored and reliably read out. Here, I focus on complementary aspects: the need for agents to dynamically reinterpret and modify memories to suit their ever-changing selves and environment. Using examples from developmental biology, evolution, and synthetic bioengineering, in addition to neuroscience, I propose that a perspective on memory as preserving salience, not fidelity, is applicable to many phenomena on scales from cells to societies. Continuous commitment to creative, adaptive confabulation, from the molecular to the behavioral levels, is the answer to the persistence paradox as it applies to individuals and whole lineages. I also speculate that a substrate-independent, processual view of life and mind suggests that memories, as patterns in the excitable medium of cognitive systems, could be seen as active agents in the sense-making process. I explore a view of life as a diverse set of embodied perspectives-nested agents who interpret each other's and their own past messages and actions as best as they can (polycomputation). This synthesis suggests unifying symmetries across scales and disciplines, which is of relevance to research programs in Diverse Intelligence and the engineering of novel embodied minds.

An Overview of the Free Energy Principle and Related Research

The free energy principle and its corollary, the active inference framework, serve as theoretical foundations in the domain of neuroscience, explaining the genesis of intelligent behavior. This principle states that the processes of perception, learning, and decision making-within an agent-are all driven by the objective of "minimizing free energy," evincing the following behaviors: learning and employing a generative model of the environment to interpret observations, thereby achieving perception, and selecting actions to maintain a stable preferred state and minimize the uncertainty about the environment, thereby achieving decision making. This fundamental principle can be used to explain how the brain processes perceptual information, learns about the environment, and selects actions. Two pivotal tenets are that the agent employs a generative model for perception and planning and that interaction with the world (and other agents) enhances the performance of the generative model and augments perception. With the evolution of control theory and deep learning tools, agents based on the FEP have been instantiated in various ways across different domains, guiding the design of a multitude of generative models and decision-making algorithms. This letter first introduces the basic concepts of the FEP, followed by its historical development and connections with other theories of intelligence, and then delves into the specific application of the FEP to perception and decision making, encompassing both low-dimensional simple situations and high-dimensional complex situations. It compares the FEP with model-based reinforcement learning to show that the FEP provides a better objective function. We illustrate this using numerical studies of Dreamer3 by adding expected information gain into the standard objective function. In a complementary fashion, existing reinforcement learning, and deep learning algorithms can also help implement the FEP-based agents. Finally, we discuss the various capabilities that agents need to possess in complex environments and state that the FEP can aid agents in acquiring these capabilities.

Complexity biomechanics: a case study of dragonfly wing design from constituting composite material to higher structural levels

Presenting a novel framework for sustainable and regenerative design and development is a fundamental future need. Here we argue that a new framework, referred to as complexity biomechanics, which can be used for holistic analysis and understanding of natural mechanical systems, is key to fulfilling this need. We also present a roadmap for the design and development of intelligent and complex engineering materials, mechanisms, structures, systems, and processes capable of automatic adaptation and self-organization in response to ever-changing environments. We apply complexity biomechanics to elucidate how the different structural components of a complex biological system as dragonfly wings, from ultrastructure of the cuticle, the constituting bio-composite material of the wing, to higher structural levels, collaboratively contribute to the functionality of the entire wing system. This framework not only proposes a paradigm shift in understanding and drawing inspiration from natural systems but also holds potential applications in various domains, including materials science and engineering, biomechanics, biomimetics, bionics, and engineering biology.

Clere Smithe T, Friedman DA, Friston K, Ramstead MJD. Shared Protentions in Multi-Agent Active Inference

In this paper, we unite concepts from Husserlian phenomenology, the active inference framework in theoretical biology, and category theory in mathematics to develop a comprehensive framework for understanding social action premised on shared goals. We begin with an overview of Husserlian phenomenology, focusing on aspects of inner time-consciousness, namely, retention, primal impression, and protention. We then review active inference as a formal approach to modeling agent behavior based on variational (approximate Bayesian) inference. Expanding upon Husserl's model of time consciousness, we consider collective goal-directed behavior, emphasizing shared protentions among agents and their connection to the shared generative models of active inference. This integrated framework aims to formalize shared goals in terms of shared protentions, and thereby shed light on the emergence of group intentionality. Building on this foundation, we incorporate mathematical tools from category theory, in particular, sheaf and topos theory, to furnish a mathematical image of individual and group interactions within a stochastic environment. Specifically, we employ morphisms between polynomial representations of individual agent models, allowing predictions not only of their own behaviors but also those of other agents and environmental responses. Sheaf and topos theory facilitates the construction of coherent agent worldviews and provides a way of representing consensus or shared understanding. We explore the emergence of shared protentions, bridging the phenomenology of temporal structure, multi-agent active inference systems, and category theory. Shared protentions are highlighted as pivotal for coordination and achieving common objectives. We conclude by acknowledging the intricacies stemming from stochastic systems and uncertainties in realizing shared goals.

Markov Blankets and Mirror Symmetries-Free Energy Minimization and Mesocortical Anatomy

A theoretical account of development in mesocortical anatomy is derived from the free energy principle, operating in a neural field with both Hebbian and anti-Hebbian neural plasticity. An elementary structural unit is proposed, in which synaptic connections at mesoscale are arranged in paired patterns with mirror symmetry. Exchanges of synaptic flux in each pattern form coupled spatial eigenmodes, and the line of mirror reflection between the paired patterns operates as a Markov blanket, so that prediction errors in exchanges between the pairs are minimized. The theoretical analysis is then compared to the outcomes from a biological model of neocortical development, in which neuron precursors are selected by apoptosis for cell body and synaptic connections maximizing synchrony and also minimizing axonal length. It is shown that this model results in patterns of connection with the anticipated mirror symmetries, at micro-, meso- and inter-arial scales, among lateral connections, and in cortical depth. This explains the spatial organization and functional significance of neuron response preferences, and is compatible with the structural form of both columnar and noncolumnar cortex. Multi-way interactions of mirrored representations can provide a preliminary anatomically realistic model of cortical information processing.

Cerebellar state estimation enables resilient coupling across behavioural domains

Cerebellar computations are necessary for fine behavioural control and may rely on internal models for estimation of behaviourally relevant states. Here, we propose that the central cerebellar function is to estimate how states interact with each other, and to use these estimates to coordinates extra-cerebellar neuronal dynamics underpinning a range of interconnected behaviours. To support this claim, we describe a cerebellar model for state estimation that includes state interactions, and link this model with the neuronal architecture and dynamics observed empirically. This is formalised using the free energy principle, which provides a dual perspective on a system in terms of both the dynamics of its physical-in this case neuronal-states, and the inferential process they entail. As a demonstration of this proposal, we simulate cerebellar-dependent synchronisation of whisking and respiration, which are known to be tightly coupled in rodents, as well as limb and tail coordination during locomotion. In summary, we propose that the ubiquitous involvement of the cerebellum in behaviour arises from its central role in precisely coupling behavioural domains.

Principled Limitations on Self-Representation for Generic Physical Systems

The ideas of self-observation and self-representation, and the concomitant idea of self-control, pervade both the cognitive and life sciences, arising in domains as diverse as immunology and robotics. Here, we ask in a very general way whether, and to what extent, these ideas make sense. Using a generic model of physical interactions, we prove a theorem and several corollaries that severely restrict applicable notions of self-observation, self-representation, and self-control. We show, in particular, that adding observational, representational, or control capabilities to a meta-level component of a system cannot, even in principle, lead to a complete meta-level representation of the system as a whole. We conclude that self-representation can at best be heuristic, and that self models cannot, in general, be empirically tested by the systems that implement them.

Generalized decomposition of multivariate information

Since its introduction, the partial information decomposition (PID) has emerged as a powerful, information-theoretic technique useful for studying the structure of (potentially higher-order) interactions in complex systems. Despite its utility, the applicability of the PID is restricted by the need to assign elements as either "sources" or "targets", as well as the specific structure of the mutual information itself. Here, I introduce a generalized information decomposition that relaxes the source/target distinction while still satisfying the basic intuitions about information. This approach is based on the decomposition of the Kullback-Leibler divergence, and consequently allows for the analysis of any information gained when updating from an arbitrary prior to an arbitrary posterior. As a result, any information-theoretic measure that can be written as a linear combination of Kullback-Leibler divergences admits a decomposition in the style of Williams and Beer, including the total correlation, the negentropy, and the mutual information as special cases. This paper explores how the generalized information decomposition can reveal novel insights into existing measures, as well as the nature of higher-order synergies. We show that synergistic information is intimately related to the well-known Tononi-Sporns-Edelman (TSE) complexity, and that synergistic information requires a similar integration/segregation balance as a high TSE complexity. Finally, I end with a discussion of how this approach fits into other attempts to generalize the PID and the possibilities for empirical applications.

Order and change in art: towards an active inference account of aesthetic experience

How to account for the power that art holds over us? Why do artworks touch us deeply, consoling, transforming or invigorating us in the process? In this paper, we argue that an answer to this question might emerge from a fecund framework in cognitive science known as predictive processing (a.k.a. active inference). We unpack how this approach connects sense-making and aesthetic experiences through the idea of an 'epistemic arc', consisting of three parts (curiosity, epistemic action and aha experiences), which we cast as aspects of active inference. We then show how epistemic arcs are built and sustained by artworks to provide us with those satisfying experiences that we tend to call 'aesthetic'. Next, we defuse two key objections to this approach; namely, that it places undue emphasis on the cognitive component of our aesthetic encounters-at the expense of affective aspects-and on closure and uncertainty minimization (order)-at the expense of openness and lingering uncertainty (change). We show that the approach offers crucial resources to account for the open-ended, free and playful behaviour inherent in aesthetic experiences. The upshot is a promising but deflationary approach, both philosophically informed and psychologically sound, that opens new empirical avenues for understanding our aesthetic encounters. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Federated inference and belief sharing

This paper concerns the distributed intelligence or federated inference that emerges under belief-sharing among agents who share a common world-and world model. Imagine, for example, several animals keeping a lookout for predators. Their collective surveillance rests upon being able to communicate their beliefs-about what they see-among themselves. But, how is this possible? Here, we show how all the necessary components arise from minimising free energy. We use numerical studies to simulate the generation, acquisition and emergence of language in synthetic agents. Specifically, we consider inference, learning and selection as minimising the variational free energy of posterior (i.e., Bayesian) beliefs about the states, parameters and structure of generative models, respectively. The common theme-that attends these optimisation processes-is the selection of actions that minimise expected free energy, leading to active inference, learning and model selection (a.k.a., structure learning). We first illustrate the role of communication in resolving uncertainty about the latent states of a partially observed world, on which agents have complementary perspectives. We then consider the acquisition of the requisite language-entailed by a likelihood mapping from an agent's beliefs to their overt expression (e.g., speech)-showing that language can be transmitted across generations by active learning. Finally, we show that language is an emergent property of free energy minimisation, when agents operate within the same econiche. We conclude with a discussion of various perspectives on these phenomena; ranging from cultural niche construction, through federated learning, to the emergence of complexity in ensembles of self-organising systems.

Myth of objectivity and the origin of symbols

An age-old challenge in epistemology and moral philosophy is whether objectivity exists independent of subjective perspective. Alfred North Whitehead labeled it a "fallacy of misplaced concreteness"; after all, knowledge is represented elusively in symbols. I employ the free energy principle (FEP) to argue that the belief in moral objectivity, although perhaps fallacious, amounts to an ancient and universal human myth that is essential for our symbolic capacity. To perceive any object in a world of non-diminishing (perhaps irreducible) uncertainty, according to the FEP, its constituent parts must display common probabilistic tendencies, known as statistical beliefs, prior to its interpretation, or active inference, as a stable entity. Behavioral bias, subjective emotions, and social norms scale the scope of identity by coalescing agents with otherwise disparate goals and aligning their perspectives into a coherent structure. I argue that by declaring belief in norms as objective, e.g., expressing that a particular theft or infidelity was generally wrong, our ancestors psychologically constructed a type of identity bound only by shared faith in a perspective that technically transcended individual subjectivity. Signaling explicit belief in what were previously non-symbolic norms, as seen in many non-human animals, simulates a top-down point of view of our social interactions and thereby constructs our cultural niche and symbolic capacity. I demonstrate that, largely by contrasting with overly reductive analytical models that assume individual rational pursuit of extrinsic rewards, shared belief in moral conceptions, i.e., what amounts to a religious faith, remains a motivational cornerstone of our language, economic and civic institutions, stories, and psychology. Finally, I hypothesize that our bias for familiar accents (shibboleth), plausibly represents the phylogenetic and ontogenetic contextual origins of our impulse to minimize social surprise by declaring belief in the myth of objectivity.

Relative fluency (unfelt vs felt) in active inference

For a growing number of researchers, it is now accepted that the brain is a predictive organ that predicts the content of the sensorium and crucially the precision of-or confidence in-its own predictions. In order to predict the precision of its predictions, the brain has to infer the reliability of its own beliefs. This means that our brains have to recognise the precision of their predictions or, at least, their accuracy. In this paper, we argue that fluency is product of this recognition process. In short, to recognise fluency is to infer that we have a precise 'grip' on the unfolding processes that generate our sensations. More specifically, we propose that it is changes in fluency - from unfelt to felt - that are both recognised and realised when updating predictions about precision. Unfelt fluency orients attention to unpredicted sensations, while felt fluency supervenes on-and contextualises-unfelt fluency; thereby rendering certain attentional processes, phenomenologically opaque. As such, fluency underwrites the precision we place in our predictions and therefore acts upon our perceptual inferences. Hence, the causes of conscious subjective inference have unconscious perceptual precursors.

Mental Gravity: Depression as Spacetime Curvature of the Self, Mind, and Brain

The principle of mental gravity contends that the mind uses physical gravity as a mental model or simulacrum to express the relation between the inner self and the outer world in terms of "UP"-ness and "DOWN"-ness. The simulation of increased gravity characterises a continuum of mental gravity which states includes depression as the paradigmatic example of being down, low, heavy, and slow. The physics of gravity can also be used to model spacetime curvature in depression, particularly gravitational time dilation as a property of MG analogous to subjective time dilation (i.e., the slowing of temporal flow in conscious experience). The principle has profound implications for the Temporo-spatial Theory of Consciousness (TTC) with regard to temporo-spatial alignment that establishes a "world-brain relation" that is centred on embodiment and the socialisation of conscious states. The principle of mental gravity provides the TTC with a way to incorporate the structure of the world into the structure of the brain, conscious experience, and thought. In concert with other theories of cognitive and neurobiological spacetime, the TTC can also work towards the "common currency" approach that also potentially connects the TTC to predictive processing frameworks such as free energy, neuronal gauge theories, and active inference accounts of depression. It gives the up/down dimension of space, as defined by the gravitational field, a unique status that is connected to both our embodied interaction with the physical world, and also the inverse, reflective, emotional but still embodied experience of ourselves.

A Variational Synthesis of Evolutionary and Developmental Dynamics

This paper introduces a variational formulation of natural selection, paying special attention to the nature of 'things' and the way that different 'kinds' of 'things' are individuated from-and influence-each other. We use the Bayesian mechanics of particular partitions to understand how slow phylogenetic processes constrain-and are constrained by-fast, phenotypic processes. The main result is a formulation of adaptive fitness as a path integral of phenotypic fitness. Paths of least action, at the phenotypic and phylogenetic scales, can then be read as inference and learning processes, respectively. In this view, a phenotype actively infers the state of its econiche under a generative model, whose parameters are learned via natural (Bayesian model) selection. The ensuing variational synthesis features some unexpected aspects. Perhaps the most notable is that it is not possible to describe or model a population of conspecifics per se. Rather, it is necessary to consider populations of distinct natural kinds that influence each other. This paper is limited to a description of the mathematical apparatus and accompanying ideas. Subsequent work will use these methods for simulations and numerical analyses-and identify points of contact with related mathematical formulations of evolution.

The mesoanatomy of the cortex, minimization of free energy, and generative cognition

Capacity for generativity and unlimited association is the defining characteristic of sentience, and this capacity somehow arises from neuronal self-organization in the cortex. We have previously argued that, consistent with the free energy principle, cortical development is driven by synaptic and cellular selection maximizing synchrony, with effects manifesting in a wide range of features of mesoscopic cortical anatomy. Here, we further argue that in the postnatal stage, as more structured inputs reach the cortex, the same principles of self-organization continue to operate at multitudes of local cortical sites. The unitary ultra-small world structures that emerged antenatally can represent sequences of spatiotemporal images. Local shifts of presynapses from excitatory to inhibitory cells result in the local coupling of spatial eigenmodes and the development of Markov blankets, minimizing prediction errors in each unit's interactions with surrounding neurons. In response to the superposition of inputs exchanged between cortical areas, more complicated, potentially cognitive structures are competitively selected by the merging of units and the elimination of redundant connections that result from the minimization of variational free energy and the elimination of redundant degrees of freedom. The trajectory along which free energy is minimized is shaped by interaction with sensorimotor, limbic, and brainstem mechanisms, providing a basis for creative and unlimited associative learning.

Really radical?

I enjoyed reading this compelling account of Conviction Narrative Theory (CNT). As a theoretical neurobiologist, I recognised - and applauded - the tenets of CNT. My commentary asks whether its claims could be installed into a Bayesian mechanics of decision-making, in a way that would enable theoreticians to model, reproduce and predict decision-making.