Collective behavior from surprise minimization

Collective motion is ubiquitous in nature; groups of animals, such as fish, birds, and ungulates appear to move as a whole, exhibiting a rich behavioral repertoire that ranges from directed movement to milling to disordered swarming. Typically, such macroscopic patterns arise from decentralized, local interactions among constituent components (e.g., individual fish in a school). Preeminent models of this process describe individuals as self-propelled particles, subject to self-generated motion and "social forces" such as short-range repulsion and long-range attraction or alignment. However, organisms are not particles; they are probabilistic decision-makers. Here, we introduce an approach to modeling collective behavior based on active inference. This cognitive framework casts behavior as the consequence of a single imperative: to minimize surprise. We demonstrate that many empirically observed collective phenomena, including cohesion, milling, and directed motion, emerge naturally when considering behavior as driven by active Bayesian inference-without explicitly building behavioral rules or goals into individual agents. Furthermore, we show that active inference can recover and generalize the classical notion of social forces as agents attempt to suppress prediction errors that conflict with their expectations. By exploring the parameter space of the belief-based model, we reveal nontrivial relationships between the individual beliefs and group properties like polarization and the tendency to visit different collective states. We also explore how individual beliefs about uncertainty determine collective decision-making accuracy. Finally, we show how agents can update their generative model over time, resulting in groups that are collectively more sensitive to external fluctuations and encode information more robustly.

Path integrals, particular kinds, and strange things

This paper describes a path integral formulation of the free energy principle. The ensuing account expresses the paths or trajectories that a particle takes as it evolves over time. The main results are a method or principle of least action that can be used to emulate the behaviour of particles in open exchange with their external milieu. Particles are defined by a particular partition, in which internal states are individuated from external states by active and sensory blanket states. The variational principle at hand allows one to interpret internal dynamics-of certain kinds of particles-as inferring external states that are hidden behind blanket states. We consider different kinds of particles, and to what extent they can be imbued with an elementary form of inference or sentience. Specifically, we consider the distinction between dissipative and conservative particles, inert and active particles and, finally, ordinary and strange particles. Strange particles can be described as inferring their own actions, endowing them with apparent autonomy or agency. In short-of the kinds of particles afforded by a particular partition-strange kinds may be apt for describing sentient behaviour.

On Bayesian mechanics: a physics of and by beliefs

The aim of this paper is to introduce a field of study that has emerged over the last decade, called Bayesian mechanics. Bayesian mechanics is a probabilistic mechanics, comprising tools that enable us to model systems endowed with a particular partition (i.e. into particles), where the internal states (or the trajectories of internal states) of a particular system encode the parameters of beliefs about external states (or their trajectories). These tools allow us to write down mechanical theories for systems that look as if they are estimating posterior probability distributions over the causes of their sensory states. This provides a formal language for modelling the constraints, forces, potentials and other quantities determining the dynamics of such systems, especially as they entail dynamics on a space of beliefs (i.e. on a statistical manifold). Here, we will review the state of the art in the literature on the free energy principle, distinguishing between three ways in which Bayesian mechanics has been applied to particular systems (i.e. path-tracking, mode-tracking and mode-matching). We go on to examine a duality between the free energy principle and the constrained maximum entropy principle, both of which lie at the heart of Bayesian mechanics, and discuss its implications.

Reward Maximization Through Discrete Active Inference

Active inference is a probabilistic framework for modeling the behavior of biological and artificial agents, which derives from the principle of minimizing free energy. In recent years, this framework has been applied successfully to a variety of situations where the goal was to maximize reward, often offering comparable and sometimes superior performance to alternative approaches. In this article, we clarify the connection between reward maximization and active inference by demonstrating how and when active inference agents execute actions that are optimal for maximizing reward. Precisely, we show the conditions under which active inference produces the optimal solution to the Bellman equation, a formulation that underlies several approaches to model-based reinforcement learning and control. On partially observed Markov decision processes, the standard active inference scheme can produce Bellman optimal actions for planning horizons of 1 but not beyond. In contrast, a recently developed recursive active inference scheme (sophisticated inference) can produce Bellman optimal actions on any finite temporal horizon. We append the analysis with a discussion of the broader relationship between active inference and reinforcement learning.

Branching Time Active Inference: The theory and its generality

Over the last 10 to 15 years, active inference has helped to explain various brain mechanisms from habit formation to dopaminergic discharge and even modelling curiosity. However, the current implementations suffer from an exponential (space and time) complexity class when computing the prior over all the possible policies up to the time-horizon. Fountas et al. (2020) used Monte Carlo tree search to address this problem, leading to impressive results in two different tasks. In this paper, we present an alternative framework that aims to unify tree search and active inference by casting planning as a structure learning problem. Two tree search algorithms are then presented. The first propagates the expected free energy forward in time (i.e., towards the leaves), while the second propagates it backward (i.e., towards the root). Then, we demonstrate that forward and backward propagations are related to active inference and sophisticated inference, respectively, thereby clarifying the differences between those two planning strategies.

Bayesian Brains and the Rényi Divergence

Under the Bayesian brain hypothesis, behavioral variations can be attributed to different priors over generative model parameters. This provides a formal explanation for why individuals exhibit inconsistent behavioral preferences when confronted with similar choices. For example, greedy preferences are a consequence of confident (or precise) beliefs over certain outcomes. Here, we offer an alternative account of behavioral variability using Rényi divergences and their associated variational bounds. Rényi bounds are analogous to the variational free energy (or evidence lower bound) and can be derived under the same assumptions. Importantly, these bounds provide a formal way to establish behavioral differences through an α parameter, given fixed priors. This rests on changes in α that alter the bound (on a continuous scale), inducing different posterior estimates and consequent variations in behavior. Thus, it looks as if individuals have different priors and have reached different conclusions. More specifically, α→0+ optimization constrains the variational posterior to be positive whenever the true posterior is positive. This leads to mass-covering variational estimates and increased variability in choice behavior. Furthermore, α→+∞ optimization constrains the variational posterior to be zero whenever the true posterior is zero. This leads to mass-seeking variational posteriors and greedy preferences. We exemplify this formulation through simulations of the multiarmed bandit task. We note that these α parameterizations may be especially relevant (i.e., shape preferences) when the true posterior is not in the same family of distributions as the assumed (simpler) approximate density, which may be the case in many real-world scenarios. The ensuing departure from vanilla variational inference provides a potentially useful explanation for differences in behavioral preferences of biological (or artificial) agents under the assumption that the brain performs variational Bayesian inference.

Bayesian mechanics for stationary processes

This paper develops a Bayesian mechanics for adaptive systems. Firstly, we model the interface between a system and its environment with a Markov blanket. This affords conditions under which states internal to the blanket encode information about external states. Second, we introduce dynamics and represent adaptive systems as Markov blankets at steady state. This allows us to identify a wide class of systems whose internal states appear to infer external states, consistent with variational inference in Bayesian statistics and theoretical neuroscience. Finally, we partition the blanket into sensory and active states. It follows that active states can be seen as performing active inference and well-known forms of stochastic control (such as PID control), which are prominent formulations of adaptive behaviour in theoretical biology and engineering.

Stochastic Chaos and Markov Blankets

In this treatment of random dynamical systems, we consider the existence-and identification-of conditional independencies at nonequilibrium steady-state. These independencies underwrite a particular partition of states, in which internal states are statistically secluded from external states by blanket states. The existence of such partitions has interesting implications for the information geometry of internal states. In brief, this geometry can be read as a physics of sentience, where internal states look as if they are inferring external states. However, the existence of such partitions-and the functional form of the underlying densities-have yet to be established. Here, using the Lorenz system as the basis of stochastic chaos, we leverage the Helmholtz decomposition-and polynomial expansions-to parameterise the steady-state density in terms of surprisal or self-information. We then show how Markov blankets can be identified-using the accompanying Hessian-to characterise the coupling between internal and external states in terms of a generalised synchrony or synchronisation of chaos. We conclude by suggesting that this kind of synchronisation may provide a mathematical basis for an elemental form of (autonomous or active) sentience in biology.

A Drive towards Thermodynamic Efficiency for Dissipative Structures in Chemical Reaction Networks

Dissipative accounts of structure formation show that the self-organisation of complex structures is thermodynamically favoured, whenever these structures dissipate free energy that could not be accessed otherwise. These structures therefore open transition channels for the state of the universe to move from a frustrated, metastable state to another metastable state of higher entropy. However, these accounts apply as well to relatively simple, dissipative systems, such as convection cells, hurricanes, candle flames, lightning strikes, or mechanical cracks, as they do to complex biological systems. Conversely, interesting computational properties-that characterize complex biological systems, such as efficient, predictive representations of environmental dynamics-can be linked to the thermodynamic efficiency of underlying physical processes. However, the potential mechanisms that underwrite the selection of dissipative structures with thermodynamically efficient subprocesses is not completely understood. We address these mechanisms by explaining how bifurcation-based, work-harvesting processes-required to sustain complex dissipative structures-might be driven towards thermodynamic efficiency. We first demonstrate a simple mechanism that leads to self-selection of efficient dissipative structures in a stochastic chemical reaction network, when the dissipated driving chemical potential difference is decreased. We then discuss how such a drive can emerge naturally in a hierarchy of self-similar dissipative structures, each feeding on the dissipative structures of a previous level, when moving away from the initial, driving disequilibrium.

Memory and Markov Blankets

In theoretical biology, we are often interested in random dynamical systems-like the brain-that appear to model their environments. This can be formalized by appealing to the existence of a (possibly non-equilibrium) steady state, whose density preserves a conditional independence between a biological entity and its surroundings. From this perspective, the conditioning set, or Markov blanket, induces a form of vicarious synchrony between creature and world-as if one were modelling the other. However, this results in an apparent paradox. If all conditional dependencies between a system and its surroundings depend upon the blanket, how do we account for the mnemonic capacity of living systems? It might appear that any shared dependence upon past blanket states violates the independence condition, as the variables on either side of the blanket now share information not available from the current blanket state. This paper aims to resolve this paradox, and to demonstrate that conditional independence does not preclude memory. Our argument rests upon drawing a distinction between the dependencies implied by a steady state density, and the density dynamics of the system conditioned upon its configuration at a previous time. The interesting question then becomes: What determines the length of time required for a stochastic system to 'forget' its initial conditions? We explore this question for an example system, whose steady state density possesses a Markov blanket, through simple numerical analyses. We conclude with a discussion of the relevance for memory in cognitive systems like us.

Sophisticated Inference

Active inference offers a first principle account of sentient behavior, from which special and important cases-for example, reinforcement learning, active learning, Bayes optimal inference, Bayes optimal design-can be derived. Active inference finesses the exploitation-exploration dilemma in relation to prior preferences by placing information gain on the same footing as reward or value. In brief, active inference replaces value functions with functionals of (Bayesian) beliefs, in the form of an expected (variational) free energy. In this letter, we consider a sophisticated kind of active inference using a recursive form of expected free energy. Sophistication describes the degree to which an agent has beliefs about beliefs. We consider agents with beliefs about the counterfactual consequences of action for states of affairs and beliefs about those latent states. In other words, we move from simply considering beliefs about "what would happen if I did that" to "what I would believe about what would happen if I did that." The recursive form of the free energy functional effectively implements a deep tree search over actions and outcomes in the future. Crucially, this search is over sequences of belief states as opposed to states per se. We illustrate the competence of this scheme using numerical simulations of deep decision problems.

Some Interesting Observations on the Free Energy Principle

Biehl et al. (2021) present some interesting observations on an early formulation of the free energy principle. We use these observations to scaffold a discussion of the technical arguments that underwrite the free energy principle. This discussion focuses on solenoidal coupling between various (subsets of) states in sparsely coupled systems that possess a Markov blanket-and the distinction between exact and approximate Bayesian inference, implied by the ensuing Bayesian mechanics.

Generative Models for Active Vision

The active visual system comprises the visual cortices, cerebral attention networks, and oculomotor system. While fascinating in its own right, it is also an important model for sensorimotor networks in general. A prominent approach to studying this system is active inference-which assumes the brain makes use of an internal (generative) model to predict proprioceptive and visual input. This approach treats action as ensuring sensations conform to predictions (i.e., by moving the eyes) and posits that visual percepts are the consequence of updating predictions to conform to sensations. Under active inference, the challenge is to identify the form of the generative model that makes these predictions-and thus directs behavior. In this paper, we provide an overview of the generative models that the brain must employ to engage in active vision. This means specifying the processes that explain retinal cell activity and proprioceptive information from oculomotor muscle fibers. In addition to the mechanics of the eyes and retina, these processes include our choices about where to move our eyes. These decisions rest upon beliefs about salient locations, or the potential for information gain and belief-updating. A key theme of this paper is the relationship between "looking" and "seeing" under the brain's implicit generative model of the visual world.

Neural Dynamics under Active Inference: Plausibility and Efficiency of Information Processing

Active inference is a normative framework for explaining behaviour under the free energy principle-a theory of self-organisation originating in neuroscience. It specifies neuronal dynamics for state-estimation in terms of a descent on (variational) free energy-a measure of the fit between an internal (generative) model and sensory observations. The free energy gradient is a prediction error-plausibly encoded in the average membrane potentials of neuronal populations. Conversely, the expected probability of a state can be expressed in terms of neuronal firing rates. We show that this is consistent with current models of neuronal dynamics and establish face validity by synthesising plausible electrophysiological responses. We then show that these neuronal dynamics approximate natural gradient descent, a well-known optimisation algorithm from information geometry that follows the steepest descent of the objective in information space. We compare the information length of belief updating in both schemes, a measure of the distance travelled in information space that has a direct interpretation in terms of metabolic cost. We show that neural dynamics under active inference are metabolically efficient and suggest that neural representations in biological agents may evolve by approximating steepest descent in information space towards the point of optimal inference.

Active inference on discrete state-spaces: A synthesis

Active inference is a normative principle underwriting perception, action, planning, decision-making and learning in biological or artificial agents. From its inception, its associated process theory has grown to incorporate complex generative models, enabling simulation of a wide range of complex behaviours. Due to successive developments in active inference, it is often difficult to see how its underlying principle relates to process theories and practical implementation. In this paper, we try to bridge this gap by providing a complete mathematical synthesis of active inference on discrete state-space models. This technical summary provides an overview of the theory, derives neuronal dynamics from first principles and relates this dynamics to biological processes. Furthermore, this paper provides a fundamental building block needed to understand active inference for mixed generative models; allowing continuous sensations to inform discrete representations. This paper may be used as follows: to guide research towards outstanding challenges, a practical guide on how to implement active inference to simulate experimental behaviour, or a pointer towards various in-silico neurophysiological responses that may be used to make empirical predictions.

Future climates: Markov blankets and active inference in the biosphere

We formalize the Gaia hypothesis about the Earth climate system using advances in theoretical biology based on the minimization of variational free energy. This amounts to the claim that non-equilibrium steady-state dynamics-that underwrite our climate-depend on the Earth system possessing a Markov blanket. Our formalization rests on how the metabolic rates of the biosphere (understood as Markov blanket's internal states) change with respect to solar radiation at the Earth's surface (i.e. external states), through the changes in greenhouse and albedo effects (i.e. active states) and ocean-driven global temperature changes (i.e. sensory states). Describing the interaction between the metabolic rates and solar radiation as climatic states-in a Markov blanket-amounts to describing the dynamics of the internal states as actively inferring external states. This underwrites climatic non-equilibrium steady-state through free energy minimization and thus a form of planetary autopoiesis.

Markov blankets, information geometry and stochastic thermodynamics

This paper considers the relationship between thermodynamics, information and inference. In particular, it explores the thermodynamic concomitants of belief updating, under a variational (free energy) principle for self-organization. In brief, any (weakly mixing) random dynamical system that possesses a Markov blanket-i.e. a separation of internal and external states-is equipped with an information geometry. This means that internal states parametrize a probability density over external states. Furthermore, at non-equilibrium steady-state, the flow of internal states can be construed as a gradient flow on a quantity known in statistics as Bayesian model evidence. In short, there is a natural Bayesian mechanics for any system that possesses a Markov blanket. Crucially, this means that there is an explicit link between the inference performed by internal states and their energetics-as characterized by their stochastic thermodynamics. This article is part of the theme issue 'Harmonizing energy-autonomous computing and intelligence'.

Severe infection in a lung transplant recipient caused by donor-transmitted carbapenem-resistant Acinetobacter baumannii

We describe a case of proven donor transmission of carbapenem-resistant Acinetobacter baumannii, which resulted in severe infectious complications after lung transplantation. A single bla(OXA-23) positive strain, belonging to a new multilocus sequence type (ST231), was isolated from donor and recipient, who died 65 days after transplantation. This report highlights the current challenges associated with the potential transmission of multidrug-resistant infections through organ transplantation.

Computer-vision-based extraction of neural dendrograms

This work introduces a new approach to the characterization of neural cells by means of semi-automated generation of dendrograms; data structures which describe the inherently hierarchical nature of neuronal arborizations. Dendrograms describe the branched structure of neurons in terms of the length, average thickness and bending energy of each of the dendritic segments and allow in a straightforward manner, the inclusion of additional measures. The bending energy quantifies the complexity of the shape and can be used to characterize the spatial coverage of the arborizations (the bending energy is an alternative for other complexity measures such as the fractal dimension). The new approach is based on the partitioning of the cell's outer contour as a function of the high curvature points followed by a syntactical analysis of the segmented contours. The semi-automated method is robust and is an improvement on the time consuming manual generation of the dendrograms. Several experimental results are included in this paper which illustrate and corroborate the effectiveness of the approach. The technique presented in this paper is limited to planar neurons but could be extended to a 3D approach.

[Use of papain in visceral infections]

This research was carried out at the University Hospital of the University of São Paulo and deals with the experiment of papain utilization for visceral irrigation in patients with severe infection. It was observed that seventy two hours after treatment, there was a considerable reduction of purulent secretion, and that the medium time of cicatrization of all lesions was thirty days.

Quality of life in patients 2 to 4 years after closed head injury

This study evaluated quality of life in 78 patients with closed head injury (CHI) 2 to 4 years postinjury. Using both interview data and mean data from the Sickness Impact Profile questionnaire, impaired quality of life was observed in the areas of psychosocial functioning, social role functioning, leisure activities, and, to a lesser extent, physical functioning, during chronic phases of recovery. Relatives and close friends reported by means of the Katz Adjustment Scale that the CHI patients showed a series of negative behavioral symptoms 2 to 4 years postinjury. These data suggest that CHI patients may experience impaired quality of life in a number of domains well beyond the acute postinjury phases. An attempt was also made to compare patients' and relatives' reports of patient quality of life. Preliminary analyses indicated modest correspondence between relatives' and patients' ratings of some areas of postinjury dysfunction, including cognitive and behavioral slowing and social withdrawal.

Predictors and indicators of quality of life in patients with closed-head injury

This study examined predictors and indicators of quality of life in 71 patients with closed-head injury (CHI), 2-4 years postinjury. Predictors included premorbid characteristics and acute injury-related data. Indicators included follow-up data, e.g., neuropsychological functioning. Exploratory canonical correlation analyses demonstrated that the combination of the predictor variable, initial Glasgow Coma Scale score, and indicator variables of neuropsychological data in the areas of motor functioning, memory, and constructional ability were related most strongly to quality of life as reported by the patients. Severity of head injury and motor disability also related strongly to quality of life, based on reports by relatives (n = 68) on the Katz Adjustment Scale (Relatives' Form). These findings suggest that quality of life is adversely affected by increased severity of head injury and greater residual motor deficits. Implications of these findings for treatment and recovery are discussed.

Interset variability on the raven coloured progressive matrices as an indicator of specific ability deficit in brain-lesioned patients

Patients with right cerebral lesions were found to have lower RCPM scores than a matched group of patients with left lesions. Analysis of interset patterns of performance demonstrated relatively greater difficulty for right lesion patients, especially those with posterior lesions on the Ab set. The findings were interpreted in terms of the abilities that Raven maintains underly test performance.

The relation of visuospatial dysfunction to digit span performance in patients with cerebral lesions

DSF and DSB performance was compared in hospitalized controls and right and left brain lesion groups dichotomized for the presence or absence of visuospatial deficits. Digit Span performance was also correlated with WAIS Similarities and Block Design and Raven Coloured Progressive Matrices. No differences between groups were observed on DSF. On DSB patients with brain lesions had lower scores than controls and brain-lesioned patients with visuospatial deficits had lower scores than those without. DSB correlated significantly with WAIS Block Design and Raven Coloured Progressive Matrices supporting the hypothesis that visuospatial ability is needed to mediate proper DSB performance. The correlation of DSB with WAIS Similarities, however, lends support to the idea that low DSB may merely reflect severity of cognitive deficit.

Test-retest reliability of serial pure-tone audiograms in children at a school for the deaf

Successive air-conduction audiograms obtained between ages 1¼ and 12 years and recorded in the charts of 75 children at a school for the deaf were analyzed statistically. Approximately half of 414 audiogram pairs differed by at least 20 dB for 2 or more frequencies, and 34% of 3203 paired frequency scores differed by at least 20 dB. Conductive losses, moderate hearing losses, and perhaps mild mental retardation were associated with unreliability. Audiograms performed at the school by a nonprofessional audiologist were especially unreliable. The desirability of retesting a child until a stable threshold is obtained and of indicating the zone of uncertainty around each threshold point was stressed. When the zone of uncertainty is wide, a verbal label describing the estimated severity of the loss might be preferable to possibly invalid numerical information.

Choice reaction time as a function of stimulus uncertainty in patients with brain lesions

Longer times and greater variabilities for 24 brain-damaged Ss than for normals were associated with increased stimulus information in a card-sorting task.