An Active Inference Approach to Modeling Structure Learning: Concept Learning as an Example Case

Mechanisms of brain self-regulation: psychological factors, mechanistic models and neural substrates

While neurofeedback represents a promising tool for neuroscience and a brain self-regulation approach to psychological rehabilitation, the field faces several problems and challenges. Current research has shown great variability and even failure among human participants in learning to self-regulate target features of brain activity with neurofeedback. A better understanding of cognitive mechanisms, psychological factors and neural substrates underlying self-regulation might help improve neurofeedback's scientific and clinical practices. This article reviews the current understanding of the neural mechanisms of brain self-regulation by drawing on findings from human and animal studies in neurofeedback, brain-computer/machine interfaces and neuroprosthetics. In this article, we look closer at the following topics: cognitive processes and psychophysiological factors affecting self-regulation, theoretical models and neural substrates underlying self-regulation, and finally, we provide an outlook on the outstanding gaps in knowledge and technical challenges. This article is part of the theme issue 'Neurofeedback: new territories and neurocognitive mechanisms of endogenous neuromodulation'.

Supervised structure learning

This paper concerns structure learning or discovery of discrete generative models. It focuses on Bayesian model selection and the assimilation of training data or content, with a special emphasis on the order in which data are ingested. A key move-in the ensuing schemes-is to place priors on the selection of models, based upon expected free energy. In this setting, expected free energy reduces to a constrained mutual information, where the constraints inherit from priors over outcomes (i.e., preferred outcomes). The resulting scheme is first used to perform image classification on the MNIST dataset to illustrate the basic idea, and then tested on a more challenging problem of discovering models with dynamics, using a simple sprite-based visual disentanglement paradigm and the Tower of Hanoi (cf., blocks world) problem. In these examples, generative models are constructed autodidactically to recover (i.e., disentangle) the factorial structure of latent states-and their characteristic paths or dynamics.

Survival in a world of complex dangers

How did our nomadic ancestors continually adapt to the seemingly limitless and unpredictable number of dangers in the natural world? We argue that human defensive behaviors are dynamically constructed to facilitate survival in capricious and itinerant environments. We first hypothesize that internal and external states result in state constructions that combine to form a meta-representation. When a threat is detected, it triggers the action construction. Action constructions are formed through two contiguous survival strategies: generalization strategies, which are used when encountering new threats and ecologies. Generalization strategies are associated with cognitive representations that have high dimensionality and which furnish flexible psychological constructs, including relations between threats, and imagination, and which converge through the construction of defensive states. We posit that generalization strategies drive 'explorative' behaviors including information seeking, where the goal is to increase knowledge that can be used to mitigate current and future threats. Conversely, specialization strategies entail lower dimensional representations, which underpin specialized, sometimes reflexive, or habitual survival behaviors that are 'exploitative'. Together, these strategies capture a central adaptive feature of human survival systems: self-preservation in response to a myriad of threats.

Predicting Learning: Understanding the Role of Executive Functions in Children's Belief Revision Using Bayesian Models

Recent studies suggest that learners who are asked to predict the outcome of an event learn more than learners who are asked to evaluate it retrospectively or not at all. One possible explanation for this "prediction boost" is that it helps learners engage metacognitive reasoning skills that may not be spontaneously leveraged, especially for individuals with still-developing executive functions. In this paper, we combined multiple analytic approaches to investigate the potential role of executive functions in elementary school-aged children's science learning. We performed an experiment that investigates children's science learning during a water displacement task where a "prediction boost" had previously been observed-children either made an explicit prediction or evaluated an event post hoc (i.e., postdiction). We then considered the relation of executive function measures and learning, which were collected following the main experiment. Via mixed effects regression models, we found that stronger executive function skills (i.e., stronger inhibition and switching scores) were associated with higher accuracy in Postdiction but not in the Prediction Condition. Using a theory-based Bayesian model, we simulated children's individual performance on the learning task (capturing "belief flexibility"), and compared this "flexibility" to the other measures to understand the relationship between belief revision, executive function, and prediction. Children in the Prediction Condition showed near-ceiling "belief flexibility" scores, which were significantly higher than among children in the Postdiction Condition. We also found a significant correlation between children's executive function measures to our "belief flexibility" parameter, but only for children in the Postdiction Condition. These results indicate that when children provided responses post hoc, they may have required stronger executive function capacities to navigate the learning task. Additionally, these results suggest that the "prediction boost" in children's science learning could be explained by increased metacognitive flexibility in the belief revision process.

Active Inference for Learning and Development in Embodied Neuromorphic Agents

Taking inspiration from humans can help catalyse embodied AI solutions for important real-world applications. Current human-inspired tools include neuromorphic systems and the developmental approach to learning. However, this developmental neurorobotics approach is currently lacking important frameworks for human-like computation and learning. We propose that human-like computation is inherently embodied, with its interface to the world being neuromorphic, and its learning processes operating across different timescales. These constraints necessitate a unified framework: active inference, underpinned by the free energy principle (FEP). Herein, we describe theoretical and empirical support for leveraging this framework in embodied neuromorphic agents with autonomous mental development. We additionally outline current implementation approaches (including toolboxes) and challenges, and we provide suggestions for next steps to catalyse this important field.

Forgetting ourselves in flow: an active inference account of flow states and how we experience ourselves within them

Flow has been described as a state of optimal performance, experienced universally across a broad range of domains: from art to athletics, gaming to writing. However, its phenomenal characteristics can, at first glance, be puzzling. Firstly, individuals in flow supposedly report a loss of self-awareness, even though they perform in a manner which seems to evince their agency and skill. Secondly, flow states are felt to be effortless, despite the prerequisite complexity of the tasks that engender them. In this paper, we unpick these features of flow, as well as others, through the active inference framework, which posits that action and perception are forms of active Bayesian inference directed at sustained self-organisation; i.e., the minimisation of variational free energy. We propose that the phenomenology of flow is rooted in the deployment of high precision weight over (i) the expected sensory consequences of action and (ii) beliefs about how action will sequentially unfold. This computational mechanism thus draws the embodied cognitive system to minimise the ensuing (i.e., expected) free energy through the exploitation of the pragmatic affordances at hand. Furthermore, given the challenging dynamics the flow-inducing situation presents, attention must be wholly focussed on the unfolding task whilst counterfactual planning is restricted, leading to the attested loss of the sense of self-as-object. This involves the inhibition of both the sense of self as a temporally extended object and higher-order, meta-cognitive forms of self-conceptualisation. Nevertheless, we stress that self-awareness is not entirely lost in flow. Rather, it is pre-reflective and bodily. Our approach to bodily-action-centred phenomenology can be applied to similar facets of seemingly agentive experience beyond canonical flow states, providing insights into the mechanisms of so-called selfless experiences, embodied expertise and wellbeing.

An Overview of the Free Energy Principle and Related Research

The free energy principle and its corollary, the active inference framework, serve as theoretical foundations in the domain of neuroscience, explaining the genesis of intelligent behavior. This principle states that the processes of perception, learning, and decision making-within an agent-are all driven by the objective of "minimizing free energy," evincing the following behaviors: learning and employing a generative model of the environment to interpret observations, thereby achieving perception, and selecting actions to maintain a stable preferred state and minimize the uncertainty about the environment, thereby achieving decision making. This fundamental principle can be used to explain how the brain processes perceptual information, learns about the environment, and selects actions. Two pivotal tenets are that the agent employs a generative model for perception and planning and that interaction with the world (and other agents) enhances the performance of the generative model and augments perception. With the evolution of control theory and deep learning tools, agents based on the FEP have been instantiated in various ways across different domains, guiding the design of a multitude of generative models and decision-making algorithms. This letter first introduces the basic concepts of the FEP, followed by its historical development and connections with other theories of intelligence, and then delves into the specific application of the FEP to perception and decision making, encompassing both low-dimensional simple situations and high-dimensional complex situations. It compares the FEP with model-based reinforcement learning to show that the FEP provides a better objective function. We illustrate this using numerical studies of Dreamer3 by adding expected information gain into the standard objective function. In a complementary fashion, existing reinforcement learning, and deep learning algorithms can also help implement the FEP-based agents. Finally, we discuss the various capabilities that agents need to possess in complex environments and state that the FEP can aid agents in acquiring these capabilities.

A social inference model of idealization and devaluation

People often form polarized beliefs, imbuing objects (e.g., themselves or others) with unambiguously positive or negative qualities. In clinical settings, this is referred to as dichotomous thinking or "splitting" and is a feature of several psychiatric disorders. Here, we introduce a Bayesian model of splitting that parameterizes a tendency to rigidly categorize objects as either entirely "Bad" or "Good," rather than to flexibly learn dispositions along a continuous scale. Distinct from the previous descriptive theories, the model makes quantitative predictions about how dichotomous beliefs emerge and are updated in light of new information. Specifically, the model addresses how splitting is context-dependent, yet exhibits stability across time. A key model feature is that phases of devaluation and/or idealization are consolidated by rationally attributing counter-evidence to external factors. For example, when another person is idealized, their less-than-perfect behavior is attributed to unfavorable external circumstances. However, sufficient counter-evidence can trigger switches of polarity, producing bistable dynamics. We show that the model can be fitted to empirical data, to measure individual susceptibility to relational instability. For example, we find that a latent categorical belief that others are "Good" accounts for less changeable, and more certain, character impressions of benevolent as opposed to malevolent others among healthy participants. By comparison, character impressions made by participants with borderline personality disorder reveal significantly higher and more symmetric splitting. The generative framework proposed invites applications for modeling oscillatory relational and affective dynamics in psychotherapeutic contexts. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Neural representation in active inference: Using generative models to interact with-and understand-the lived world

This paper considers neural representation through the lens of active inference, a normative framework for understanding brain function. It delves into how living organisms employ generative models to minimize the discrepancy between predictions and observations (as scored with variational free energy). The ensuing analysis suggests that the brain learns generative models to navigate the world adaptively, not (or not solely) to understand it. Different living organisms may possess an array of generative models, spanning from those that support action-perception cycles to those that underwrite planning and imagination; namely, from explicit models that entail variables for predicting concurrent sensations, like objects, faces, or people-to action-oriented models that predict action outcomes. It then elucidates how generative models and belief dynamics might link to neural representation and the implications of different types of generative models for understanding an agent's cognitive capabilities in relation to its ecological niche. The paper concludes with open questions regarding the evolution of generative models and the development of advanced cognitive abilities-and the gradual transition from pragmatic to detached neural representations. The analysis on offer foregrounds the diverse roles that generative models play in cognitive processes and the evolution of neural representation.

Active Inference and Social Actors: Towards a Neuro-Bio-Social Theory of Brains and Bodies in Their Worlds

Although research including biological concepts and variables has gained more prominence in sociology, progress assimilating the organ of experience, the brain, has been theoretically and technically challenging. Formal uptake and assimilation have thus been slow. Within psychology and neuroscience, the traditional brain, which has made brief appearances in sociological research, is a "bottom-up" processor in which sensory signals are passed up the neural hierarchy where they are eventually cognitively and emotionally processed, after which actions and responses are generated. In this paper, we introduce the Active Inference Framework (AIF), which casts the brain as a Bayesian "inference engine" that tests its "top-down" predictive models against "bottom-up" sensory error streams in its attempts to resolve uncertainty and make the world more predictable. After assembling and presenting key concepts in the AIF, we describe an integrated neuro-bio-social model that prioritizes the microsociological assertion that the scene of action is the situation, wherein brains enculturate. Through such social dynamics, enculturated brains share models of the world with one another, enabling collective realities that disclose the actions afforded in those times and places. We conclude by discussing this neuro-bio-social model within the context of exemplar sociological research areas, including the sociology of stress and health, the sociology of emotions, and cognitive cultural sociology, all areas where the brain has received some degree of recognition and incorporation. In each case, sociological insights that do not fit naturally with the traditional brain model emerge intuitively from the predictive AIF model, further underscoring the interconnections and interdependencies between these areas, while also providing a foundation for a probabilistic sociology.

The empirical status of predictive coding and active inference

Research on predictive processing models has focused largely on two specific algorithmic theories: Predictive Coding for perception and Active Inference for decision-making. While these interconnected theories possess broad explanatory potential, they have only recently begun to receive direct empirical evaluation. Here, we review recent studies of Predictive Coding and Active Inference with a focus on evaluating the degree to which they are empirically supported. For Predictive Coding, we find that existing empirical evidence offers modest support. However, some positive results can also be explained by alternative feedforward (e.g., feature detection-based) models. For Active Inference, most empirical studies have focused on fitting these models to behavior as a means of identifying and explaining individual or group differences. While Active Inference models tend to explain behavioral data reasonably well, there has not been a focus on testing empirical validity of active inference theory per se, which would require formal comparison to other models (e.g., non-Bayesian or model-free reinforcement learning models). This review suggests that, while promising, a number of specific research directions are still necessary to evaluate the empirical adequacy and explanatory power of these algorithms.

Cultivating creativity: predictive brains and the enlightened room problem

How can one conciliate the claim that humans are uncertainty minimizing systems that seek to navigate predictable and familiar environments with the claim that humans can be creative? We call this the Enlightened Room Problem (ERP). The solution, we suggest, lies not (or not only) in the error-minimizing brain but in the environment itself. Creativity emerges from various degrees of interplay between predictive brains and changing environments: ones that repeatedly move the goalposts for our own error-minimizing machinery. By (co)constructing these challenging worlds, we effectively alter and expand the space within which our own prediction engines operate, and that function as 'exploration bubbles' that enable information seeking, uncertainty minimizing minds to penetrate deeper and deeper into artistic, scientific and engineering space. In what follows, we offer a proof of principle for this kind of environmentally led cognitive expansion. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Scientists Invent New Hypotheses, Do Brains?

How are new Bayesian hypotheses generated within the framework of predictive processing? This explanatory framework purports to provide a unified, systematic explanation of cognition by appealing to Bayes rule and hierarchical Bayesian machinery alone. Given that the generation of new hypotheses is fundamental to Bayesian inference, the predictive processing framework faces an important challenge in this regard. By examining several cognitive-level and neurobiological architecture-inspired models of hypothesis generation, we argue that there is an essential difference between the two types of models. Cognitive-level models do not specify how they can be implemented in brains and include structures and assumptions that are external to the predictive processing framework. By contrast, neurobiological architecture-inspired models, which aim to better resemble brain processes, fail to explain important capacities of cognition, such as categorization and few-shot learning. The "scaling-up" challenge for proponents of predictive processing is to explain the relationship between these two types of models using only the theoretical and conceptual machinery of Bayesian inference.

Federated inference and belief sharing

This paper concerns the distributed intelligence or federated inference that emerges under belief-sharing among agents who share a common world-and world model. Imagine, for example, several animals keeping a lookout for predators. Their collective surveillance rests upon being able to communicate their beliefs-about what they see-among themselves. But, how is this possible? Here, we show how all the necessary components arise from minimising free energy. We use numerical studies to simulate the generation, acquisition and emergence of language in synthetic agents. Specifically, we consider inference, learning and selection as minimising the variational free energy of posterior (i.e., Bayesian) beliefs about the states, parameters and structure of generative models, respectively. The common theme-that attends these optimisation processes-is the selection of actions that minimise expected free energy, leading to active inference, learning and model selection (a.k.a., structure learning). We first illustrate the role of communication in resolving uncertainty about the latent states of a partially observed world, on which agents have complementary perspectives. We then consider the acquisition of the requisite language-entailed by a likelihood mapping from an agent's beliefs to their overt expression (e.g., speech)-showing that language can be transmitted across generations by active learning. Finally, we show that language is an emergent property of free energy minimisation, when agents operate within the same econiche. We conclude with a discussion of various perspectives on these phenomena; ranging from cultural niche construction, through federated learning, to the emergence of complexity in ensembles of self-organising systems.

An Introduction to Predictive Processing Models of Perception and Decision-Making

The predictive processing framework includes a broad set of ideas, which might be articulated and developed in a variety of ways, concerning how the brain may leverage predictive models when implementing perception, cognition, decision-making, and motor control. This article provides an up-to-date introduction to the two most influential theories within this framework: predictive coding and active inference. The first half of the paper (Sections 2-5) reviews the evolution of predictive coding, from early ideas about efficient coding in the visual system to a more general model encompassing perception, cognition, and motor control. The theory is characterized in terms of the claims it makes at Marr's computational, algorithmic, and implementation levels of description, and the conceptual and mathematical connections between predictive coding, Bayesian inference, and variational free energy (a quantity jointly evaluating model accuracy and complexity) are explored. The second half of the paper (Sections 6-8) turns to recent theories of active inference. Like predictive coding, active inference models assume that perceptual and learning processes minimize variational free energy as a means of approximating Bayesian inference in a biologically plausible manner. However, these models focus primarily on planning and decision-making processes that predictive coding models were not developed to address. Under active inference, an agent evaluates potential plans (action sequences) based on their expected free energy (a quantity that combines anticipated reward and information gain). The agent is assumed to represent the world as a partially observable Markov decision process with discrete time and discrete states. Current research applications of active inference models are described, including a range of simulation work, as well as studies fitting models to empirical data. The paper concludes by considering future research directions that will be important for further development of both models.

An active inference model of hierarchical action understanding, learning and imitation

We advance a novel active inference model of the cognitive processing that underlies the acquisition of a hierarchical action repertoire and its use for observation, understanding and imitation. We illustrate the model in four simulations of a tennis learner who observes a teacher performing tennis shots, forms hierarchical representations of the observed actions, and imitates them. Our simulations show that the agent's oculomotor activity implements an active information sampling strategy that permits inferring the kinematic aspects of the observed movement, which lie at the lowest level of the action hierarchy. In turn, this low-level kinematic inference supports higher-level inferences about deeper aspects of the observed actions: proximal goals and intentions. Finally, the inferred action representations can steer imitative responses, but interfere with the execution of different actions. Our simulations show that hierarchical active inference provides a unified account of action observation, understanding, learning and imitation and helps explain the neurobiological underpinnings of visuomotor cognition, including the multiple routes for action understanding in the dorsal and ventral streams and mirror mechanisms.

Self-regulation learning as active inference: dynamic causal modeling of an fMRI neurofeedback task

Introduction

Learning to self-regulate brain activity by neurofeedback has been shown to lead to changes in the brain and behavior, with beneficial clinical and non-clinical outcomes. Neurofeedback uses a brain-computer interface to guide participants to change some feature of their brain activity. However, the neural mechanism of self-regulation learning remains unclear, with only 50% of the participants succeeding in achieving it. To bridge this knowledge gap, our study delves into the neural mechanisms of self-regulation learning via neurofeedback and investigates the brain processes associated with successful brain self-regulation.

Methods

We study the neural underpinnings of self-regulation learning by employing dynamical causal modeling (DCM) in conjunction with real-time functional MRI data. The study involved a cohort of 18 participants undergoing neurofeedback training targeting the supplementary motor area. A critical focus was the comparison between top-down hierarchical connectivity models proposed by Active Inference and alternative bottom-up connectivity models like reinforcement learning.

Results

Our analysis revealed a crucial distinction in brain connectivity patterns between successful and non-successful learners. Particularly, successful learners evinced a significantly stronger top-down effective connectivity towards the target area implicated in self-regulation. This heightened top-down network engagement closely resembles the patterns observed in goal-oriented and cognitive control studies, shedding light on the intricate cognitive processes intertwined with self-regulation learning.

Discussion

The findings from our investigation underscore the significance of cognitive mechanisms in the process of self-regulation learning through neurofeedback. The observed stronger top-down effective connectivity in successful learners indicates the involvement of hierarchical cognitive control, which aligns with the tenets of Active Inference. This study contributes to a deeper understanding of the neural dynamics behind successful self-regulation learning and provides insights into the potential cognitive architecture underpinning this process.

A Variational Synthesis of Evolutionary and Developmental Dynamics

This paper introduces a variational formulation of natural selection, paying special attention to the nature of 'things' and the way that different 'kinds' of 'things' are individuated from-and influence-each other. We use the Bayesian mechanics of particular partitions to understand how slow phylogenetic processes constrain-and are constrained by-fast, phenotypic processes. The main result is a formulation of adaptive fitness as a path integral of phenotypic fitness. Paths of least action, at the phenotypic and phylogenetic scales, can then be read as inference and learning processes, respectively. In this view, a phenotype actively infers the state of its econiche under a generative model, whose parameters are learned via natural (Bayesian model) selection. The ensuing variational synthesis features some unexpected aspects. Perhaps the most notable is that it is not possible to describe or model a population of conspecifics per se. Rather, it is necessary to consider populations of distinct natural kinds that influence each other. This paper is limited to a description of the mathematical apparatus and accompanying ideas. Subsequent work will use these methods for simulations and numerical analyses-and identify points of contact with related mathematical formulations of evolution.

(Re)conceptualizing movement behavior in sport as a problem-solving activity

The use of the term problem-solving in relation to movement behavior is an often-broached topic within kinesiology. Here we present a clear rationale for the concept of problem-solving, specifically pertaining to the skilled organization of movement behaviors in sport performance, and the respective processes that underpin it, conceptualized within an ecological dynamics framework. The movement behavior that emerges in sport can be viewed as a problem-solving activity for the athlete, where integrated movement solutions are underpinned by intertwined processes of perception, cognition, and action. This movement problem-solving process becomes functionally aligned with sport performance challenges through a tight coupling to relevant information sources in the environment, which specify affordances offered to the athlete. This ecological perspective can shape our lens on how movements are coordinated and controlled in the context of sport, influencing practical approaches utilized towards facilitating dexterity of athletes. These ideas imply how coaches could set alive movement problems for athletes to solve within practice environments, where they would be required to continuously (re)organize movement system degrees of freedom in relation to dynamic and emergent opportunities, across diverse, complex problems. Through these experiences, athletes could become attuned, intentional, and adaptable, capable of (re)organizing a behavioral fit to performance problems in context-essentially allowing them to become one with the movement problem.

Really radical?

I enjoyed reading this compelling account of Conviction Narrative Theory (CNT). As a theoretical neurobiologist, I recognised - and applauded - the tenets of CNT. My commentary asks whether its claims could be installed into a Bayesian mechanics of decision-making, in a way that would enable theoreticians to model, reproduce and predict decision-making.

Quantifying aberrant approach-avoidance conflict in psychopathology: A review of computational approaches

Making effective decisions during approach-avoidance conflict is critical in daily life. Aberrant decision-making during approach-avoidance conflict is evident in a range of psychological disorders, including anxiety, depression, trauma-related disorders, substance use disorders, and alcohol use disorders. To help clarify etiological pathways and reveal novel intervention targets, clinical research into decision-making is increasingly adopting a computational psychopathology approach. This approach uses mathematical models that can identify specific decision-making related processes that are altered in mental health disorders. In our review, we highlight foundational approach-avoidance conflict research, followed by more in-depth discussion of computational approaches that have been used to model behavior in these tasks. Specifically, we describe the computational models that have been applied to approach-avoidance conflict (e.g., drift-diffusion, active inference, and reinforcement learning models), and provide resources to guide clinical researchers who may be interested in applying computational modeling. Finally, we identify notable gaps in the current literature and potential future directions for computational approaches aimed at identifying mechanisms of approach-avoidance conflict in psychopathology.

To copy or not to copy? That is the question! From chimpanzees to the foundation of human technological culture

A prerequisite for copying innovative behaviour faithfully is the capacity of observers' brains, regarded as 'hierarchically mechanistic minds', to overcome cognitive 'surprisal' (see 2.), by maximising the evidence for their internal models, through active inference. Unlike modern humans, chimpanzees and other great apes show considerable limitations in their ability, or 'Zone of Bounded Surprisal', to overcome cognitive surprisal induced by innovative or unorthodox behaviour that rarely, therefore, is copied precisely or accurately. Most can copy adequately what is within their phenotypically habitual behavioural repertoire, in which technology plays scant part. Widespread intra- and intergenerational social transmission of complex technological innovations is not a hall-mark of great-ape taxa. 3 Ma, precursors of the genus Homo made stone artefacts, and stone-flaking likely was habitual before 2 Ma. After that time, early Homo erectus has left traces of technological innovations, though faithful copying of these and their intra- and intergenerational social transmission were rare before 1 Ma. This likely owed to a cerebral infrastructure of interconnected neuronal systems more limited than ours. Brains were smaller in size than ours, and cerebral neuronal systems ceased to develop when early Homo erectus attained full adult maturity by the mid-teen years, whereas its development continues until our mid-twenties nowadays. Pleistocene Homo underwent remarkable evolutionary adaptation of neurobiological propensities, and cerebral aspects are discussed that, it is proposed here, plausibly, were fundamental for faithful copying, which underpinned social transmission of technologies, cumulative learning, and culture. Here, observers' responses to an innovation are more important for ensuring its transmission than is an innovator's production of it, because, by themselves, the minimal cognitive prerequisites that are needed for encoding and assimilating innovations are insufficient for practical outcomes to accumulate and spread intra- and intergenerationally.

Differentiating between Bayesian parameter learning and structure learning based on behavioural and pupil measures

Within predictive processing two kinds of learning can be distinguished: parameter learning and structure learning. In Bayesian parameter learning, parameters under a specific generative model are continuously being updated in light of new evidence. However, this learning mechanism cannot explain how new parameters are added to a model. Structure learning, unlike parameter learning, makes structural changes to a generative model by altering its causal connections or adding or removing parameters. Whilst these two types of learning have recently been formally differentiated, they have not been empirically distinguished. The aim of this research was to empirically differentiate between parameter learning and structure learning on the basis of how they affect pupil dilation. Participants took part in a within-subject computer-based learning experiment with two phases. In the first phase, participants had to learn the relationship between cues and target stimuli. In the second phase, they had to learn a conditional change in this relationship. Our results show that the learning dynamics were indeed qualitatively different between the two experimental phases, but in the opposite direction as we originally expected. Participants were learning more gradually in the second phase compared to the first phase. This might imply that participants built multiple models from scratch in the first phase (structure learning) before settling on one of these models. In the second phase, participants possibly just needed to update the probability distribution over the model parameters (parameter learning).

The Active Inference Model of Coherence Therapy

Coherence Therapy is an empirically derived experiential psychotherapy based on Psychological Constructivism. Symptoms are viewed as necessary output from an implicit model of the world. The therapist curates experiences and directs attention toward discovering the model. Rendered explicit, the model is juxtaposed with contradictory knowledge driving memory re-consolidation with resolution of the symptom. The Bayesian Brain views perception and action as inferential processes. Prior beliefs are combined in a generative model to explain the hidden causes of sensations through a process of Active Inference. Prior beliefs that are poor fits to the real world are suboptimal. Suboptimal priors with optimal inference produce Bayes Optimal Pathology with behavioral symptoms. The Active Inference Model of Coherence Therapy posits that Coherence Therapy is a dyadic act of therapist guided Active Inference that renders the (probable) hidden causes of a client's behavior conscious. The therapist's sustained attention on the goal of inference helps to overcome memory control bias against retrieval of the affectively charged suboptimal prior. Serial experiences cue memory retrieval and re-instantiation of the physiological/affective state that necessitates production of the symptom in a particular context. As this process continues there is a break in modularity with assimilation into broader networks of experience. Typically, the symptom produced by optimal inference with the suboptimal prior is experienced as unnecessary/inappropriate when taken out of the particular context. The implicit construct has been re-represented and rendered consciously accessible, by a more complex but more accurate model in which the symptom is necessary in some contexts but not others. There is an experience of agency and control in symptom creation, accompanied by the spontaneous production of context appropriate behavior. The capacity for inference has been restored. The Active Inference Model of Coherence Therapy provides a framework for Coherence Therapy as a computational process which can serve as the basis for new therapeutic interventions and experimental designs integrating biological, cognitive, behavioral, and environmental factors.

Structure learning enhances concept formation in synthetic Active Inference agents

Humans display astonishing skill in learning about the environment in which they operate. They assimilate a rich set of affordances and interrelations among different elements in particular contexts, and form flexible abstractions (i.e., concepts) that can be generalised and leveraged with ease. To capture these abilities, we present a deep hierarchical Active Inference model of goal-directed behaviour, and the accompanying belief update schemes implied by maximising model evidence. Using simulations, we elucidate the potential mechanisms that underlie and influence concept learning in a spatial foraging task. We show that the representations formed–as a result of foraging–reflect environmental structure in a way that is enhanced and nuanced by Bayesian model reduction, a special case of structure learning that typifies learning in the absence of new evidence. Synthetic agents learn associations and form concepts about environmental context and configuration as a result of inferential, parametric learning, and structure learning processes–three processes that can produce a diversity of beliefs and belief structures. Furthermore, the ensuing representations reflect symmetries for environments with identical configurations.

Oversampled and undersolved: Depressive rumination from an active inference perspective

Rumination is a widely recognized cognitive deviation in depression. Despite the recognition, researchers have struggled to explain why patients cannot disengage from the process, although it depresses their mood and fails to lead to effective problem-solving. We rethink rumination as repetitive but unsuccessful problem-solving attempts. Appealing to an active inference account, we suggest that adaptive problem-solving is based on the generation, evaluation, and performance of candidate policies that increase an organism's knowledge of its environment. We argue that the problem-solving process is distorted during rumination. Specifically, rumination is understood as engaging in excessive yet unsuccessful oversampling of policy candidates that do not resolve uncertainty. Because candidates are sampled from policies that were selected in states resembling one's current state, "bad" starting points (e.g., depressed mood, physical inactivity) make the problem-solving process vulnerable for generating a ruminative "halting problem". This problem leads to high opportunity costs, learned helplessness and diminished overt behavior. Besides reviewing evidence for the conceptual paths of this model, we discuss its neurophysiological correlates and point towards clinical implications.

Start-Ups as Adaptable Stable Systems Based on Synchronous Business Models

Business models have been a popular topic in research and practice for more than twenty years. During this time, frameworks for formulating business models have been developed, such as the business model canvas. Moreover, different business model frameworks have been proposed for different sectors. Yet, these frameworks have the fundamental shortcoming of not addressing directly and persistently the primary objective of start-ups: to survive in changing environments. The aim of the action research reported in this paper is to overcome that fundamental shortcoming. This is an important topic because the majority of start-ups do not survive. In this paper, first principles for survival in changing environments are related to business models. In particular, action research to reframe start-ups as adaptable stable systems based on synchronous business models is reported. The paper provides three principal contributions. The contribution to business model theory building is to relate survival first principles revealed through natural science research to business models. Reference to first principles highlight that survival depends on maintaining both external adaptability and internal stability through synchronization with changing environments. The second contribution is to business model practice through describing a simple business modeling method that is based on the scientific first principles. The third contribution is to provide an example that bridges the rigor–relevance gap between scientific research and business practice.

A free energy principle for generic quantum systems

The Free Energy Principle (FEP) states that under suitable conditions of weak coupling, random dynamical systems with sufficient degrees of freedom will behave so as to minimize an upper bound, formalized as a variational free energy, on surprisal (a.k.a., self-information). This upper bound can be read as a Bayesian prediction error. Equivalently, its negative is a lower bound on Bayesian model evidence (a.k.a., marginal likelihood). In short, certain random dynamical systems evince a kind of self-evidencing. Here, we reformulate the FEP in the formal setting of spacetime-background free, scale-free quantum information theory. We show how generic quantum systems can be regarded as observers, which with the standard freedom of choice assumption become agents capable of assigning semantics to observational outcomes. We show how such agents minimize Bayesian prediction error in environments characterized by uncertainty, insufficient learning, and quantum contextuality. We show that in its quantum-theoretic formulation, the FEP is asymptotically equivalent to the Principle of Unitarity. Based on these results, we suggest that biological systems employ quantum coherence as a computational resource and - implicitly - as a communication resource. We summarize a number of problems for future research, particularly involving the resources required for classical communication and for detecting and responding to quantum context switches.

Emergence of associative learning in a neuromorphic inference network

OBJECTIVE

In the theoretical framework of predictive coding and active inference, the brain can be viewed as instantiating a rich generative model of the world that predicts incoming sensory data while continuously updating its parameters via minimization of prediction errors. While this theory has been successfully applied to cognitive processes - by modelling the activity of functional neural networks at a mesoscopic scale - the validity of the approach when modelling neurons as an ensemble of inferring agents, in a biologically plausible architecture, remained to be explored.

APPROACH

We modelled a simplified cerebellar circuit with individual neurons acting as Bayesian agents to simulate the classical delayed eyeblink conditioning protocol. Neurons and synapses adjusted their activity to minimize their prediction error, which was used as the network cost function. This cerebellar network was then implemented in hardware by replicating digital neuronal elements via a low-power microcontroller.

MAIN RESULTS

Persistent changes of synaptic strength - that mirrored neurophysiological observations - emerged via local (neurocentric) prediction error minimization, leading to the expression of associative learning. The same paradigm was effectively emulated in low-power hardware showing remarkably efficient performance compared to conventional neuromorphic architectures.

SIGNIFICANCE

These findings show that: i) an ensemble of free energy minimizing neurons - organized in a biological plausible architecture - can recapitulate functional self-organization observed in nature, such as associative plasticity, and ii) a neuromorphic network of inference units can learn unsupervised tasks without embedding predefined learning rules in the circuit, thus providing a potential avenue to a novel form of brain-inspired artificial intelligence.

Organizational Neuroscience of Industrial Adaptive Behavior

Organizational neuroscience is recognized in organizational behavior literature as offering an interpretive framework that can shed new light on existing organizational challenges. In this paper, findings from neuroscience studies concerned with adaptive behavior for ecological fitness are applied to explore industrial adaptive behavior. This is important because many companies are not able to manage dynamics between adaptability and stability. The reported analysis relates business-to-business signaling in competitive environments to three levels of inference. In accordance with neuroscience studies concerned with adaptive behavior, trade-offs between complexity and accuracy in business-to-business signaling and inference are explained. In addition, signaling and inference are related to risks and ambiguities in competitive industrial markets. Overall, the paper provides a comprehensive analysis of industrial adaptive behavior in terms of relevant neuroscience constructs. In doing so, the paper makes a contribution to the field of organizational neuroscience, and to research concerned with industrial adaptive behavior. The reported analysis is relevant to organizational adaptive behavior that involves combining human intelligence and artificial intelligence.

A step-by-step tutorial on active inference and its application to empirical data

The active inference framework, and in particular its recent formulation as a partially observable Markov decision process (POMDP), has gained increasing popularity in recent years as a useful approach for modeling neurocognitive processes. This framework is highly general and flexible in its ability to be customized to model any cognitive process, as well as simulate predicted neuronal responses based on its accompanying neural process theory. It also affords both simulation experiments for proof of principle and behavioral modeling for empirical studies. However, there are limited resources that explain how to build and run these models in practice, which limits their widespread use. Most introductions assume a technical background in programming, mathematics, and machine learning. In this paper we offer a step-by-step tutorial on how to build POMDPs, run simulations using standard MATLAB routines, and fit these models to empirical data. We assume a minimal background in programming and mathematics, thoroughly explain all equations, and provide exemplar scripts that can be customized for both theoretical and empirical studies. Our goal is to provide the reader with the requisite background knowledge and practical tools to apply active inference to their own research. We also provide optional technical sections and multiple appendices, which offer the interested reader additional technical details. This tutorial should provide the reader with all the tools necessary to use these models and to follow emerging advances in active inference research.

From Generative Models to Generative Passages: A Computational Approach to (Neuro) Phenomenology

This paper presents a version of neurophenomenology based on generative modelling techniques developed in computational neuroscience and biology. Our approach can be described as computational phenomenology because it applies methods originally developed in computational modelling to provide a formal model of the descriptions of lived experience in the phenomenological tradition of philosophy (e.g., the work of Edmund Husserl, Maurice Merleau-Ponty, etc.). The first section presents a brief review of the overall project to naturalize phenomenology. The second section presents and evaluates philosophical objections to that project and situates our version of computational phenomenology with respect to these projects. The third section reviews the generative modelling framework. The final section presents our approach in detail. We conclude by discussing how our approach differs from previous attempts to use generative modelling to help understand consciousness. In summary, we describe a version of computational phenomenology which uses generative modelling to construct a computational model of the inferential or interpretive processes that best explain this or that kind of lived experience.

Model Reduction Through Progressive Latent Space Pruning in Deep Active Inference

Although still not fully understood, sleep is known to play an important role in learning and in pruning synaptic connections. From the active inference perspective, this can be cast as learning parameters of a generative model and Bayesian model reduction, respectively. In this article, we show how to reduce dimensionality of the latent space of such a generative model, and hence model complexity, in deep active inference during training through a similar process. While deep active inference uses deep neural networks for state space construction, an issue remains in that the dimensionality of the latent space must be specified beforehand. We investigate two methods that are able to prune the latent space of deep active inference models. The first approach functions similar to sleep and performs model reduction post hoc. The second approach is a novel method which is more similar to reflection, operates during training and displays “aha” moments when the model is able to reduce latent space dimensionality. We show for two well-known simulated environments that model performance is retained in the first approach and only diminishes slightly in the second approach. We also show that reconstructions from a real world example are indistinguishable before and after reduction. We conclude that the most important difference constitutes a trade-off between training time and model performance in terms of accuracy and the ability to generalize, via minimization of model complexity.

How Active Inference Could Help Revolutionise Robotics

Recent advances in neuroscience have characterised brain function using mathematical formalisms and first principles that may be usefully applied elsewhere. In this paper, we explain how active inference-a well-known description of sentient behaviour from neuroscience-can be exploited in robotics. In short, active inference leverages the processes thought to underwrite human behaviour to build effective autonomous systems. These systems show state-of-the-art performance in several robotics settings; we highlight these and explain how this framework may be used to advance robotics.

Synthetic Spatial Foraging With Active Inference in a Geocaching Task

Humans are highly proficient in learning about the environments in which they operate. They form flexible spatial representations of their surroundings that can be leveraged with ease during spatial foraging and navigation. To capture these abilities, we present a deep Active Inference model of goal-directed behavior, and the accompanying belief updating. Active Inference rests upon optimizing Bayesian beliefs to maximize model evidence or marginal likelihood. Bayesian beliefs are probability distributions over the causes of observable outcomes. These causes include an agent's actions, which enables one to treat planning as inference. We use simulations of a geocaching task to elucidate the belief updating-that underwrites spatial foraging-and the associated behavioral and neurophysiological responses. In a geocaching task, the aim is to find hidden objects in the environment using spatial coordinates. Here, synthetic agents learn about the environment via inference and learning (e.g., learning about the likelihoods of outcomes given latent states) to reach a target location, and then forage locally to discover the hidden object that offers clues for the next location.

Synchronous Generative Development amidst Situated Entropy

The Sustainable Development Goals have been criticized for not providing sufficient balance between human well-being and environmental well-being. By contrast, joint agent-environment systems theory is focused on reciprocal synchronous generative development. The purpose of this paper is to extend this theory towards practical application in sustainable development projects. This purpose is fulfilled through three interrelated contributions. First, a practitioner description of the theory is provided. Then, the theory is extended through reference to research concerned with multilevel pragmatics, competing signals, commitment processes, technological mediation, and psychomotor functioning. In addition, the theory is related to human-driven biosocial-technical innovation through the example of digital twins for agroecological urban farming. Digital twins being digital models that mirror physical processes; that are connected to physical processes through, for example, sensors and actuators; and which carry out analyses of physical processes in order to improve their performance. Together, these contributions extend extant theory towards application for synchronous generative development that balances human well-being and environmental well-being. However, the practical examples in the paper indicate that counterproductive complexity can arise from situated entropy amidst biosocial-technical innovations: even when those innovations are compatible with synchronous generative development.

Accessing Active Inference Theory through Its Implicit and Deliberative Practice in Human Organizations

Active inference theory (AIT) is a corollary of the free-energy principle, which formalizes cognition of living system’s autopoietic organization. AIT comprises specialist terminology and mathematics used in theoretical neurobiology. Yet, active inference is common practice in human organizations, such as private companies, public institutions, and not-for-profits. Active inference encompasses three interrelated types of actions, which are carried out to minimize uncertainty about how organizations will survive. The three types of action are updating work beliefs, shifting work attention, and/or changing how work is performed. Accordingly, an alternative starting point for grasping active inference, rather than trying to understand AIT specialist terminology and mathematics, is to reflect upon lived experience. In other words, grasping active inference through autoethnographic research. In this short communication paper, accessing AIT through autoethnography is explained in terms of active inference in existing organizational practice (implicit active inference), new organizational methodologies that are informed by AIT (deliberative active inference), and combining implicit and deliberative active inference. In addition, these autoethnographic options for grasping AIT are related to generative learning.

Active Inferants: An Active Inference Framework for Ant Colony Behavior

In this paper, we introduce an active inference model of ant colony foraging behavior, and implement the model in a series of in silico experiments. Active inference is a multiscale approach to behavioral modeling that is being applied across settings in theoretical biology and ethology. The ant colony is a classic case system in the function of distributed systems in terms of stigmergic decision-making and information sharing. Here we specify and simulate a Markov decision process (MDP) model for ant colony foraging. We investigate a well-known paradigm from laboratory ant colony behavioral experiments, the alternating T-maze paradigm, to illustrate the ability of the model to recover basic colony phenomena such as trail formation after food location discovery. We conclude by outlining how the active inference ant colony foraging behavioral model can be extended and situated within a nested multiscale framework and systems approaches to biology more generally.

Simulating the computational mechanisms of cognitive and behavioral psychotherapeutic interventions: insights from active inference

Cognitive-behavioral therapy (CBT) leverages interactions between thoughts, feelings, and behaviors. To deepen understanding of these interactions, we present a computational (active inference) model of CBT that allows formal simulations of interactions between cognitive interventions (i.e., cognitive restructuring) and behavioral interventions (i.e., exposure) in producing adaptive behavior change (i.e., reducing maladaptive avoidance behavior). Using spider phobia as a concrete example of maladaptive avoidance more generally, we show simulations indicating that when conscious beliefs about safety/danger have strong interactions with affective/behavioral outcomes, behavioral change during exposure therapy is mediated by changes in these beliefs, preventing generalization. In contrast, when these interactions are weakened, and cognitive restructuring only induces belief uncertainty (as opposed to strong safety beliefs), behavior change leads to generalized learning (i.e., "over-writing" the implicit beliefs about action-outcome mappings that directly produce avoidance). The individual is therefore equipped to face any new context, safe or dangerous, remaining in a content state without the need for avoidance behavior-increasing resilience from a CBT perspective. These results show how the same changes in behavior during CBT can be due to distinct underlying mechanisms; they predict lower rates of relapse when cognitive interventions focus on inducing uncertainty and on reducing the effects of automatic negative thoughts on behavior.

Neural Dynamics under Active Inference: Plausibility and Efficiency of Information Processing

Active inference is a normative framework for explaining behaviour under the free energy principle-a theory of self-organisation originating in neuroscience. It specifies neuronal dynamics for state-estimation in terms of a descent on (variational) free energy-a measure of the fit between an internal (generative) model and sensory observations. The free energy gradient is a prediction error-plausibly encoded in the average membrane potentials of neuronal populations. Conversely, the expected probability of a state can be expressed in terms of neuronal firing rates. We show that this is consistent with current models of neuronal dynamics and establish face validity by synthesising plausible electrophysiological responses. We then show that these neuronal dynamics approximate natural gradient descent, a well-known optimisation algorithm from information geometry that follows the steepest descent of the objective in information space. We compare the information length of belief updating in both schemes, a measure of the distance travelled in information space that has a direct interpretation in terms of metabolic cost. We show that neural dynamics under active inference are metabolically efficient and suggest that neural representations in biological agents may evolve by approximating steepest descent in information space towards the point of optimal inference.