Active Inference, Curiosity and Insight

To know or not to know? Curiosity and the value of prospective information in animals

Humans and other animals often seek instrumental information to strategically improve their decisions in the present. Our curiosity also leads us to acquire non-instrumental information that is not immediately useful but can be encoded in memory and stored for use in the future by means of episodic recall. Despite its adaptive benefits and central role in human cognition, questions remain about the cognitive mechanisms and evolutionary origins that underpin curiosity. Here, we comparatively review recent empirical studies that some authors have suggested reflects curiosity in nonhuman animals. We focus on findings from laboratory tasks in which individuals can choose to gain advanced information about uncertain future outcomes, even though the information cannot be used to increase future rewards and is often costly. We explore the prevalence of preferences in these tasks across animals, discuss the theoretical advances that they have promoted, and outline some limitations in contemporary research. We also discuss several features of human curiosity that can guide future empirical research aimed at characterising and understanding curiosity in animals. Though the prevalence of curiosity in animals is actively debated, we surmise that investigating behavioural candidates for curiosity-motivated behaviour in a broader range of species and contexts, should help promote theoretical advances in our understanding of cognitive principles and evolutionary pressures that support curiosity-driven behaviour.

SURVIVAL IN A WORLD OF COMPLEX DANGERS

How do nomadic animals, including humans, continually adapt to the dual challenge of recurrent threats and the seemingly limitless and unpredictable, number of dangers in the natural world? We posit that human defensive behaviors are dynamically constructed to facilitate survival in capricious and itinerant environments. We first hypothesize that internal and external states result in state constructions that combine to form a meta-representation. When a threat is detected, it triggers the action construction. Action constructions are formed through two contiguous survival strategies: generalization strategies, which are used when encountering new threats and ecologies. Generalization strategies are associated with cognitive representations that have high dimensionality and which furnish flexible psychological constructs, including relations between threats, and imagination, and which converge through the construction of defensive states. We posit that generalization strategies drive 'explorative' behaviors including information seeking, where the goal is to increase knowledge that can be used to mitigate current and future threats. Conversely, specialization strategies entail lower dimensional representations, which underpin specialized, sometimes reflexive, or habitual survival behaviors that are 'exploitative'. Together, this captures a central adaptive feature of human survival systems: self-preservation in response to myriad threats.

Meeting the multidimensional self: fostering selfhood at the interface of Creative Arts Therapies and neuroscience

Intriguing explorations at the intersection of the fields of neuroscience and psychology are driven by the quest to understand the neural underpinnings of "the self" and their psychotherapeutic implications. These translational efforts pertain to the unique Creative Arts Therapies (CATs) and the attributes and value of the self-related processes they offer. The self is considered as a multi-layered complex construct, comprising bodily and mental constituents, subjective-objective perspectives, spatial and temporal dimensions. Neuroscience research, mostly functional brain imaging, has proposed cogent models of the constitution, development and experience of the self, elucidating how the multiple dimensions of the self are supported by integrated hierarchical brain processes. The psychotherapeutic use of the art-forms, generating aesthetic experiences and creative processes, touch upon and connect the various layers of self-experience, nurturing the sense of self. The present conceptual analysis will describe and interweave the neural mechanisms and neural network configuration suggested to lie at the core of the ongoing self-experience, its deviations in psychopathology, and implications regarding the psychotherapeutic use of the arts. The well-established, parsimonious and neurobiologically plausible predictive processing account of brain-function will be discussed with regard to selfhood and consciousness. The epistemic affordance of the experiential CATs will further be portrayed, enabling and facilitating the creation of updated self-models of the body in the world. The neuropsychological impact of the relational therapeutic encounter will be delineated, acknowledging the intersubjective brain synchronization through communicative verbal and non-verbal means and aesthetic experiences. The recognition and assimilation of neuroscientific, phenomenological and clinical perspectives concerning the nested dimensionality of the self, ground the relational therapeutic process and the neuroplastic modulations that CATs have to offer on the premise of fostering, shaping and integrating selfhood.

Nature heals: An informational entropy account of self-organization and change in field psychotherapy

This paper reviews biophysical models of psychotherapeutic change based on synergetics and the free energy principle. These models suggest that introducing sensory surprise into the patient-therapist system can lead to self-organization and the formation of new attractor states, disrupting entrenched patterns of thoughts, emotions, and behaviours. We propose that the therapist can facilitate this process by cultivating epistemic trust and modulating embodied attention to allow surprising affective states to enter shared awareness. Transient increases in free energy enable the update of generative models, expanding the range of experiences available within the patient-therapist phenomenal field. We hypothesize that patterns of disorganization at behavioural and physiological levels, indexed by increased entropy, complexity, and lower determinism, are key markers and predictors of psychotherapeutic gains. Future research should investigate how the therapist's openness to novelty shapes therapeutic outcomes.

Predicting Learning: Understanding the Role of Executive Functions in Children's Belief Revision Using Bayesian Models

Recent studies suggest that learners who are asked to predict the outcome of an event learn more than learners who are asked to evaluate it retrospectively or not at all. One possible explanation for this "prediction boost" is that it helps learners engage metacognitive reasoning skills that may not be spontaneously leveraged, especially for individuals with still-developing executive functions. In this paper, we combined multiple analytic approaches to investigate the potential role of executive functions in elementary school-aged children's science learning. We performed an experiment that investigates children's science learning during a water displacement task where a "prediction boost" had previously been observed-children either made an explicit prediction or evaluated an event post hoc (i.e., postdiction). We then considered the relation of executive function measures and learning, which were collected following the main experiment. Via mixed effects regression models, we found that stronger executive function skills (i.e., stronger inhibition and switching scores) were associated with higher accuracy in Postdiction but not in the Prediction Condition. Using a theory-based Bayesian model, we simulated children's individual performance on the learning task (capturing "belief flexibility"), and compared this "flexibility" to the other measures to understand the relationship between belief revision, executive function, and prediction. Children in the Prediction Condition showed near-ceiling "belief flexibility" scores, which were significantly higher than among children in the Postdiction Condition. We also found a significant correlation between children's executive function measures to our "belief flexibility" parameter, but only for children in the Postdiction Condition. These results indicate that when children provided responses post hoc, they may have required stronger executive function capacities to navigate the learning task. Additionally, these results suggest that the "prediction boost" in children's science learning could be explained by increased metacognitive flexibility in the belief revision process.

An Integrated theory of false insights and beliefs under psychedelics

Psychedelics are recognised for their potential to re-orient beliefs. We propose a model of how psychedelics can, in some cases, lead to false insights and thus false beliefs. We first review experimental work on laboratory-based false insights and false memories. We then connect this to insights and belief formation under psychedelics using the active inference framework. We propose that subjective and brain-based alterations caused by psychedelics increases the quantity and subjective intensity of insights and thence beliefs, including false ones. We offer directions for future research in minimising the risk of false and potentially harmful beliefs arising from psychedelics. Ultimately, knowing how psychedelics may facilitate false insights and beliefs is crucial if we are to optimally leverage their therapeutic potential.

Keeping an Eye Out for Change: Anxiety Disrupts Adaptive Resolution of Policy Uncertainty

BACKGROUND

Human learning unfolds under uncertainty. Uncertainty is heterogeneous with different forms exerting distinct influences on learning. While one can be uncertain about what to do to maximize rewarding outcomes, known as policy uncertainty, one can also be uncertain about general world knowledge, known as epistemic uncertainty (EU). In complex and naturalistic environments such as the social world, adaptive learning may hinge on striking a balance between attending to and resolving each type of uncertainty. Prior work illustrates that people with anxiety-those with increased threat and uncertainty sensitivity-learn less from aversive outcomes, particularly as outcomes become more uncertain. How does a learner adaptively trade-off between attending to these distinct sources of uncertainty to successfully learn about their social environment?

METHODS

We developed a novel eye-tracking method to capture highly granular estimates of policy uncertainty and EU based on gaze patterns and pupil diameter (a physiological estimate of arousal).

RESULTS

These empirically derived uncertainty measures revealed that humans (N = 94) flexibly switched between resolving policy uncertainty and EU to adaptively learn about which individuals can be trusted and which should be avoided. However, those with increased anxiety (n = 49) did not flexibly switch between resolving policy uncertainty and EU and instead expressed less uncertainty overall.

CONCLUSIONS

Combining modeling and eye-tracking techniques, we show that altered learning in people with anxiety emerged from an insensitivity to policy uncertainty and rigid choice policies, leading to maladaptive behaviors with untrustworthy people.

'Snakes and ladders' in paleoanthropology: From cognitive surprise to skillfulness a million years ago

A paradigmatic account may suffice to explain behavioral evolution in early Homo. We propose a parsimonious account that (1) could explain a particular, frequently-encountered, archeological outcome of behavior in early Homo - namely, the fashioning of a Paleolithic stone 'handaxe' - from a biological theoretic perspective informed by the free energy principle (FEP); and that (2) regards instances of the outcome as postdictive or retrodictive, circumstantial corroboration. Our proposal considers humankind evolving as a self-organizing biological ecosystem at a geological time-scale. We offer a narrative treatment of this self-organization in terms of the FEP. Specifically, we indicate how 'cognitive surprises' could underwrite an evolving propensity in early Homo to express sporadic unorthodox or anomalous behavior. This co-evolutionary propensity has left us a legacy of Paleolithic artifacts that is reminiscent of a 'snakes and ladders' board game of appearances, disappearances, and reappearances of particular archeological traces of Paleolithic behavior. When detected in the Early and Middle Pleistocene record, anthropologists and archeologists often imagine evidence of unusual or novel behavior in terms of early humankind ascending the rungs of a figurative phylogenetic 'ladder' - as if these corresponded to progressive evolution of cognitive abilities that enabled incremental achievements of increasingly innovative technical prowess, culminating in the cognitive ascendancy of Homo sapiens. The conjecture overlooks a plausible likelihood that behavior by an individual who was atypical among her conspecifics could have been disregarded in a community of Hominina (for definition see Appendix 1) that failed to recognize, imagine, or articulate potential advantages of adopting hitherto unorthodox behavior. Such failure, as well as diverse fortuitous demographic accidents, would cause exceptional personal behavior to be ignored and hence unremembered. It could disappear by a pitfall, down a 'snake', as it were, in the figurative evolutionary board game; thereby causing a discontinuity in the evolution of human behavior that presents like an evolutionary puzzle. The puzzle discomforts some paleoanthropologists trained in the natural and life sciences. They often dismiss it, explaining it away with such self-justifying conjectures as that, maybe, separate paleospecies of Homo differentially possessed different cognitive abilities, which, supposedly, could account for the presence or absence in the Pleistocene archeological record of traces of this or that behavioral outcome or skill. We argue that an alternative perspective - that inherits from the FEP and an individual's 'active inference' about its surroundings and of its own responses - affords a prosaic, deflationary, and parsimonious way to account for appearances, disappearances, and reappearances of particular behavioral outcomes and skills of early humankind.

Meaningfulness and attachment: what dreams, psychosis and psychedelic states tell us about our need for connection

The human need to find meaning in life and the human need for connection may be two sides of the same coin, a coin forged in the developmental crucible of attachment. Our need for meaningfulness can be traced to our developmental need for connection in the attachment relationship. The free energy principle dictates that in order to resist a natural tendency towards disorder self-organizing systems must generate models that predict the hidden causes of phenomenal experience. In other words, they must make sense of things. In both an evolutionary and ontogenetic sense, the narrative self develops as a model that makes sense of experience. However, the self-model skews the interpretation of experience towards that which is predictable, or already "known." One may say it causes us to "take things personally." Meaning is felt more acutely when defenses are compromised, when the narrative self is offline. This enables meaning-making that is less egocentrically motivated. Dreams, psychosis, and psychedelic states offer glimpses of how we make sense of things absent a coherent narrative self. This has implications for the way we understand such states, and lays bare the powerful reach of attachment in shaping what we experience as meaningful.

A transdisciplinary view on curiosity beyond linguistic humans: animals, infants, and artificial intelligence

Curiosity is a core driver for life-long learning, problem-solving and decision-making. In a broad sense, curiosity is defined as the intrinsically motivated acquisition of novel information. Despite a decades-long history of curiosity research and the earliest human theories arising from studies of laboratory rodents, curiosity has mainly been considered in two camps: 'linguistic human' and 'other'. This is despite psychology being heritable, and there are many continuities in cognitive capacities across the animal kingdom. Boundary-pushing cross-disciplinary debates on curiosity are lacking, and the relative exclusion of pre-linguistic infants and non-human animals has led to a scientific impasse which more broadly impedes the development of artificially intelligent systems modelled on curiosity in natural agents. In this review, we synthesize literature across multiple disciplines that have studied curiosity in non-verbal systems. By highlighting how similar findings have been produced across the separate disciplines of animal behaviour, developmental psychology, neuroscience, and computational cognition, we discuss how this can be used to advance our understanding of curiosity. We propose, for the first time, how features of curiosity could be quantified and therefore studied more operationally across systems: across different species, developmental stages, and natural or artificial agents.

Metacognition of curiosity: People underestimate the seductive lure of non-instrumental information

Curiosity - the desire to seek information - is fundamental for learning and performance. Studies on curiosity have shown that people are intrinsically motivated to seek information even if it does not bring an immediate tangible benefit (i.e., non-instrumental information), but little is known as to whether people have the metacognitive capability to accurately monitor their motivation for seeking information. We examined whether people can accurately predict their own non-instrumental information-seeking behavior. Across six experiments (Experiments 1A-1E and 2, total N = 579), participants predicted that they would engage in information-seeking behavior less frequently than they actually did, suggesting that people tend to underestimate the motivational lure of curiosity. Overall, there was no consistent statistical evidence that this underestimation was altered by contextual factors (e.g., the cost to seek information). These results were consistent with the theoretical account that it is difficult for people to make sense of the internally rewarding value of information in advance.

Prediction error minimization as a common computational principle for curiosity and creativity

We propose expanding the authors' shared novelty-seeking basis for creativity and curiosity by emphasizing an underlying computational principle: Minimizing prediction errors (mismatch between predictions and incoming data). Curiosity is tied to the anticipation of minimizing prediction errors through future, novel information, whereas creative AHA moments are connected to the actual minimization of prediction errors through current, novel information.

Surprise!-Clarifying the link between insight and prediction error

The AHA experience, a moment of deep understanding during insightful problem-solving involving feelings of certainty, pleasure, and surprise, has captivated psychologists for more than a century. Recently, a new theoretical framework has proposed a link between the AHA experience and prediction error (PE), a popular concept in decision-making and reinforcement learning. This framework suggests that participants maintain a meta-cognitive prediction about the time it takes to solve a problem and the AHA experience arises when the problem is solved earlier than expected, resulting in a meta-cognitive PE. In our preregistered online study, we delved deeper into this idea, investigating whether prediction errors also pertain to participants' predictions regarding the solvability of the problem itself, and which dimension of the AHA experience aligns with the meta-cognitive PE. Utilizing verbal insight problems, we found a positive association between the AHA experience and the meta-cognitive PE, specifically in regards to problem solvability. Specifically, the element of surprise, a critical AHA dimension, emerged as a key indicator of the meta-cognitive PE, while other dimensions-such as pleasure, certainty, and suddenness-showed no signs for similar relationships, with suddenness exhibiting a negative correlation with meta-cognitive PE. This new finding provides further evidence that aspects of the AHA experience, surprise in particular, correspond to a meta-cognitive PE. The finding also underscores the multifaceted nature of this phenomenon, linking insights with learning theories and enhancing our understanding of this intriguing phenomenon.

Curiosity and the dynamics of optimal exploration

What drives our curiosity remains an elusive and hotly debated issue, with multiple hypotheses proposed but a cohesive account yet to be established. This review discusses traditional and emergent theories that frame curiosity as a desire to know and a drive to learn, respectively. We adopt a model-based approach that maps the temporal dynamics of various factors underlying curiosity-based exploration, such as uncertainty, information gain, and learning progress. In so doing, we identify the limitations of past theories and posit an integrated account that harnesses their strengths in describing curiosity as a tool for optimal environmental exploration. In our unified account, curiosity serves as a 'common currency' for exploration, which must be balanced with other drives such as safety and hunger to achieve efficient action.

Imagination vs. routines: festive time, weekly time, and the predictive brain

This paper examines the relationship between societal structures shaped by traditions, norms, laws, and customs, and creative expressions in arts and media through the lens of the predictive coding framework in cognitive science. The article proposes that both dimensions of culture can be viewed as adaptations designed to enhance and train the brain's predictive abilities in the social domain. Traditions, norms, laws, and customs foster shared predictions and expectations among individuals, thereby reducing uncertainty in social environments. On the other hand, arts and media expose us to simulated experiences that explore alternative social realities, allowing the predictive machinery of the brain to hone its skills through exposure to a wider array of potentially relevant social circumstances and scenarios. We first review key principles of predictive coding and active inference, and then explore the rationale of cultural traditions and artistic culture in this perspective. Finally, we draw parallels between institutionalized normative habits that stabilize social worlds and creative and imaginative acts that temporarily subvert established conventions to inject variability.

An Overview of the Free Energy Principle and Related Research

The free energy principle and its corollary, the active inference framework, serve as theoretical foundations in the domain of neuroscience, explaining the genesis of intelligent behavior. This principle states that the processes of perception, learning, and decision making-within an agent-are all driven by the objective of "minimizing free energy," evincing the following behaviors: learning and employing a generative model of the environment to interpret observations, thereby achieving perception, and selecting actions to maintain a stable preferred state and minimize the uncertainty about the environment, thereby achieving decision making. This fundamental principle can be used to explain how the brain processes perceptual information, learns about the environment, and selects actions. Two pivotal tenets are that the agent employs a generative model for perception and planning and that interaction with the world (and other agents) enhances the performance of the generative model and augments perception. With the evolution of control theory and deep learning tools, agents based on the FEP have been instantiated in various ways across different domains, guiding the design of a multitude of generative models and decision-making algorithms. This letter first introduces the basic concepts of the FEP, followed by its historical development and connections with other theories of intelligence, and then delves into the specific application of the FEP to perception and decision making, encompassing both low-dimensional simple situations and high-dimensional complex situations. It compares the FEP with model-based reinforcement learning to show that the FEP provides a better objective function. We illustrate this using numerical studies of Dreamer3 by adding expected information gain into the standard objective function. In a complementary fashion, existing reinforcement learning, and deep learning algorithms can also help implement the FEP-based agents. Finally, we discuss the various capabilities that agents need to possess in complex environments and state that the FEP can aid agents in acquiring these capabilities.

Neural representation in active inference: Using generative models to interact with-and understand-the lived world

This paper considers neural representation through the lens of active inference, a normative framework for understanding brain function. It delves into how living organisms employ generative models to minimize the discrepancy between predictions and observations (as scored with variational free energy). The ensuing analysis suggests that the brain learns generative models to navigate the world adaptively, not (or not solely) to understand it. Different living organisms may possess an array of generative models, spanning from those that support action-perception cycles to those that underwrite planning and imagination; namely, from explicit models that entail variables for predicting concurrent sensations, like objects, faces, or people-to action-oriented models that predict action outcomes. It then elucidates how generative models and belief dynamics might link to neural representation and the implications of different types of generative models for understanding an agent's cognitive capabilities in relation to its ecological niche. The paper concludes with open questions regarding the evolution of generative models and the development of advanced cognitive abilities-and the gradual transition from pragmatic to detached neural representations. The analysis on offer foregrounds the diverse roles that generative models play in cognitive processes and the evolution of neural representation.

Co-representation of Functional Brain Networks Is Shaped by Cortical Myeloarchitecture and Reveals Individual Behavioral Ability

Large-scale functional networks are spatially distributed in the human brain. Despite recent progress in differentiating their functional roles, how the brain navigates the spatial coordination among them and the biological relevance of this coordination is still not fully understood. Capitalizing on canonical individualized networks derived from functional MRI data, we proposed a new concept, that is, co-representation of functional brain networks, to delineate the spatial coordination among them. To further quantify the co-representation pattern, we defined two indexes, that is, the co-representation specificity (CoRS) and intensity (CoRI), for separately measuring the extent of specific and average expression of functional networks at each brain location by using the data from both sexes. We found that the identified pattern of co-representation was anchored by cortical regions with three types of cytoarchitectural classes along a sensory-fugal axis, including, at the first end, primary (idiotypic) regions showing high CoRS, at the second end, heteromodal regions showing low CoRS and high CoRI, at the third end, paralimbic regions showing low CoRI. Importantly, we demonstrated the critical role of myeloarchitecture in sculpting the spatial distribution of co-representation by assessing the association with the myelin-related neuroanatomical and transcriptomic profiles. Furthermore, the significance of manifesting the co-representation was revealed in its prediction of individual behavioral ability. Our findings indicated that the spatial coordination among functional networks was built upon an anatomically configured blueprint to facilitate neural information processing, while advancing our understanding of the topographical organization of the brain by emphasizing the assembly of functional networks.

Active inference as a theory of sentient behavior

This review paper offers an overview of the history and future of active inference-a unifying perspective on action and perception. Active inference is based upon the idea that sentient behavior depends upon our brains' implicit use of internal models to predict, infer, and direct action. Our focus is upon the conceptual roots and development of this theory of (basic) sentience and does not follow a rigid chronological narrative. We trace the evolution from Helmholtzian ideas on unconscious inference, through to a contemporary understanding of action and perception. In doing so, we touch upon related perspectives, the neural underpinnings of active inference, and the opportunities for future development. Key steps in this development include the formulation of predictive coding models and related theories of neuronal message passing, the use of sequential models for planning and policy optimization, and the importance of hierarchical (temporally) deep internal (i.e., generative or world) models. Active inference has been used to account for aspects of anatomy and neurophysiology, to offer theories of psychopathology in terms of aberrant precision control, and to unify extant psychological theories. We anticipate further development in all these areas and note the exciting early work applying active inference beyond neuroscience. This suggests a future not just in biology, but in robotics, machine learning, and artificial intelligence.

Generating meaning: active inference and the scope and limits of passive AI

Prominent accounts of sentient behavior depict brains as generative models of organismic interaction with the world, evincing intriguing similarities with current advances in generative artificial intelligence (AI). However, because they contend with the control of purposive, life-sustaining sensorimotor interactions, the generative models of living organisms are inextricably anchored to the body and world. Unlike the passive models learned by generative AI systems, they must capture and control the sensory consequences of action. This allows embodied agents to intervene upon their worlds in ways that constantly put their best models to the test, thus providing a solid bedrock that is - we argue - essential to the development of genuine understanding. We review the resulting implications and consider future directions for generative AI.

Learning beyond sensations: How dreams organize neuronal representations

Semantic representations in higher sensory cortices form the basis for robust, yet flexible behavior. These representations are acquired over the course of development in an unsupervised fashion and continuously maintained over an organism's lifespan. Predictive processing theories propose that these representations emerge from predicting or reconstructing sensory inputs. However, brains are known to generate virtual experiences, such as during imagination and dreaming, that go beyond previously experienced inputs. Here, we suggest that virtual experiences may be just as relevant as actual sensory inputs in shaping cortical representations. In particular, we discuss two complementary learning principles that organize representations through the generation of virtual experiences. First, "adversarial dreaming" proposes that creative dreams support a cortical implementation of adversarial learning in which feedback and feedforward pathways engage in a productive game of trying to fool each other. Second, "contrastive dreaming" proposes that the invariance of neuronal representations to irrelevant factors of variation is acquired by trying to map similar virtual experiences together via a contrastive learning process. These principles are compatible with known cortical structure and dynamics and the phenomenology of sleep thus providing promising directions to explain cortical learning beyond the classical predictive processing paradigm.

Cultivating creativity: predictive brains and the enlightened room problem

How can one conciliate the claim that humans are uncertainty minimizing systems that seek to navigate predictable and familiar environments with the claim that humans can be creative? We call this the Enlightened Room Problem (ERP). The solution, we suggest, lies not (or not only) in the error-minimizing brain but in the environment itself. Creativity emerges from various degrees of interplay between predictive brains and changing environments: ones that repeatedly move the goalposts for our own error-minimizing machinery. By (co)constructing these challenging worlds, we effectively alter and expand the space within which our own prediction engines operate, and that function as 'exploration bubbles' that enable information seeking, uncertainty minimizing minds to penetrate deeper and deeper into artistic, scientific and engineering space. In what follows, we offer a proof of principle for this kind of environmentally led cognitive expansion. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

The aesthetic valve: how aesthetic appreciation may switch emotional states from anxiety to curiosity

Pursuing new knowledge in the entropic environment is pivotal for survival. However, dealing with uncertainty is a costly challenge for the agent surrounded by the stochastic sensory world, giving rise to different epistemic emotions, such as curiosity and anxiety. We recently proposed that aesthetic appreciation may have the role of associating pleasant feedback with the update of predictive representations. According to this idea, aesthetic appreciation and its associated rewarding feeling could drive people to seek new knowledge over anxiety. However, the relationship between aesthetic appreciation, curiosity, and anxiety has been still under-examined in the literature. Here, we explore the relationship between these epistemic emotions in a series of three experiments. In study 1, we examined whether music-induced aesthetic appreciation would influence curiosity in a gambling task. In studies 2a and 2b, we explore the relationship between music-induced aesthetic appreciation and anxiety state. Overall, aesthetic appreciation promoted curiosity-driven behaviour while it was negatively associated with anxiety. These results were consistent with the idea that aesthetic appreciation could act as a 'valve', prompting the individual to perceive curiosity (i.e. to consider novelty as a valuable opportunity to acquire new knowledge) rather than anxiety (i.e. to consider novelty as a risk to be avoided). This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Aesthetics and predictive processing: grounds and prospects of a fruitful encounter

In the last few years, a remarkable convergence of interests and results has emerged between scholars interested in the arts and aesthetics from a variety of perspectives and cognitive scientists studying the mind and brain within the predictive processing (PP) framework. This convergence has so far proven fruitful for both sides: while PP is increasingly adopted as a framework for understanding aesthetic phenomena, the arts and aesthetics, examined under the lens of PP, are starting to be seen as important windows into our mental functioning. The result is a vast and fast-growing research programme that promises to deliver important insights into our aesthetic encounters as well as a wide range of psychological phenomena of general interest. Here, we present this developing research programme, describing its grounds and highlighting its prospects. We start by clarifying how the study of the arts and aesthetics encounters the PP picture of mental functioning (§1). We then go on to outline the prospects of this encounter for the fields involved: philosophy and history of art (§2), psychology of aesthetics and neuroaesthetics (§3) and psychology and neuroscience more generally (§4). The upshot is an ambitious but well-defined framework within which aesthetics and cognitive science can partner up to illuminate crucial aspects of the human mind. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Fluency, prediction and motivation: how processing dynamics, expectations and epistemic goals shape aesthetic judgements

What psychological mechanisms underlie aesthetic judgements? An influential account known as the Hedonic Marking of Fluency, later developed into a Processing Fluency Theory of Aesthetic Pleasure, posits that ease of processing elicits positive feelings and thus enhances stimulus evaluations. However, the theory faces empirical and conceptual challenges. In this paper, we extend it by integrating insights from predictive processing frameworks (PPF) and the epistemic motivation model (EMM). We propose four extensions. First, fluency of a stimulus depends on perceivers' expectations-their internal model of the world. Second, perceivers also form expectations about fluency itself and thus can experience surprising fluency. These expectations can come from the individual's history, their current task and their environment. Third, perceivers can value fluency but also disfluency, reflecting their non-directional epistemic goals. Fourth, perceivers also have directional epistemic goals, preferring specific conclusions or belief content. Consequently, affective reactions depend on whether the stimulus satisfies those goals. These directional epistemic goals may override concerns about fluency or change the value of fluency associated with specific content. We review supporting evidence and introduce novel predictions. By integrating insights from PPF and EMM, our framework can better capture established fluency effects and highlights their limitations and extensions. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Path integrals, particular kinds, and strange things

This paper describes a path integral formulation of the free energy principle. The ensuing account expresses the paths or trajectories that a particle takes as it evolves over time. The main results are a method or principle of least action that can be used to emulate the behaviour of particles in open exchange with their external milieu. Particles are defined by a particular partition, in which internal states are individuated from external states by active and sensory blanket states. The variational principle at hand allows one to interpret internal dynamics-of certain kinds of particles-as inferring external states that are hidden behind blanket states. We consider different kinds of particles, and to what extent they can be imbued with an elementary form of inference or sentience. Specifically, we consider the distinction between dissipative and conservative particles, inert and active particles and, finally, ordinary and strange particles. Strange particles can be described as inferring their own actions, endowing them with apparent autonomy or agency. In short-of the kinds of particles afforded by a particular partition-strange kinds may be apt for describing sentient behaviour.

Neural correlates of intelligence: ERP Components of temporary storage predict fluid intelligence over and above those of executive functions

Previous research has revealed that individuals with different levels of intelligence exhibit distinct patterns of event-related potentials (ERPs) related to executive functions and temporary storage. However, there is a lack of studies investigating the relative contributions of these ERPs in predicting individual differences in fluid intelligence. This study aims to examine the extent to which ERPs associated with executive functions and temporary storage can predict individual differences in fluid intelligence. Special attention is given to determining whether electrophysiological activities of temporary storage can predict fluid intelligence after accounting for executive functions, and vice versa. Both executive attention and temporary storage were measured by two experimental tasks, while electroencephalographic data were collected simultaneously. Fluid intelligence was assessed by two established tests. To address previous inconsistencies due to small sample sizes, a relatively large sample of young adults (N = 136) was recruited. The results revealed that participants with lower fluid intelligence displayed larger P3 amplitudes in the executive functions and temporary storage tasks compared to those with higher fluid intelligence. Additionally, the amplitudes of frontal and parietal P3s elicited by both executive functions and temporary storage significantly predicted fluid intelligence. Interestingly, the frontal and parietal P3s associated with temporary storage predicted fluid intelligence beyond the contributions of executive functions, supporting the storage account of individual differences in fluid intelligence. This study provides an original and fresh understanding of how executive functions and temporary storage contribute to fluid intelligence, offering new insights into the neurocognitive mechanisms underlying intelligence.

Mindfulness meditation and psychedelics: potential synergies and commonalities

There has been increasing scientific and clinical interest in studying psychedelic and meditation-based interventions in recent years, both in the context of improving mental health and as tools for understanding the mind. Several authors suggest neurophysiological and phenomenological parallels and overlaps between psychedelic and meditative states and suggest synergistic effects of both methods. Both psychedelic-assisted therapy and meditation training in the form of mindfulness-based interventions have been experimentally validated with moderate to large effects as alternative treatments for a variety of mental health problems, including depression, addictions, and anxiety disorders. Both demonstrated significant post-acute and long-term decreases in clinical symptoms and enhancements in well-being in healthy participants, in addition. Postulated shared salutogenic mechanisms, include, among others the ability to alter self-consciousness, present-moment awareness and antidepressant action via corresponding neuromodulatory effects. These shared mechanisms between mindfulness training and psychedelic intervention have led to scientists theorizing, and recently demonstrating, positive synergistic effects when both are used in combination. Research findings suggest that these two approaches can complement each other, enhancing the positive effects of both interventions. However, more theoretical accounts and methodologically sound research are needed before they can be extended into clinical practice. The current review aims to discuss the theoretical rationale of combining psychedelics with mindfulness training, including the predictive coding framework as well as research findings regarding synergies and commonalities between mindfulness training and psychedelic intervention. In addition, suggestions how to combine the two modalities are provided.

Modeling and controlling the body in maladaptive ways: an active inference perspective on non-suicidal self-injury behaviors

A significant number of persons engage in paradoxical behaviors, such as extreme food restriction (up to starvation) and non-suicidal self-injuries, especially during periods of rapid changes, such as adolescence. Here, we contextualize these and related paradoxical behavior within an active inference view of brain functions, which assumes that the brain forms predictive models of bodily variables, emotional experiences, and the embodied self and continuously strives to reduce the uncertainty of such models. We propose that not only in conditions of excessive or prolonged uncertainty, such as in clinical conditions, but also during pivotal periods of developmental transition, paradoxical behaviors might emerge as maladaptive strategies to reduce uncertainty-by "acting on the body"- soliciting salient perceptual and interoceptive sensations, such as pain or excessive levels of hunger. Although such strategies are maladaptive and run against our basic homeostatic imperatives, they might be functional not only to provide some short-term reward (e.g. relief from emotional distress)-as previously proposed-but also to reduce uncertainty and possibly to restore a coherent model of one's bodily experience and the self, affording greater confidence in who we are and what course of actions we should pursue.

An Introduction to Predictive Processing Models of Perception and Decision-Making

The predictive processing framework includes a broad set of ideas, which might be articulated and developed in a variety of ways, concerning how the brain may leverage predictive models when implementing perception, cognition, decision-making, and motor control. This article provides an up-to-date introduction to the two most influential theories within this framework: predictive coding and active inference. The first half of the paper (Sections 2-5) reviews the evolution of predictive coding, from early ideas about efficient coding in the visual system to a more general model encompassing perception, cognition, and motor control. The theory is characterized in terms of the claims it makes at Marr's computational, algorithmic, and implementation levels of description, and the conceptual and mathematical connections between predictive coding, Bayesian inference, and variational free energy (a quantity jointly evaluating model accuracy and complexity) are explored. The second half of the paper (Sections 6-8) turns to recent theories of active inference. Like predictive coding, active inference models assume that perceptual and learning processes minimize variational free energy as a means of approximating Bayesian inference in a biologically plausible manner. However, these models focus primarily on planning and decision-making processes that predictive coding models were not developed to address. Under active inference, an agent evaluates potential plans (action sequences) based on their expected free energy (a quantity that combines anticipated reward and information gain). The agent is assumed to represent the world as a partially observable Markov decision process with discrete time and discrete states. Current research applications of active inference models are described, including a range of simulation work, as well as studies fitting models to empirical data. The paper concludes by considering future research directions that will be important for further development of both models.

Myth of objectivity and the origin of symbols

An age-old challenge in epistemology and moral philosophy is whether objectivity exists independent of subjective perspective. Alfred North Whitehead labeled it a "fallacy of misplaced concreteness"; after all, knowledge is represented elusively in symbols. I employ the free energy principle (FEP) to argue that the belief in moral objectivity, although perhaps fallacious, amounts to an ancient and universal human myth that is essential for our symbolic capacity. To perceive any object in a world of non-diminishing (perhaps irreducible) uncertainty, according to the FEP, its constituent parts must display common probabilistic tendencies, known as statistical beliefs, prior to its interpretation, or active inference, as a stable entity. Behavioral bias, subjective emotions, and social norms scale the scope of identity by coalescing agents with otherwise disparate goals and aligning their perspectives into a coherent structure. I argue that by declaring belief in norms as objective, e.g., expressing that a particular theft or infidelity was generally wrong, our ancestors psychologically constructed a type of identity bound only by shared faith in a perspective that technically transcended individual subjectivity. Signaling explicit belief in what were previously non-symbolic norms, as seen in many non-human animals, simulates a top-down point of view of our social interactions and thereby constructs our cultural niche and symbolic capacity. I demonstrate that, largely by contrasting with overly reductive analytical models that assume individual rational pursuit of extrinsic rewards, shared belief in moral conceptions, i.e., what amounts to a religious faith, remains a motivational cornerstone of our language, economic and civic institutions, stories, and psychology. Finally, I hypothesize that our bias for familiar accents (shibboleth), plausibly represents the phylogenetic and ontogenetic contextual origins of our impulse to minimize social surprise by declaring belief in the myth of objectivity.

Insight and the selection of ideas

Perhaps it is no accident that insight moments accompany some of humanity's most important discoveries in science, medicine, and art. Here we propose that feelings of insight play a central role in (heuristically) selecting an idea from the stream of consciousness by capturing attention and eliciting a sense of intuitive confidence permitting fast action under uncertainty. The mechanisms underlying this Eureka heuristic are explained within an active inference framework. First, implicit restructuring via Bayesian reduction leads to a higher-order prediction error (i.e., the content of insight). Second, dopaminergic precision-weighting of the prediction error accounts for the intuitive confidence, pleasure, and attentional capture (i.e., the feeling of insight). This insight as precision account is consistent with the phenomenology, accuracy, and neural unfolding of insight, as well as its effects on belief and decision-making. We conclude by reflecting on dangers of the Eureka Heuristic, including the arising and entrenchment of false beliefs and the vulnerability of insights under psychoactive substances and misinformation.

Hybrid predictive coding: Inferring, fast and slow

Predictive coding is an influential model of cortical neural activity. It proposes that perceptual beliefs are furnished by sequentially minimising "prediction errors"-the differences between predicted and observed data. Implicit in this proposal is the idea that successful perception requires multiple cycles of neural activity. This is at odds with evidence that several aspects of visual perception-including complex forms of object recognition-arise from an initial "feedforward sweep" that occurs on fast timescales which preclude substantial recurrent activity. Here, we propose that the feedforward sweep can be understood as performing amortized inference (applying a learned function that maps directly from data to beliefs) and recurrent processing can be understood as performing iterative inference (sequentially updating neural activity in order to improve the accuracy of beliefs). We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner by describing both in terms of a dual optimization of a single objective function. We show that the resulting scheme can be implemented in a biologically plausible neural architecture that approximates Bayesian inference utilising local Hebbian update rules. We demonstrate that our hybrid predictive coding model combines the benefits of both amortized and iterative inference-obtaining rapid and computationally cheap perceptual inference for familiar data while maintaining the context-sensitivity, precision, and sample efficiency of iterative inference schemes. Moreover, we show how our model is inherently sensitive to its uncertainty and adaptively balances iterative and amortized inference to obtain accurate beliefs using minimum computational expense. Hybrid predictive coding offers a new perspective on the functional relevance of the feedforward and recurrent activity observed during visual perception and offers novel insights into distinct aspects of visual phenomenology.

Models of the Translation Process and the Free Energy Principle

Translation process research (TPR) has generated a large number of models that aim at explaining human translation processes. In this paper, I suggest an extension of the monitor model to incorporate aspects of relevance theory (RT) and to adopt the free energy principle (FEP) as a generative model to elucidate translational behaviour. The FEP-and its corollary, active inference-provide a general, mathematical framework to explain how organisms resist entropic erosion so as to remain within their phenotypic bounds. It posits that organisms reduce the gap between their expectations and observations by minimising a quantity called free energy. I map these concepts on the translation process and exemplify them with behavioural data. The analysis is based on the notion of translation units (TUs) which exhibit observable traces of the translator's epistemic and pragmatic engagement with their translation environment, (i.e., the text) that can be measured in terms of translation effort and effects. Sequences of TUs cluster into translation states (steady state, orientation, and hesitation). Drawing on active inference, sequences of translation states combine into translation policies that reduce expected free energy. I show how the notion of free energy is compatible with the concept of relevance, as developed in RT, and how essential concepts of the monitor model and RT can be formalised as deep temporal generative models that can be interpreted under a representationalist view, but also support a non-representationalist account.

Quantifying aberrant approach-avoidance conflict in psychopathology: A review of computational approaches

Making effective decisions during approach-avoidance conflict is critical in daily life. Aberrant decision-making during approach-avoidance conflict is evident in a range of psychological disorders, including anxiety, depression, trauma-related disorders, substance use disorders, and alcohol use disorders. To help clarify etiological pathways and reveal novel intervention targets, clinical research into decision-making is increasingly adopting a computational psychopathology approach. This approach uses mathematical models that can identify specific decision-making related processes that are altered in mental health disorders. In our review, we highlight foundational approach-avoidance conflict research, followed by more in-depth discussion of computational approaches that have been used to model behavior in these tasks. Specifically, we describe the computational models that have been applied to approach-avoidance conflict (e.g., drift-diffusion, active inference, and reinforcement learning models), and provide resources to guide clinical researchers who may be interested in applying computational modeling. Finally, we identify notable gaps in the current literature and potential future directions for computational approaches aimed at identifying mechanisms of approach-avoidance conflict in psychopathology.

Restructuring insight: An integrative review of insight in problem-solving, meditation, psychotherapy, delusions and psychedelics

Occasionally, a solution or idea arrives as a sudden understanding - an insight. Insight has been considered an "extra" ingredient of creative thinking and problem-solving. Here we propose that insight is central in seemingly distinct areas of research. Drawing on literature from a variety of fields, we show that besides being commonly studied in problem-solving literature, insight is also a core component in psychotherapy and meditation, a key process underlying the emergence of delusions in schizophrenia, and a factor in the therapeutic effects of psychedelics. In each case, we discuss the event of insight and its prerequisites and consequences. We review evidence for the commonalities and differences between the fields and discuss their relevance for capturing the essence of the insight phenomenon. The goal of this integrative review is to bridge the gap between the different views and inspire interdisciplinary research efforts for understanding this central process of human cognition.

To copy or not to copy? That is the question! From chimpanzees to the foundation of human technological culture

A prerequisite for copying innovative behaviour faithfully is the capacity of observers' brains, regarded as 'hierarchically mechanistic minds', to overcome cognitive 'surprisal' (see 2.), by maximising the evidence for their internal models, through active inference. Unlike modern humans, chimpanzees and other great apes show considerable limitations in their ability, or 'Zone of Bounded Surprisal', to overcome cognitive surprisal induced by innovative or unorthodox behaviour that rarely, therefore, is copied precisely or accurately. Most can copy adequately what is within their phenotypically habitual behavioural repertoire, in which technology plays scant part. Widespread intra- and intergenerational social transmission of complex technological innovations is not a hall-mark of great-ape taxa. 3 Ma, precursors of the genus Homo made stone artefacts, and stone-flaking likely was habitual before 2 Ma. After that time, early Homo erectus has left traces of technological innovations, though faithful copying of these and their intra- and intergenerational social transmission were rare before 1 Ma. This likely owed to a cerebral infrastructure of interconnected neuronal systems more limited than ours. Brains were smaller in size than ours, and cerebral neuronal systems ceased to develop when early Homo erectus attained full adult maturity by the mid-teen years, whereas its development continues until our mid-twenties nowadays. Pleistocene Homo underwent remarkable evolutionary adaptation of neurobiological propensities, and cerebral aspects are discussed that, it is proposed here, plausibly, were fundamental for faithful copying, which underpinned social transmission of technologies, cumulative learning, and culture. Here, observers' responses to an innovation are more important for ensuring its transmission than is an innovator's production of it, because, by themselves, the minimal cognitive prerequisites that are needed for encoding and assimilating innovations are insufficient for practical outcomes to accumulate and spread intra- and intergenerationally.

A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension

Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity to contextualize to facilitate this process, how internal knowledge is deployed online remains an open question. Here, we present a model that extracts multiple levels of information from continuous speech online. The model applies linguistic and nonlinguistic knowledge to speech processing, by periodically generating top-down predictions and incorporating bottom-up incoming evidence in a nested temporal hierarchy. We show that a nonlinguistic context level provides semantic predictions informed by sensory inputs, which are crucial for disambiguating among multiple meanings of the same word. The explicit knowledge hierarchy of the model enables a more holistic account of the neurophysiological responses to speech compared to using lexical predictions generated by a neural network language model (GPT-2). We also show that hierarchical predictions reduce peripheral processing via minimizing uncertainty and prediction error. With this proof-of-concept model, we demonstrate that the deployment of hierarchical predictions is a possible strategy for the brain to dynamically utilize structured knowledge and make sense of the speech input.

Differentiating between Bayesian parameter learning and structure learning based on behavioural and pupil measures

Within predictive processing two kinds of learning can be distinguished: parameter learning and structure learning. In Bayesian parameter learning, parameters under a specific generative model are continuously being updated in light of new evidence. However, this learning mechanism cannot explain how new parameters are added to a model. Structure learning, unlike parameter learning, makes structural changes to a generative model by altering its causal connections or adding or removing parameters. Whilst these two types of learning have recently been formally differentiated, they have not been empirically distinguished. The aim of this research was to empirically differentiate between parameter learning and structure learning on the basis of how they affect pupil dilation. Participants took part in a within-subject computer-based learning experiment with two phases. In the first phase, participants had to learn the relationship between cues and target stimuli. In the second phase, they had to learn a conditional change in this relationship. Our results show that the learning dynamics were indeed qualitatively different between the two experimental phases, but in the opposite direction as we originally expected. Participants were learning more gradually in the second phase compared to the first phase. This might imply that participants built multiple models from scratch in the first phase (structure learning) before settling on one of these models. In the second phase, participants possibly just needed to update the probability distribution over the model parameters (parameter learning).

Curiosity as a metacognitive feeling

Curious information-seeking is known to be a key driver for learning, but characterizing this important psychological phenomenon remains a challenge. In this article, we argue that solving this challenge requires qualifying the relationships between metacognition and curiosity. The idea that curiosity is a metacognitive competence has been resisted: researchers have assumed both that young children and non-human animals can be genuinely curious, and that metacognition requires conceptual and culturally situated resources that are unavailable to young children and non-human animals. Here, we argue that this resistance is unwarranted given accumulating evidence that metacognition can be deployed procedurally, and we defend the view that curiosity is a metacognitive feeling. Our metacognitive view singles out two monitoring steps as a triggering condition for curiosity: evaluating one's own informational needs, and predicting the likelihood that explorations of the proximate environment afford significant information gains. We review empirical evidence and computational models of curiosity, and show that they fit well with this metacognitive account, while on the contrary, they remain difficult to explain by a competing account according to which curiosity is a basic attitude of questioning. Finally, we propose a new way to construe the relationships between curiosity and the human-specific communicative practice of questioning, discuss the issue of how children may learn to express their curiosity through interactions with others, and conclude by briefly exploring the implications of our proposal for educational practices.

Small steps for mankind: Modeling the emergence of cumulative culture from joint active inference communication

Although the increase in the use of dynamical modeling in the literature on cultural evolution makes current models more mathematically sophisticated, these models have yet to be tested or validated. This paper provides a testable deep active inference formulation of social behavior and accompanying simulations of cumulative culture in two steps: First, we cast cultural transmission as a bi-directional process of communication that induces a generalized synchrony (operationalized as a particular convergence) between the belief states of interlocutors. Second, we cast social or cultural exchange as a process of active inference by equipping agents with the choice of who to engage in communication with. This induces trade-offs between confirmation of current beliefs and exploration of the social environment. We find that cumulative culture emerges from belief updating (i.e., active inference and learning) in the form of a joint minimization of uncertainty. The emergent cultural equilibria are characterized by a segregation into groups, whose belief systems are actively sustained by selective, uncertainty minimizing, dyadic exchanges. The nature of these equilibria depends sensitively on the precision afforded by various probabilistic mappings in each individual's generative model of their encultured niche.

The Active Inference Model of Coherence Therapy

Coherence Therapy is an empirically derived experiential psychotherapy based on Psychological Constructivism. Symptoms are viewed as necessary output from an implicit model of the world. The therapist curates experiences and directs attention toward discovering the model. Rendered explicit, the model is juxtaposed with contradictory knowledge driving memory re-consolidation with resolution of the symptom. The Bayesian Brain views perception and action as inferential processes. Prior beliefs are combined in a generative model to explain the hidden causes of sensations through a process of Active Inference. Prior beliefs that are poor fits to the real world are suboptimal. Suboptimal priors with optimal inference produce Bayes Optimal Pathology with behavioral symptoms. The Active Inference Model of Coherence Therapy posits that Coherence Therapy is a dyadic act of therapist guided Active Inference that renders the (probable) hidden causes of a client's behavior conscious. The therapist's sustained attention on the goal of inference helps to overcome memory control bias against retrieval of the affectively charged suboptimal prior. Serial experiences cue memory retrieval and re-instantiation of the physiological/affective state that necessitates production of the symptom in a particular context. As this process continues there is a break in modularity with assimilation into broader networks of experience. Typically, the symptom produced by optimal inference with the suboptimal prior is experienced as unnecessary/inappropriate when taken out of the particular context. The implicit construct has been re-represented and rendered consciously accessible, by a more complex but more accurate model in which the symptom is necessary in some contexts but not others. There is an experience of agency and control in symptom creation, accompanied by the spontaneous production of context appropriate behavior. The capacity for inference has been restored. The Active Inference Model of Coherence Therapy provides a framework for Coherence Therapy as a computational process which can serve as the basis for new therapeutic interventions and experimental designs integrating biological, cognitive, behavioral, and environmental factors.

A neural active inference model of perceptual-motor learning

The active inference framework (AIF) is a promising new computational framework grounded in contemporary neuroscience that can produce human-like behavior through reward-based learning. In this study, we test the ability for the AIF to capture the role of anticipation in the visual guidance of action in humans through the systematic investigation of a visual-motor task that has been well-explored-that of intercepting a target moving over a ground plane. Previous research demonstrated that humans performing this task resorted to anticipatory changes in speed intended to compensate for semi-predictable changes in target speed later in the approach. To capture this behavior, our proposed "neural" AIF agent uses artificial neural networks to select actions on the basis of a very short term prediction of the information about the task environment that these actions would reveal along with a long-term estimate of the resulting cumulative expected free energy. Systematic variation revealed that anticipatory behavior emerged only when required by limitations on the agent's movement capabilities, and only when the agent was able to estimate accumulated free energy over sufficiently long durations into the future. In addition, we present a novel formulation of the prior mapping function that maps a multi-dimensional world-state to a uni-dimensional distribution of free-energy/reward. Together, these results demonstrate the use of AIF as a plausible model of anticipatory visually guided behavior in humans.

Resilience and active inference

In this article, we aim to conceptualize and formalize the construct of resilience using the tools of active inference, a new physics-based modeling approach apt for the description and analysis of complex adaptive systems. We intend this as a first step toward a computational model of resilient systems. We begin by offering a conceptual analysis of resilience, to clarify its meaning, as established in the literature. We examine an orthogonal, threefold distinction between meanings of the word "resilience": (i) inertia, or the ability to resist change (ii) elasticity, or the ability to bounce back from a perturbation, and (iii) plasticity, or the ability to flexibly expand the repertoire of adaptive states. We then situate all three senses of resilience within active inference. We map resilience as inertia onto high precision beliefs, resilience as elasticity onto relaxation back to characteristic (i.e., attracting) states, and resilience as plasticity onto functional redundancy and structural degeneracy.

Counterfactual curiosity: motivated thinking about what might have been

Counterfactual information, information about what might have been, forms the content of counterfactual thoughts and emotions like regret and relief. Recent research suggests that human adults and children, as well as rhesus monkeys, demonstrate 'counterfactual curiosity': they are motivated to seek out counterfactual information after making decisions. Based on contemporary theories of curiosity and information seeking and a broad range of empirical literature, we suggest multiple heterogeneous psychological processes that contribute to people's motivation for counterfactual information. This includes processes that are identified in the curiosity literature more generally-the potential use of counterfactual information for adaptive decision making (its long-term instrumental value) and the drive to reduce uncertainty. Additionally, we suggest that counterfactual information may be particularly alluring because of its role in causal reasoning; its relationship with prediction and decision making; and its potential to fulfil emotion regulation and self-serving goals. Some future directions have been suggested, including investigating the role of individual differences in counterfactual curiosity on learning and wellbeing. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.

Structure learning enhances concept formation in synthetic Active Inference agents

Humans display astonishing skill in learning about the environment in which they operate. They assimilate a rich set of affordances and interrelations among different elements in particular contexts, and form flexible abstractions (i.e., concepts) that can be generalised and leveraged with ease. To capture these abilities, we present a deep hierarchical Active Inference model of goal-directed behaviour, and the accompanying belief update schemes implied by maximising model evidence. Using simulations, we elucidate the potential mechanisms that underlie and influence concept learning in a spatial foraging task. We show that the representations formed–as a result of foraging–reflect environmental structure in a way that is enhanced and nuanced by Bayesian model reduction, a special case of structure learning that typifies learning in the absence of new evidence. Synthetic agents learn associations and form concepts about environmental context and configuration as a result of inferential, parametric learning, and structure learning processes–three processes that can produce a diversity of beliefs and belief structures. Furthermore, the ensuing representations reflect symmetries for environments with identical configurations.

Rethinking statistical learning as a continuous dynamic stochastic process, from the motor systems perspective

The brain integrates streams of sensory input and builds accurate predictions, while arriving at stable percepts under disparate time scales. This stochastic process bears different unfolding dynamics for different people, yet statistical learning (SL) currently averages out, as noise, individual fluctuations in data streams registered from the brain as the person learns. We here adopt a new analytical approach that instead of averaging out fluctuations in continuous electroencephalographic (EEG)-based data streams, takes these gross data as the important signals. Our new approach reassesses how individuals dynamically learn predictive information in stable and unstable environments. We find neural correlates for two types of learners in a visuomotor task: narrow-variance learners, who retain explicit knowledge of the regularity embedded in the stimuli. They seem to use an error-correction strategy steadily present in both stable and unstable environments. This strategy can be captured by current optimization-based computational frameworks. In contrast, broad-variance learners emerge only in the unstable environment. Local analyses of the moment-by-moment fluctuations, naïve to the overall outcome, reveal an initial period of memoryless learning, well characterized by a continuous gamma process starting out exponentially distributed whereby all future events are equally probable, with high signal (mean) to noise (variance) ratio. The empirically derived continuous Gamma process smoothly converges to predictive Gaussian signatures comparable to those observed for the error-corrective mode that is captured by current optimization-driven computational models. We coin this initially seemingly purposeless stage exploratory. Globally, we examine a posteriori the fluctuations in distributions' shapes over the empirically estimated stochastic signatures. We then confirm that the exploratory mode of those learners, free of expectation, random and memoryless, but with high signal, precedes the acquisition of the error-correction mode boasting smooth transition from exponential to symmetric distributions' shapes. This early naïve phase of the learning process has been overlooked by current models driven by expected, predictive information and error-based learning. Our work demonstrates that (statistical) learning is a highly dynamic and stochastic process, unfolding at different time scales, and evolving distinct learning strategies on demand.