Impulsivity and Active Inference

Computational models of behavioral addictions: State of the art and future directions

Non-pharmacological behavioral addictions, such as pathological gambling, videogaming, social networking, or internet use, are becoming major public health concerns. It is not yet clear how behavioral addictions could share many major neurobiological and behavioral characteristics with substance use disorders, despite the absence of direct pharmacological influences. A deeper understanding of the neurocognitive mechanisms of addictive behavior is needed, and computational modeling could be one promising approach to explain intricately entwined cognitive and neural dynamics. This review describes computational models of addiction based on reinforcement learning algorithms, Bayesian inference, and biophysical neural simulations. We discuss whether computational frameworks originally conceived to explain maladaptive behavior in substance use disorders can be effectively extended to non-substance-related behavioral addictions. Moreover, we introduce recent studies on behavioral addictions that exemplify the possibility of such extension and propose future directions.

Contextual perception under active inference

Human social interactions depend on the ability to resolve uncertainty about the mental states of others. The context in which social interactions take place is crucial for mental state attribution as sensory inputs may be perceived differently depending on the context. In this paper, we introduce a mental state attribution task where a target-face with either an ambiguous or an unambiguous emotion is embedded in different social contexts. The social context is determined by the emotions conveyed by other faces in the scene. This task involves mental state attribution to a target-face (either happy or sad) depending on the social context. Using active inference models, we provide a proof of concept that an agent's perception of sensory stimuli may be altered by social context. We show with simulations that context congruency and facial expression coherency improve behavioural performance in terms of decision times. Furthermore, we show through simulations that the abnormal viewing strategies employed by patients with schizophrenia may be due to (i) an imbalance between the precisions of local and global features in the scene and (ii) a failure to modulate the sensory precision to contextualise emotions.

Whence the Expected Free Energy?

The expected free energy (EFE) is a central quantity in the theory of active inference. It is the quantity that all active inference agents are mandated to minimize through action, and its decomposition into extrinsic and intrinsic value terms is key to the balance of exploration and exploitation that active inference agents evince. Despite its importance, the mathematical origins of this quantity and its relation to the variational free energy (VFE) remain unclear. In this letter, we investigate the origins of the EFE in detail and show that it is not simply "the free energy in the future." We present a functional that we argue is the natural extension of the VFE but actively discourages exploratory behavior, thus demonstrating that exploration does not directly follow from free energy minimization into the future. We then develop a novel objective, the free energy of the expected future (FEEF), which possesses both the epistemic component of the EFE and an intuitive mathematical grounding as the divergence between predicted and desired futures.

Active inference on discrete state-spaces: A synthesis

Active inference is a normative principle underwriting perception, action, planning, decision-making and learning in biological or artificial agents. From its inception, its associated process theory has grown to incorporate complex generative models, enabling simulation of a wide range of complex behaviours. Due to successive developments in active inference, it is often difficult to see how its underlying principle relates to process theories and practical implementation. In this paper, we try to bridge this gap by providing a complete mathematical synthesis of active inference on discrete state-space models. This technical summary provides an overview of the theory, derives neuronal dynamics from first principles and relates this dynamics to biological processes. Furthermore, this paper provides a fundamental building block needed to understand active inference for mixed generative models; allowing continuous sensations to inform discrete representations. This paper may be used as follows: to guide research towards outstanding challenges, a practical guide on how to implement active inference to simulate experimental behaviour, or a pointer towards various in-silico neurophysiological responses that may be used to make empirical predictions.

Deep Active Inference and Scene Construction

Adaptive agents must act in intrinsically uncertain environments with complex latent structure. Here, we elaborate a model of visual foraging-in a hierarchical context-wherein agents infer a higher-order visual pattern (a "scene") by sequentially sampling ambiguous cues. Inspired by previous models of scene construction-that cast perception and action as consequences of approximate Bayesian inference-we use active inference to simulate decisions of agents categorizing a scene in a hierarchically-structured setting. Under active inference, agents develop probabilistic beliefs about their environment, while actively sampling it to maximize the evidence for their internal generative model. This approximate evidence maximization (i.e., self-evidencing) comprises drives to both maximize rewards and resolve uncertainty about hidden states. This is realized via minimization of a free energy functional of posterior beliefs about both the world as well as the actions used to sample or perturb it, corresponding to perception and action, respectively. We show that active inference, in the context of hierarchical scene construction, gives rise to many empirical evidence accumulation phenomena, such as noise-sensitive reaction times and epistemic saccades. We explain these behaviors in terms of the principled drives that constitute the expected free energy, the key quantity for evaluating policies under active inference. In addition, we report novel behaviors exhibited by these active inference agents that furnish new predictions for research on evidence accumulation and perceptual decision-making. We discuss the implications of this hierarchical active inference scheme for tasks that require planned sequences of information-gathering actions to infer compositional latent structure (such as visual scene construction and sentence comprehension). This work sets the stage for future experiments to investigate active inference in relation to other formulations of evidence accumulation (e.g., drift-diffusion models) in tasks that require planning in uncertain environments with higher-order structure.

Introducing a Bayesian model of selective attention based on active inference

Information gathering comprises actions whose (sensory) consequences resolve uncertainty (i.e., are salient). In other words, actions that solicit salient information cause the greatest shift in beliefs (i.e., information gain) about the causes of our sensations. However, not all information is relevant to the task at hand: this is especially the case in complex, naturalistic scenes. This paper introduces a formal model of selective attention based on active inference and contextual epistemic foraging. We consider a visual search task with a special emphasis on goal-directed and task-relevant exploration. In this scheme, attention modulates the expected fidelity (precision) of the mapping between observations and hidden states in a state-dependent or context-sensitive manner. This ensures task-irrelevant observations have little expected information gain, and so the agent - driven to reduce expected surprise (i.e., uncertainty) - does not actively seek them out. Instead, it selectively samples task-relevant observations, which inform (task-relevant) hidden states. We further show, through simulations, that the atypical exploratory behaviours in conditions such as autism and anxiety may be due to a failure to appropriately modulate sensory precision in a context-specific way.

Parameters as Trait Indicators: Exploring a Complementary Neurocomputational Approach to Conceptualizing and Measuring Trait Differences in Emotional Intelligence

Current assessments of trait emotional intelligence (EI) rely on self-report inventories. While this approach has seen considerable success, a complementary approach allowing objective assessment of EI-relevant traits would provide some potential advantages. Among others, one potential advantage is that it would aid in emerging efforts to assess the brain basis of trait EI, where self-reported competency levels do not always match real-world behavior. In this paper, we review recent experimental paradigms in computational cognitive neuroscience (CCN), which allow behavioral estimates of individual differences in range of parameter values within computational models of neurocognitive processes. Based on this review, we illustrate how several of these parameters appear to correspond well to EI-relevant traits (i.e., differences in mood stability, stress vulnerability, self-control, and flexibility, among others). In contrast, although estimated objectively, these parameters do not correspond well to the optimal performance abilities assessed within competing “ability models” of EI. We suggest that adapting this approach from CCN—by treating parameter value estimates as objective trait EI measures—could (1) provide novel research directions, (2) aid in characterizing the neural basis of trait EI, and (3) offer a promising complementary assessment method.