Active Inference and Learning in the Cerebellum

Understanding music and aging through the lens of Bayesian inference

Bayesian inference has recently gained momentum in explaining music perception and aging. A fundamental mechanism underlying Bayesian inference is the notion of prediction. This framework could explain how predictions pertaining to musical (melodic, rhythmic, harmonic) structures engender action, emotion, and learning, expanding related concepts of music research, such as musical expectancies, groove, pleasure, and tension. Moreover, a Bayesian perspective of music perception may shed new insights on the beneficial effects of music in aging. Aging could be framed as an optimization process of Bayesian inference. As predictive inferences refine over time, the reliance on consolidated priors increases, while the updating of prior models through Bayesian inference attenuates. This may affect the ability of older adults to estimate uncertainties in their environment, limiting their cognitive and behavioral repertoire. With Bayesian inference as an overarching framework, this review synthesizes the literature on predictive inferences in music and aging, and details how music could be a promising tool in preventive and rehabilitative interventions for older adults through the lens of Bayesian inference.

Variability in white matter structure relates to hallucination proneness

Hallucinations are a prominent transdiagnostic psychiatric symptom but are also prevalent in individuals who do not require clinical care. Moreover, persistent psychosis-like experience in otherwise healthy individuals may be related to an increased risk to transition to a psychotic disorder. This suggests a common etiology across clinical and non-clinical individuals along a multidimensional psychosis continuum that may be detectable in structural variations of the brain. The current diffusion tensor imaging study assessed 50 healthy individuals (35 females) to identify possible differences in white matter associated with hallucination proneness (HP). This approach circumvents potential confounds related to medication, hospitalization, and disease progression common in clinical individuals. We determined how HP relates to white matter structure in selected association, commissural, and projection fiber pathways putatively linked to psychosis. Increased HP was associated with enhanced fractional anisotropy (FA) in the right uncinate fasciculus, the right anterior and posterior arcuate fasciculus, and the corpus callosum. These findings support the notion of a psychosis continuum, providing first evidence of structural white matter variability associated with HP in healthy individuals. Furthermore, alterations in the targeted pathways likely indicate an association between HP-related structural variations and the putative salience and attention mechanisms that these pathways subserve.

Cerebellar state estimation enables resilient coupling across behavioural domains

Cerebellar computations are necessary for fine behavioural control and may rely on internal models for estimation of behaviourally relevant states. Here, we propose that the central cerebellar function is to estimate how states interact with each other, and to use these estimates to coordinates extra-cerebellar neuronal dynamics underpinning a range of interconnected behaviours. To support this claim, we describe a cerebellar model for state estimation that includes state interactions, and link this model with the neuronal architecture and dynamics observed empirically. This is formalised using the free energy principle, which provides a dual perspective on a system in terms of both the dynamics of its physical-in this case neuronal-states, and the inferential process they entail. As a demonstration of this proposal, we simulate cerebellar-dependent synchronisation of whisking and respiration, which are known to be tightly coupled in rodents, as well as limb and tail coordination during locomotion. In summary, we propose that the ubiquitous involvement of the cerebellum in behaviour arises from its central role in precisely coupling behavioural domains.

The empirical status of predictive coding and active inference

Research on predictive processing models has focused largely on two specific algorithmic theories: Predictive Coding for perception and Active Inference for decision-making. While these interconnected theories possess broad explanatory potential, they have only recently begun to receive direct empirical evaluation. Here, we review recent studies of Predictive Coding and Active Inference with a focus on evaluating the degree to which they are empirically supported. For Predictive Coding, we find that existing empirical evidence offers modest support. However, some positive results can also be explained by alternative feedforward (e.g., feature detection-based) models. For Active Inference, most empirical studies have focused on fitting these models to behavior as a means of identifying and explaining individual or group differences. While Active Inference models tend to explain behavioral data reasonably well, there has not been a focus on testing empirical validity of active inference theory per se, which would require formal comparison to other models (e.g., non-Bayesian or model-free reinforcement learning models). This review suggests that, while promising, a number of specific research directions are still necessary to evaluate the empirical adequacy and explanatory power of these algorithms.

The cerebellum plays more than one role in the dysregulation of appetite: Review of structural evidence from typical and eating disorder populations

OBJECTIVE

Dysregulated appetite control is characteristic of anorexia nervosa (AN), bulimia nervosa (BN), and obesity (OB). Studies using a broad range of methods suggest the cerebellum plays an important role in aspects of weight and appetite control, and is implicated in both AN and OB by reports of aberrant gray matter volume (GMV) compared to nonclinical populations. As functions of the cerebellum are anatomically segregated, specific localization of aberrant anatomy may indicate the mechanisms of its relationship with weight and appetite in different states. We sought to determine if there were consistencies in regions of cerebellar GMV changes in AN/BN and OB, as well as across normative (NOR) variation.

METHOD

Systematic review and meta-analysis using GingerALE.

RESULTS

Twenty-six publications were identified as either case-control studies (nOB  = 277; nAN/BN  = 510) or regressed weight from NOR data against brain volume (total n = 3830). AN/BN and OB analyses both showed consistently decreased GMV within Crus I and Lobule VI, but volume reduction was bilateral for AN/BN and unilateral for OB. Analysis of the NOR data set identified a cluster in right posterior lobe that overlapped with AN/BN cerebellar reduction. Sensitivity analyses indicated robust repeatability for NOR and AN/BN cohorts, but found OB-specific heterogeneity.

DISCUSSION

Findings suggest that more than one area of the cerebellum is involved in control of eating behavior and may be differentially affected in normal variation and pathological conditions. Specifically, we hypothesize an association with sensorimotor and emotional learning via Lobule VI in AN/BN, and executive function via Crus I in OB.

From the origins to the stream of consciousness and its neural correlates

There are now dozens of very different theories of consciousness, each somehow contributing to our understanding of its nature. The science of consciousness needs therefore not new theories but a general framework integrating insights from those, yet not making it a still-born "Frankenstein" theory. First, the framework must operate explicitly on the stream of consciousness, not on its static description. Second, this dynamical account must also be put on the evolutionary timeline to explain the origins of consciousness. The Cognitive Evolution Theory (CET), outlined here, proposes such a framework. This starts with the assumption that brains have primarily evolved as volitional subsystems of organisms, inherited from primitive (fast and random) reflexes of simplest neural networks, only then resembling error-minimizing prediction machines. CET adopts the tools of critical dynamics to account for metastability, scale-free avalanches, and self-organization which are all intrinsic to brain dynamics. This formalizes the stream of consciousness as a discrete (transitive, irreflexive) chain of momentary states derived from critical brain dynamics at points of phase transitions and mapped then onto a state space as neural correlates of a particular conscious state. The continuous/discrete dichotomy appears naturally between the brain dynamics at the causal level and conscious states at the phenomenal level, each volitionally triggered from arousal centers of the brainstem and cognitively modulated by thalamocortical systems. Their objective observables can be entropy-based complexity measures, reflecting the transient level or quantity of consciousness at that moment.

Emergence of associative learning in a neuromorphic inference network

OBJECTIVE

In the theoretical framework of predictive coding and active inference, the brain can be viewed as instantiating a rich generative model of the world that predicts incoming sensory data while continuously updating its parameters via minimization of prediction errors. While this theory has been successfully applied to cognitive processes - by modelling the activity of functional neural networks at a mesoscopic scale - the validity of the approach when modelling neurons as an ensemble of inferring agents, in a biologically plausible architecture, remained to be explored.

APPROACH

We modelled a simplified cerebellar circuit with individual neurons acting as Bayesian agents to simulate the classical delayed eyeblink conditioning protocol. Neurons and synapses adjusted their activity to minimize their prediction error, which was used as the network cost function. This cerebellar network was then implemented in hardware by replicating digital neuronal elements via a low-power microcontroller.

MAIN RESULTS

Persistent changes of synaptic strength - that mirrored neurophysiological observations - emerged via local (neurocentric) prediction error minimization, leading to the expression of associative learning. The same paradigm was effectively emulated in low-power hardware showing remarkably efficient performance compared to conventional neuromorphic architectures.

SIGNIFICANCE

These findings show that: i) an ensemble of free energy minimizing neurons - organized in a biological plausible architecture - can recapitulate functional self-organization observed in nature, such as associative plasticity, and ii) a neuromorphic network of inference units can learn unsupervised tasks without embedding predefined learning rules in the circuit, thus providing a potential avenue to a novel form of brain-inspired artificial intelligence.

The Involvement of the Posterior Cerebellum in Reconstructing and Predicting Social Action Sequences

Recent advances in social neuroscience have highlighted the critical role of the cerebellum and especially the posterior cerebellar Crus in social mentalizing (i.e., theory of mind). Research in the past 5 years has provided growing evidence supporting the view that the posterior cerebellum builds internal action models of our social interactions to predict how other people's actions will be executed, and what our most likely responses to these actions will be. This paper presents an overview of a series of fMRI experiments on novel tasks involving a combination of (a) the learning or generation of chronological sequences of social actions either in an explicit or implicit manner, which (b) require social mentalizing on another person's mental state such as goals, beliefs, and implied traits. Together, the results strongly confirm the central role of the posterior cerebellar Crus in identifying and automatizing action sequencing during social mentalizing, and in predicting future action sequences based on social mentalizing inferences about others. This research program has important implications: It provides for the first time (a) fruitful starting points for diagnosing and investigating social sequencing dysfunctions in a variety of mental disorders which have also been related to cerebellar dysfunctions, (b) provides the necessary tools for testing whether non-invasive neurostimulation targeting the posterior cerebellum has a causal effect on social functioning, and (c) whether these stimulation techniques and training programs guided by novel cerebellar social sequencing insights, can be exploited to increase posterior cerebellar plasticity in order to alleviate social impairments in mental disorders.

Consciousness and complexity: a consilience of evidence

Over the last years, a surge of empirical studies converged on complexity-related measures as reliable markers of consciousness across many different conditions, such as sleep, anesthesia, hallucinatory states, coma, and related disorders. Most of these measures were independently proposed by researchers endorsing disparate frameworks and employing different methods and techniques. Since this body of evidence has not been systematically reviewed and coherently organized so far, this positive trend has remained somewhat below the radar. The aim of this paper is to make this consilience of evidence in the science of consciousness explicit. We start with a systematic assessment of the growing literature on complexity-related measures and identify their common denominator, tracing it back to core theoretical principles and predictions put forward more than 20 years ago. In doing this, we highlight a consistent trajectory spanning two decades of consciousness research and provide a provisional taxonomy of the present literature. Finally, we consider all of the above as a positive ground to approach new questions and devise future experiments that may help consolidate and further develop a promising field where empirical research on consciousness appears to have, so far, naturally converged.

Memory and Markov Blankets

In theoretical biology, we are often interested in random dynamical systems-like the brain-that appear to model their environments. This can be formalized by appealing to the existence of a (possibly non-equilibrium) steady state, whose density preserves a conditional independence between a biological entity and its surroundings. From this perspective, the conditioning set, or Markov blanket, induces a form of vicarious synchrony between creature and world-as if one were modelling the other. However, this results in an apparent paradox. If all conditional dependencies between a system and its surroundings depend upon the blanket, how do we account for the mnemonic capacity of living systems? It might appear that any shared dependence upon past blanket states violates the independence condition, as the variables on either side of the blanket now share information not available from the current blanket state. This paper aims to resolve this paradox, and to demonstrate that conditional independence does not preclude memory. Our argument rests upon drawing a distinction between the dependencies implied by a steady state density, and the density dynamics of the system conditioned upon its configuration at a previous time. The interesting question then becomes: What determines the length of time required for a stochastic system to 'forget' its initial conditions? We explore this question for an example system, whose steady state density possesses a Markov blanket, through simple numerical analyses. We conclude with a discussion of the relevance for memory in cognitive systems like us.

Cerebellar Activation Deficits in Schizophrenia During an Eyeblink Conditioning Task

The cognitive dysmetria theory of psychotic disorders posits that cerebellar circuit abnormalities give rise to difficulties coordinating motor and cognitive functions. However, brain activation during cerebellar-mediated tasks is understudied in schizophrenia. Accordingly, this study examined whether individuals with schizophrenia have diminished neural activation compared to controls in key regions of the delay eyeblink conditioning (dEBC) cerebellar circuit (eg, lobule VI) and cerebellar regions associated with cognition (eg, Crus I). Participants with schizophrenia-spectrum disorders (n = 31) and healthy controls (n = 43) underwent dEBC during functional magnetic resonance imaging (fMRI). Images were normalized using the Spatially Unbiased Infratentorial Template (SUIT) of the cerebellum and brainstem. Activation contrasts of interest were "early" and "late" stages of paired tone and air puff trials minus unpaired trials. Preliminary whole brain analyses were conducted, followed by cerebellar-specific SUIT and region of interest (ROI) analyses of lobule VI and Crus I. Correlation analyses were conducted between cerebellar activation, neuropsychological test scores, and psychotic symptom scores. In controls, the largest clusters of cerebellar activation peaked in lobule VI during early dEBC and Crus I during late dEBC. The schizophrenia group showed robust cortical activation to unpaired trials but no significant conditioning-related cerebellar activation. Crus I ROI activation during late dEBC was greater in the control than schizophrenia group. Greater Crus I activation correlated with higher working memory scores in the full sample and lower positive psychotic symptom severity in schizophrenia. Findings indicate functional cerebellar abnormalities in schizophrenia which relate to psychotic symptoms, lending direct support to the cognitive dysmetria framework.

Consensus Paper: Cerebellum and Social Cognition

The traditional view on the cerebellum is that it controls motor behavior. Although recent work has revealed that the cerebellum supports also nonmotor functions such as cognition and affect, only during the last 5 years it has become evident that the cerebellum also plays an important social role. This role is evident in social cognition based on interpreting goal-directed actions through the movements of individuals (social "mirroring") which is very close to its original role in motor learning, as well as in social understanding of other individuals' mental state, such as their intentions, beliefs, past behaviors, future aspirations, and personality traits (social "mentalizing"). Most of this mentalizing role is supported by the posterior cerebellum (e.g., Crus I and II). The most dominant hypothesis is that the cerebellum assists in learning and understanding social action sequences, and so facilitates social cognition by supporting optimal predictions about imminent or future social interaction and cooperation. This consensus paper brings together experts from different fields to discuss recent efforts in understanding the role of the cerebellum in social cognition, and the understanding of social behaviors and mental states by others, its effect on clinical impairments such as cerebellar ataxia and autism spectrum disorder, and how the cerebellum can become a potential target for noninvasive brain stimulation as a therapeutic intervention. We report on the most recent empirical findings and techniques for understanding and manipulating cerebellar circuits in humans. Cerebellar circuitry appears now as a key structure to elucidate social interactions.

Bayesian Filtering with Multiple Internal Models: Toward a Theory of Social Intelligence

To exhibit social intelligence, animals have to recognize whom they are communicating with. One way to make this inference is to select among internal generative models of each conspecific who may be encountered. However, these models also have to be learned via some form of Bayesian belief updating. This induces an interesting problem: When receiving sensory input generated by a particular conspecific, how does an animal know which internal model to update? We consider a theoretical and neurobiologically plausible solution that enables inference and learning of the processes that generate sensory inputs (e.g., listening and understanding) and reproduction of those inputs (e.g., talking or singing), under multiple generative models. This is based on recent advances in theoretical neurobiology-namely, active inference and post hoc (online) Bayesian model selection. In brief, this scheme fits sensory inputs under each generative model. Model parameters are then updated in proportion to the probability that each model could have generated the input (i.e., model evidence). The proposed scheme is demonstrated using a series of (real zebra finch) birdsongs, where each song is generated by several different birds. The scheme is implemented using physiologically plausible models of birdsong production. We show that generalized Bayesian filtering, combined with model selection, leads to successful learning across generative models, each possessing different parameters. These results highlight the utility of having multiple internal models when making inferences in social environments with multiple sources of sensory information.

Theories of Error Back-Propagation in the Brain

This review article summarises recently proposed theories on how neural circuits in the brain could approximate the error back-propagation algorithm used by artificial neural networks. Computational models implementing these theories achieve learning as efficient as artificial neural networks, but they use simple synaptic plasticity rules based on activity of presynaptic and postsynaptic neurons. The models have similarities, such as including both feedforward and feedback connections, allowing information about error to propagate throughout the network. Furthermore, they incorporate experimental evidence on neural connectivity, responses, and plasticity. These models provide insights on how brain networks might be organised such that modification of synaptic weights on multiple levels of cortical hierarchy leads to improved performance on tasks.

Predicting change: Approximate inference under explicit representation of temporal structure in changing environments

In our daily lives timing of our actions plays an essential role when we navigate the complex everyday environment. It is an open question though how the representations of the temporal structure of the world influence our behavior. Here we propose a probabilistic model with an explicit representation of state durations which may provide novel insights in how the brain predicts upcoming changes. We illustrate several properties of the behavioral model using a standard reversal learning design and compare its task performance to standard reinforcement learning models. Furthermore, using experimental data, we demonstrate how the model can be applied to identify participants' beliefs about the latent temporal task structure. We found that roughly one quarter of participants seem to have learned the latent temporal structure and used it to anticipate changes, whereas the remaining participants' behavior did not show signs of anticipatory responses, suggesting a lack of precise temporal expectations. We expect that the introduced behavioral model will allow, in future studies, for a systematic investigation of how participants learn the underlying temporal structure of task environments and how these representations shape behavior.

The Anatomy of Inference: Generative Models and Brain Structure

To infer the causes of its sensations, the brain must call on a generative (predictive) model. This necessitates passing local messages between populations of neurons to update beliefs about hidden variables in the world beyond its sensory samples. It also entails inferences about how we will act. Active inference is a principled framework that frames perception and action as approximate Bayesian inference. This has been successful in accounting for a wide range of physiological and behavioral phenomena. Recently, a process theory has emerged that attempts to relate inferences to their neurobiological substrates. In this paper, we review and develop the anatomical aspects of this process theory. We argue that the form of the generative models required for inference constrains the way in which brain regions connect to one another. Specifically, neuronal populations representing beliefs about a variable must receive input from populations representing the Markov blanket of that variable. We illustrate this idea in four different domains: perception, planning, attention, and movement. In doing so, we attempt to show how appealing to generative models enables us to account for anatomical brain architectures. Ultimately, committing to an anatomical theory of inference ensures we can form empirical hypotheses that can be tested using neuroimaging, neuropsychological, and electrophysiological experiments.

The Discrete and Continuous Brain: From Decisions to Movement-And Back Again

To act upon the world, creatures must change continuous variables such as muscle length or chemical concentration. In contrast, decision making is an inherently discrete process, involving the selection among alternative courses of action. In this article, we consider the interface between the discrete and continuous processes that translate our decisions into movement in a Newtonian world-and how movement informs our decisions. We do so by appealing to active inference, with a special focus on the oculomotor system. Within this exemplar system, we argue that the superior colliculus is well placed to act as a discrete-continuous interface. Interestingly, when the neuronal computations within the superior colliculus are formulated in terms of active inference, we find that many aspects of its neuroanatomy emerge from the computations it must perform in this role.

Trace eyeblink conditioning is associated with changes in synaptophysin immunoreactivity in the cerebellar interpositus nucleus in guinea pigs

Synaptic plasticity plays a role during trace eyeblink conditioning (TEBC). Synaptophysin (Syn) is a major integral transmembrane protein, located particularly in the synaptic vesicles, and is considered a molecular marker of synapses. In addition, Syn immunoreactivity is an important indicator of synaptic plasticity. In the present study, we used immunohistochemical techniques to assess changes in Syn expression in the cerebellar interpositus nucleus (IN) of guinea pigs exposed to TEBC and pseudoconditioning. Additionally, we analyzed the relationship between Syn immunoreactivity and the percentage of trace-conditioned responses. Guinea pigs underwent trace conditioning or pseudoconditioning. Following two, six, or ten sessions, they were perfused and the cerebellum was removed for Syn immunohistochemical evaluation. After sessions 6 and 10, a significant increase in conditioned response (CR) percentage was observed in the trace-conditioned group, with the CR percentage reaching the learning criteria following session 10. Besides, for trace-conditioned animals, the Syn expression in IN was found significantly up-regulated after session 10 compared with pseudoconditioned ones. Our data suggest that the increase in Syn expression links to synaptic plasticity changes in the cerebellar IN and provides a histological substrate in the IN relating to TEBC training. The changing trend of Syn immunoreactivity in the IN is associated with CR percentage.

The Discrete and Continuous Brain: From Decisions to Movement-And Back Again

To act upon the world, creatures must change continuous variables such as muscle length or chemical concentration. In contrast, decision making is an inherently discrete process, involving the selection among alternative courses of action. In this article, we consider the interface between the discrete and continuous processes that translate our decisions into movement in a Newtonian world—and how movement informs our decisions. We do so by appealing to active inference, with a special focus on the oculomotor system. Within this exemplar system, we argue that the superior colliculus is well placed to act as a discrete-continuous interface. Interestingly, when the neuronal computations within the superior colliculus are formulated in terms of active inference, we find that many aspects of its neuroanatomy emerge from the computations it must perform in this role.