Perceptions as hypotheses

Human regulatory systems in the age of abundance: A predictive processing perspective

Human regulatory systems largely evolved under conditions of food and information scarcity but are now being forced to deal with abundance. The impact of abundance and the inability of human regulatory systems to adapt to it have fed a surge in dual health challenges: (1) a rise in obesity related to food abundance and (2) a rise in stress and anxiety related to information abundance. No single framework has been developed to describe why and how the transition from scarcity to abundance has been so challenging. Here, we provide a speculative model based on predictive processing. We suggest that whereas scarcity (above destructive lower bounds like famine or information voids) preserves the fidelity of the relationship between prediction errors and predictions, abundance distorts this relationship. Furthermore, prediction error minimization is enhanced under scarcity (as the number of competing states in the niche is restricted), whereas the opposite is true under abundance. We also discuss how abundance warps the fundamental drive for seeking novelty by fueling the brain's exploration (as opposed to exploitation) mode. Ameliorative strategies for regulating food and information abundance may largely depend on simulating scarcity, that environmental condition to which human regulatory systems have adapted over millennia.

Motor and vestibular signals in the visual cortex permit the separation of self versus externally generated visual motion

Knowing whether we are moving or something in the world is moving around us is possibly the most critical sensory discrimination we need to perform. How the brain and, in particular, the visual system solves this motion-source separation problem is not known. Here, we find that motor, vestibular, and visual motion signals are used by the mouse primary visual cortex (VISp) to differentially represent the same visual flow information according to whether the head is stationary or experiencing passive versus active translation. During locomotion, we find that running suppresses running-congruent translation input and that translation signals dominate VISp activity when running and translation speed become incongruent. This cross-modal interaction between the motor and vestibular systems was found throughout the cortex, indicating that running and translation signals provide a brain-wide egocentric reference frame for computing the internally generated and actual speed of self when moving through and sensing the external world.

Learning to segment self-generated from externally caused optic flow through sensorimotor mismatch circuits

Efficient sensory detection requires the capacity to ignore task-irrelevant information, for example when optic flow patterns created by egomotion need to be disentangled from object perception. To investigate how this is achieved in the visual system, predictive coding with sensorimotor mismatch detection is an attractive starting point. Indeed, experimental evidence for sensorimotor mismatch signals in early visual areas exists, but it is not understood how they are integrated into cortical networks that perform input segmentation and categorization. Our model advances a biologically plausible solution by extending predictive coding models with the ability to distinguish self-generated from externally caused optic flow. We first show that a simple three neuron circuit produces experience-dependent sensorimotor mismatch responses, in agreement with calcium imaging data from mice. This microcircuit is then integrated into a neural network with two generative streams. The motor-to-visual stream consists of parallel microcircuits between motor and visual areas and learns to spatially predict optic flow resulting from self-motion. The second stream bidirectionally connects a motion-selective higher visual area (mHVA) to V1, assigning a crucial role to the abundant feedback connections to V1: the maintenance of a generative model of externally caused optic flow. In the model, area mHVA learns to segment moving objects from the background, and facilitates object categorization. Based on shared neurocomputational principles across species, the model also maps onto primate visual cortex. Our work extends Hebbian predictive coding to sensorimotor settings, in which the agent actively moves - and learns to predict the consequences of its own movements.

Generative models for sequential dynamics in active inference

A central theme of theoretical neurobiology is that most of our cognitive operations require processing of discrete sequences of items. This processing in turn emerges from continuous neuronal dynamics. Notable examples are sequences of words during linguistic communication or sequences of locations during navigation. In this perspective, we address the problem of sequential brain processing from the perspective of active inference, which inherits from a Helmholtzian view of the predictive (Bayesian) brain. Underneath the active inference lies a generative model; namely, a probabilistic description of how (observable) consequences are generated by (unobservable) causes. We show that one can account for many aspects of sequential brain processing by assuming the brain entails a generative model of the sensed world that comprises central pattern generators, narratives, or well-defined sequences. We provide examples in the domains of motor control (e.g., handwriting), perception (e.g., birdsong recognition) through to planning and understanding (e.g., language). The solutions to these problems include the use of sequences of attracting points to direct complex movements-and the move from continuous representations of auditory speech signals to the discrete words that generate those signals.

Perceptual information processing in table tennis players: based on top-down hierarchical predictive coding

Long-term training induces neural plasticity in the visual cognitive processing cortex of table tennis athletes, who perform cognitive processing in a resource-conserving manner. However, further discussion is needed to determine whether the spatial processing advantage of table tennis players manifests in the early stage of sensory input or the late stage of processing. This study aims to explore the processing styles and neural activity characteristics of table tennis players during spatial cognitive processing. Spatial cognitive tasks were completed by 28 college students and 20 s-level table tennis players, and event-related potentials (ERP) data were recorded during the task. The behavioral results showed that the table tennis group performed better on the task than the college students group (control). The ERP results showed that the amplitude of the N1 component of the table tennis group was significantly lower than that of the control group. The amplitude of the P2 and P3 components of the table tennis group was higher than that of the control group. Table tennis players showed significant synergistic activity between electrodes in the β-band. The results of this study suggest that table tennis players significantly deploy attentional resources and cognitive control. Further, they employ stored motor experience to process spatial information in a hierarchical predictive coding manner.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-024-10171-4.

Associative Learning and Active Inference

Associative learning is a behavioral phenomenon in which individuals develop connections between stimuli or events based on their co-occurrence. Initially studied by Pavlov in his conditioning experiments, the fundamental principles of learning have been expanded on through the discovery of a wide range of learning phenomena. Computational models have been developed based on the concept of minimizing reward prediction errors. The Rescorla-Wagner model, in particular, is a well-known model that has greatly influenced the field of reinforcement learning. However, the simplicity of these models restricts their ability to fully explain the diverse range of behavioral phenomena associated with learning. In this study, we adopt the free energy principle, which suggests that living systems strive to minimize surprise or uncertainty under their internal models of the world. We consider the learning process as the minimization of free energy and investigate its relationship with the Rescorla-Wagner model, focusing on the informational aspects of learning, different types of surprise, and prediction errors based on beliefs and values. Furthermore, we explore how well-known behavioral phenomena such as blocking, overshadowing, and latent inhibition can be modeled within the active inference framework. We accomplish this by using the informational and novelty aspects of attention, which share similar ideas proposed by seemingly contradictory models such as Mackintosh and Pearce-Hall models. Thus, we demonstrate that the free energy principle, as a theoretical framework derived from first principles, can integrate the ideas and models of associative learning proposed based on empirical experiments and serve as a framework for a better understanding of the computational processes behind associative learning in the brain.

Imagination vs. routines: festive time, weekly time, and the predictive brain

This paper examines the relationship between societal structures shaped by traditions, norms, laws, and customs, and creative expressions in arts and media through the lens of the predictive coding framework in cognitive science. The article proposes that both dimensions of culture can be viewed as adaptations designed to enhance and train the brain's predictive abilities in the social domain. Traditions, norms, laws, and customs foster shared predictions and expectations among individuals, thereby reducing uncertainty in social environments. On the other hand, arts and media expose us to simulated experiences that explore alternative social realities, allowing the predictive machinery of the brain to hone its skills through exposure to a wider array of potentially relevant social circumstances and scenarios. We first review key principles of predictive coding and active inference, and then explore the rationale of cultural traditions and artistic culture in this perspective. Finally, we draw parallels between institutionalized normative habits that stabilize social worlds and creative and imaginative acts that temporarily subvert established conventions to inject variability.

Spontaneously emerging internal models of visual sequences combine abstract and event-specific information in the prefrontal cortex

When exposed to sensory sequences, do macaque monkeys spontaneously form abstract internal models that generalize to novel experiences? Here, we show that neuronal populations in macaque ventrolateral prefrontal cortex jointly encode visual sequences by separate codes for the specific pictures presented and for their abstract sequential structure. We recorded prefrontal neurons while macaque monkeys passively viewed visual sequences and sequence mismatches in the local-global paradigm. Even without any overt task or response requirements, prefrontal populations spontaneously form representations of sequence structure, serial order, and image identity within distinct but superimposed neuronal subspaces. Representations of sequence structure rapidly update following single exposure to a mismatch sequence, while distinct populations represent mismatches for sequences of different complexity. Finally, those representations generalize across sequences following the same repetition structure but comprising different images. These results suggest that prefrontal populations spontaneously encode rich internal models of visual sequences reflecting both content-specific and abstract information.

Amyloid induced hyperexcitability in default mode network drives medial temporal hyperactivity and early tau accumulation

In early Alzheimer's disease (AD) β-amyloid (Aβ) deposits throughout association cortex and tau appears in the entorhinal cortex (EC). Why these initially appear in disparate locations is not understood. Using task-based fMRI and multimodal PET imaging, we assess the impact of local AD pathology on network-to-network interactions. We show that AD pathologies flip interactions between the default mode network (DMN) and the medial temporal lobe (MTL) from inhibitory to excitatory. The DMN is hyperexcited with increasing levels of Aβ, which drives hyperexcitability within the MTL and this directed hyperexcitation of the MTL by the DMN predicts the rate of tau accumulation within the EC. Our results support a model whereby Aβ induces disruptions to local excitatory-inhibitory balance in the DMN, driving hyperexcitability in the MTL, leading to tau accumulation. We propose that Aβ-induced disruptions to excitatory-inhibitory balance is a candidate causal route between Aβ and remote EC-tau accumulation.

Active inference as a theory of sentient behavior

This review paper offers an overview of the history and future of active inference-a unifying perspective on action and perception. Active inference is based upon the idea that sentient behavior depends upon our brains' implicit use of internal models to predict, infer, and direct action. Our focus is upon the conceptual roots and development of this theory of (basic) sentience and does not follow a rigid chronological narrative. We trace the evolution from Helmholtzian ideas on unconscious inference, through to a contemporary understanding of action and perception. In doing so, we touch upon related perspectives, the neural underpinnings of active inference, and the opportunities for future development. Key steps in this development include the formulation of predictive coding models and related theories of neuronal message passing, the use of sequential models for planning and policy optimization, and the importance of hierarchical (temporally) deep internal (i.e., generative or world) models. Active inference has been used to account for aspects of anatomy and neurophysiology, to offer theories of psychopathology in terms of aberrant precision control, and to unify extant psychological theories. We anticipate further development in all these areas and note the exciting early work applying active inference beyond neuroscience. This suggests a future not just in biology, but in robotics, machine learning, and artificial intelligence.

Order and change in art: towards an active inference account of aesthetic experience

How to account for the power that art holds over us? Why do artworks touch us deeply, consoling, transforming or invigorating us in the process? In this paper, we argue that an answer to this question might emerge from a fecund framework in cognitive science known as predictive processing (a.k.a. active inference). We unpack how this approach connects sense-making and aesthetic experiences through the idea of an 'epistemic arc', consisting of three parts (curiosity, epistemic action and aha experiences), which we cast as aspects of active inference. We then show how epistemic arcs are built and sustained by artworks to provide us with those satisfying experiences that we tend to call 'aesthetic'. Next, we defuse two key objections to this approach; namely, that it places undue emphasis on the cognitive component of our aesthetic encounters-at the expense of affective aspects-and on closure and uncertainty minimization (order)-at the expense of openness and lingering uncertainty (change). We show that the approach offers crucial resources to account for the open-ended, free and playful behaviour inherent in aesthetic experiences. The upshot is a promising but deflationary approach, both philosophically informed and psychologically sound, that opens new empirical avenues for understanding our aesthetic encounters. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Aesthetics and predictive processing: grounds and prospects of a fruitful encounter

In the last few years, a remarkable convergence of interests and results has emerged between scholars interested in the arts and aesthetics from a variety of perspectives and cognitive scientists studying the mind and brain within the predictive processing (PP) framework. This convergence has so far proven fruitful for both sides: while PP is increasingly adopted as a framework for understanding aesthetic phenomena, the arts and aesthetics, examined under the lens of PP, are starting to be seen as important windows into our mental functioning. The result is a vast and fast-growing research programme that promises to deliver important insights into our aesthetic encounters as well as a wide range of psychological phenomena of general interest. Here, we present this developing research programme, describing its grounds and highlighting its prospects. We start by clarifying how the study of the arts and aesthetics encounters the PP picture of mental functioning (§1). We then go on to outline the prospects of this encounter for the fields involved: philosophy and history of art (§2), psychology of aesthetics and neuroaesthetics (§3) and psychology and neuroscience more generally (§4). The upshot is an ambitious but well-defined framework within which aesthetics and cognitive science can partner up to illuminate crucial aspects of the human mind. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Serial Dependence in Perception

Much evidence has shown that perception is biased towards previously presented similar stimuli, an effect recently termed serial dependence. Serial dependence affects nearly every aspect of perception, often causing gross perceptual distortions, especially for weak and ambiguous stimuli. Despite unwanted side-effects, empirical evidence and Bayesian modeling show that serial dependence acts to improve efficiency and is generally beneficial to the system. Consistent with models of predictive coding, the Bayesian priors of serial dependence are generated at high levels of cortical analysis, incorporating much perceptual experience, but feed back to lower sensory areas. These feedback loops may drive oscillations in the alpha range, linked strongly with serial dependence. The discovery of top-down predictive perceptual processes is not new, but the new, more quantitative approach characterizing serial dependence promises to lead to a deeper understanding of predictive perceptual processes and their underlying neural mechanisms.

Bistable perception, precision and neuromodulation

Bistable perception follows from observing a static, ambiguous, (visual) stimulus with two possible interpretations. Here, we present an active (Bayesian) inference account of bistable perception and posit that perceptual transitions between different interpretations (i.e. inferences) of the same stimulus ensue from specific eye movements that shift the focus to a different visual feature. Formally, these inferences are a consequence of precision control that determines how confident beliefs are and change the frequency with which one can perceive-and alternate between-two distinct percepts. We hypothesized that there are multiple, but distinct, ways in which precision modulation can interact to give rise to a similar frequency of bistable perception. We validated this using numerical simulations of the Necker cube paradigm and demonstrate the multiple routes that underwrite the frequency of perceptual alternation. Our results provide an (enactive) computational account of the intricate precision balance underwriting bistable perception. Importantly, these precision parameters can be considered the computational homologs of particular neurotransmitters-i.e. acetylcholine, noradrenaline, dopamine-that have been previously implicated in controlling bistable perception, providing a computational link between the neurochemistry and perception.

Computational components of visual predictive coding circuitry

If a full visual percept can be said to be a 'hypothesis', so too can a neural 'prediction' - although the latter addresses one particular component of image content (such as 3-dimensional organisation, the interplay between lighting and surface colour, the future trajectory of moving objects, and so on). And, because processing is hierarchical, predictions generated at one level are conveyed in a backward direction to a lower level, seeking to predict, in fact, the neural activity at that prior stage of processing, and learning from errors signalled in the opposite direction. This is the essence of 'predictive coding', at once an algorithm for information processing and a theoretical basis for the nature of operations performed by the cerebral cortex. Neural models for the implementation of predictive coding invoke specific functional classes of neuron for generating, transmitting and receiving predictions, and for producing reciprocal error signals. Also a third general class, 'precision' neurons, tasked with regulating the magnitude of error signals contingent upon the confidence placed upon the prediction, i.e., the reliability and behavioural utility of the sensory data that it predicts. So, what is the ultimate source of a 'prediction'? The answer is multifactorial: knowledge of the current environmental context and the immediate past, allied to memory and lifetime experience of the way of the world, doubtless fine-tuned by evolutionary history too. There are, in consequence, numerous potential avenues for experimenters seeking to manipulate subjects' expectation, and examine the neural signals elicited by surprising, and less surprising visual stimuli. This review focuses upon the predictive physiology of mouse and monkey visual cortex, summarising and commenting on evidence to date, and placing it in the context of the broader field. It is concluded that predictive coding has a firm grounding in basic neuroscience and that, unsurprisingly, there remains much to learn.

A multimodal cortical network of sensory expectation violation revealed by fMRI

The brain is subjected to multi-modal sensory information in an environment governed by statistical dependencies. Mismatch responses (MMRs), classically recorded with EEG, have provided valuable insights into the brain's processing of regularities and the generation of corresponding sensory predictions. Only few studies allow for comparisons of MMRs across multiple modalities in a simultaneous sensory stream and their corresponding cross-modal context sensitivity remains unknown. Here, we used a tri-modal version of the roving stimulus paradigm in fMRI to elicit MMRs in the auditory, somatosensory and visual modality. Participants (N = 29) were simultaneously presented with sequences of low and high intensity stimuli in each of the three senses while actively observing the tri-modal input stream and occasionally reporting the intensity of the previous stimulus in a prompted modality. The sequences were based on a probabilistic model, defining transition probabilities such that, for each modality, stimuli were more likely to repeat (p = .825) than change (p = .175) and stimulus intensities were equiprobable (p = .5). Moreover, each transition was conditional on the configuration of the other two modalities comprising global (cross-modal) predictive properties of the sequences. We identified a shared mismatch network of modality general inferior frontal and temporo-parietal areas as well as sensory areas, where the connectivity (psychophysiological interaction) between these regions was modulated during mismatch processing. Further, we found deviant responses within the network to be modulated by local stimulus repetition, which suggests highly comparable processing of expectation violation across modalities. Moreover, hierarchically higher regions of the mismatch network in the temporo-parietal area around the intraparietal sulcus were identified to signal cross-modal expectation violation. With the consistency of MMRs across audition, somatosensation and vision, our study provides insights into a shared cortical network of uni- and multi-modal expectation violation in response to sequence regularities.

Accelerating scientific progress through Bayesian adversarial collaboration

Adversarial collaboration has been championed as the gold standard for resolving scientific disputes but has gained relatively limited traction in neuroscience and allied fields. In this perspective, we argue that adversarial collaborative research has been stymied by an overly restrictive concern with the falsification of scientific theories. We advocate instead for a more expansive view that frames adversarial collaboration in terms of Bayesian belief updating, model comparison, and evidence accumulation. This framework broadens the scope of adversarial collaboration to accommodate a wide range of informative (but not necessarily definitive) studies while affording the requisite formal tools to guide experimental design and data analysis in the adversarial setting. We provide worked examples that demonstrate how these tools can be deployed to score theoretical models in terms of a common metric of evidence, thereby furnishing a means of tracking the amount of empirical support garnered by competing theories over time.

Autogenous cerebral processes: an invitation to look at the brain from inside out

While external stimulation can reliably trigger neuronal activity, cerebral processes can operate independently from the environment. In this study, we conceptualize autogenous cerebral processes (ACPs) as intrinsic operations of the brain that exist on multiple scales and can influence or shape stimulus responses, behavior, homeostasis, and the physiological state of an organism. We further propose that the field should consider exploring to what extent perception, arousal, behavior, or movement, as well as other cognitive functions previously investigated mainly regarding their stimulus-response dynamics, are ACP-driven.

Local minimization of prediction errors drives learning of invariant object representations in a generative network model of visual perception

The ventral visual processing hierarchy of the cortex needs to fulfill at least two key functions: perceived objects must be mapped to high-level representations invariantly of the precise viewing conditions, and a generative model must be learned that allows, for instance, to fill in occluded information guided by visual experience. Here, we show how a multilayered predictive coding network can learn to recognize objects from the bottom up and to generate specific representations via a top-down pathway through a single learning rule: the local minimization of prediction errors. Trained on sequences of continuously transformed objects, neurons in the highest network area become tuned to object identity invariant of precise position, comparable to inferotemporal neurons in macaques. Drawing on this, the dynamic properties of invariant object representations reproduce experimentally observed hierarchies of timescales from low to high levels of the ventral processing stream. The predicted faster decorrelation of error-neuron activity compared to representation neurons is of relevance for the experimental search for neural correlates of prediction errors. Lastly, the generative capacity of the network is confirmed by reconstructing specific object images, robust to partial occlusion of the inputs. By learning invariance from temporal continuity within a generative model, the approach generalizes the predictive coding framework to dynamic inputs in a more biologically plausible way than self-supervised networks with non-local error-backpropagation. This was achieved simply by shifting the training paradigm to dynamic inputs, with little change in architecture and learning rule from static input-reconstructing Hebbian predictive coding networks.

Markov chains as a proxy for the predictive memory representations underlying mismatch negativity

Events not conforming to a regularity inherent to a sequence of events elicit prediction error signals of the brain such as the Mismatch Negativity (MMN) and impair behavioral task performance. Events conforming to a regularity lead to attenuation of brain activity such as stimulus-specific adaptation (SSA) and behavioral benefits. Such findings are usually explained by theories stating that the information processing system predicts the forthcoming event of the sequence via detected sequential regularities. A mathematical model that is widely used to describe, to analyze and to generate event sequences are Markov chains: They contain a set of possible events and a set of probabilities for transitions between these events (transition matrix) that allow to predict the next event on the basis of the current event and the transition probabilities. The accuracy of such a prediction depends on the distribution of the transition probabilities. We argue that Markov chains also have useful applications when studying cognitive brain functions. The transition matrix can be regarded as a proxy for generative memory representations that the brain uses to predict the next event. We assume that detected regularities in a sequence of events correspond to (a subset of) the entries in the transition matrix. We apply this idea to the Mismatch Negativity (MMN) research and examine three types of MMN paradigms: classical oddball paradigms emphasizing sound probabilities, between-sound regularity paradigms manipulating transition probabilities between adjacent sounds, and action-sound coupling paradigms in which sounds are associated with actions and their intended effects. We show that the Markovian view on MMN yields theoretically relevant insights into the brain processes underlying MMN and stimulates experimental designs to study the brain's processing of event sequences.

No evidence for high inflexible precision of prediction errors in autism during lexical processing

Research has shown that information processing differences associated with autism could impact on language and literacy development. This study tested an approach to autistic cognition that suggests learning occurs via prediction errors, and autistic people have very precise and inflexible predictions that result in more sensitivity to meaningless signal errors than non-autistic readers. We used this theoretical background to investigate whether differences in prediction coding influence how orthographic (Experiment 1) and semantic information (Experiment 2) is processed by autistic readers. Experiment 1 used a lexical decision task to test whether letter position information was processed less flexibly by autistic than non-autistic readers. Three types of letter strings: words, transposed letter and substituted letters nonwords were presented. Experiment 2 used a semantic relatedness task to test whether autistic readers processed words with high and low semantic diversity differently to non-autistic readers. Results showed similar transposed letter and semantic diversity effects for all readers; indicating that orthographic and semantic information are processed similarly by autistic and non-autistic readers; and therefore, differences in prediction coding were not evident for these lexical processing tasks.

Adversarial inference: predictive minds in the attention economy

What is it about our current digital technologies that seemingly makes it difficult for users to attend to what matters to them? According to the dominant narrative in the literature on the "attention economy," a user's lack of attention is due to the large amounts of information available in their everyday environments. I will argue that information-abundance fails to account for some of the central manifestations of distraction, such as sudden urges to check a particular information-source in the absence of perceptual information. I will use active inference, and in particular models of action selection based on the minimization of expected free energy, to develop an alternative answer to the question about what makes it difficult to attend. Besides obvious adversarial forms of inference, in which algorithms build up models of users in order to keep them scrolling, I will show that active inference provides the tools to identify a number of problematic structural features of current digital technologies: they contain limitless sources of novelty, they can be navigated by very simple and effortless motor movements, and they offer their action possibilities everywhere and anytime independent of place or context. Moreover, recent models of motivated control show an intricate interplay between motivation and control that can explain sudden transitions in motivational state and the consequent alteration of the salience of actions. I conclude, therefore, that the challenges users encounter when engaging with digital technologies are less about information overload or inviting content, but more about the continuous availability of easily available possibilities for action.

The impact of spatial variance on precision estimates in an auditory oddball paradigm

Predictive processing theories suggest that a principal function of the brain is to reduce the surprise of incoming sensory information by creating accurate and precise models of the environment. These models are commonly explored by looking at the prediction errors elicited when experience departs from predictions. One such prediction error is the mismatch negativity (MMN). Using this component, it is possible to examine the effect of external noise on the precision of the developed model. Recent studies have shown that the brain may not update its model every time there is a change in the environment, rather it will only update it when doing so will increase precision and or accuracy of the model. The current study examined this process using oddball sound sequences with high and low spatial variability and examining how this affected the elicited MMN to a duration deviant sound. The results showed a strong null effect of spatial variance both at a local and sequence levels. These results indicate that variability in the sound sequence will not invariably affect model precision estimates and thus the amplitude of the MMN component.

Intact predictive processing in autistic adults: evidence from statistical learning

Impairment in predictive processes gained a lot of attention in recent years as an explanation for autistic symptoms. However, empirical evidence does not always underpin this framework. Thus, it is unclear what aspects of predictive processing are affected in autism spectrum disorder. In this study, we tested autistic adults on a task in which participants acquire probability-based regularities (that is, a statistical learning task). Twenty neurotypical and 22 autistic adults learned a probabilistic, temporally distributed regularity for about 40 min. Using frequentist and Bayesian methods, we found that autistic adults performed comparably to neurotypical adults, and the dynamics of learning did not differ between groups either. Thus, our study provides evidence for intact statistical learning in autistic adults. Furthermore, we discuss potential ways this result can extend the scope of the predictive processing framework, noting that atypical processing might not always mean a deficit in performance.

A framework for understanding post-detection deception in predator-prey interactions

Predators and prey exist in persistent conflict that often hinges on deception-the transmission of misleading or manipulative signals-as a means for survival. Deceptive traits are widespread across taxa and sensory systems, representing an evolutionarily successful and common strategy. Moreover, the highly conserved nature of the major sensory systems often extends these traits past single species predator-prey interactions toward a broader set of perceivers. As such, deceptive traits can provide a unique window into the capabilities, constraints and commonalities across divergent and phylogenetically-related perceivers. Researchers have studied deceptive traits for centuries, but a unified framework for categorizing different types of post-detection deception in predator-prey conflict still holds potential to inform future research. We suggest that deceptive traits can be distinguished by their effect on object formation processes. Perceptual objects are composed of physical attributes (what) and spatial (where) information. Deceptive traits that operate after object formation can therefore influence the perception and processing of either or both of these axes. We build upon previous work using a perceiver perspective approach to delineate deceptive traits by whether they closely match the sensory information of another object or create a discrepancy between perception and reality by exploiting the sensory shortcuts and perceptual biases of their perceiver. We then further divide this second category, sensory illusions, into traits that distort object characteristics along either the what or where axes, and those that create the perception of whole novel objects, integrating the what/where axes. Using predator-prey examples, we detail each step in this framework and propose future avenues for research. We suggest that this framework will help organize the many forms of deceptive traits and help generate predictions about selective forces that have driven animal form and behavior across evolutionary time.

Luminance contrast provides metric depth information

The perception of depth from retinal images depends on information from multiple visual cues. One potential depth cue is the statistical relationship between luminance and distance; darker points in a local region of an image tend to be farther away than brighter points. We establish that this statistical relationship acts as a quantitative cue to depth. We show that luminance variations affect depth in naturalistic scenes containing multiple cues to depth. This occurred when the correlation between variations of luminance and depth was manipulated within an object, but not between objects. This is consistent with the local nature of the statistical relationship in natural scenes. We also showed that perceived depth increases as contrast is increased, but only when the depth signalled by luminance and binocular disparity are consistent. Our results show that the negative correlation between luminance and distance, as found under diffuse lighting, provides a depth cue that is combined with depth from binocular disparity, in a way that is consistent with the simultaneous estimation of surface depth and reflectance variations. Adopting more complex lighting models such as ambient occlusion in computer rendering will thus contribute to the accuracy as well as the aesthetic appearance of three-dimensional graphics.

Computational psychiatry: from synapses to sentience

This review considers computational psychiatry from a particular viewpoint: namely, a commitment to explaining psychopathology in terms of pathophysiology. It rests on the notion of a generative model as underwriting (i) sentient processing in the brain, and (ii) the scientific process in psychiatry. The story starts with a view of the brain-from cognitive and computational neuroscience-as an organ of inference and prediction. This offers a formal description of neuronal message passing, distributed processing and belief propagation in neuronal networks; and how certain kinds of dysconnection lead to aberrant belief updating and false inference. The dysconnections in question can be read as a pernicious synaptopathy that fits comfortably with formal notions of how we-or our brains-encode uncertainty or its complement, precision. It then considers how the ensuing process theories are tested empirically, with an emphasis on the computational modelling of neuronal circuits and synaptic gain control that mediates attentional set, active inference, learning and planning. The opportunities afforded by this sort of modelling are considered in light of in silico experiments; namely, computational neuropsychology, computational phenotyping and the promises of a computational nosology for psychiatry. The resulting survey of computational approaches is not scholarly or exhaustive. Rather, its aim is to review a theoretical narrative that is emerging across subdisciplines within psychiatry and empirical scales of investigation. These range from epilepsy research to neurodegenerative disorders; from post-traumatic stress disorder to the management of chronic pain, from schizophrenia to functional medical symptoms.

A generative adversarial model of intrusive imagery in the human brain

The mechanisms underlying the subjective experiences of mental disorders remain poorly understood. This is partly due to long-standing over-emphasis on behavioral and physiological symptoms and a de-emphasis of the patient's subjective experiences when searching for treatments. Here, we provide a new perspective on the subjective experience of mental disorders based on findings in neuroscience and artificial intelligence (AI). Specifically, we propose the subjective experience that occurs in visual imagination depends on mechanisms similar to generative adversarial networks that have recently been developed in AI. The basic idea is that a generator network fabricates a prediction of the world, and a discriminator network determines whether it is likely real or not. Given that similar adversarial interactions occur in the two major visual pathways of perception in people, we explored whether we could leverage this AI-inspired approach to better understand the intrusive imagery experiences of patients suffering from mental illnesses such as post-traumatic stress disorder (PTSD) and acute stress disorder. In our model, a nonconscious visual pathway generates predictions of the environment that influence the parallel but interacting conscious pathway. We propose that in some patients, an imbalance in these adversarial interactions leads to an overrepresentation of disturbing content relative to current reality, and results in debilitating flashbacks. By situating the subjective experience of intrusive visual imagery in the adversarial interaction of these visual pathways, we propose testable hypotheses on novel mechanisms and clinical applications for controlling and possibly preventing symptoms resulting from intrusive imagery.

The response set theory of hypnosis reconsidered: toward an integrative model

Irving Kirsch is a leading figure in the field of psychological science who has advanced our understanding of hypnosis in key respects that have withstood the tests of time and replication. We honor his prodigious contributions over his distinguished career and extend his response expectancy theory in an integrative model that encompasses predictive coding. We review the construct of expectancies that he articulated and championed for decades and extended in response set theory. We propose novel hypotheses to align his innovative contributions with the most current findings in psychological science and to acknowledge the heuristic value of his work. We especially focus on (I) how the response set theory can be conceptualized in terms of the predictive coding model and (II) psycho-social constructs that need to be considered to better understand the effects of expectancies on hypnotic phenomena in an open and evidence-based integrative model of hypnosis.

The language marker hypothesis

According to the language marker hypothesis language has provided homo sapiens with a rich symbolic system that plays a central role in interpreting signals delivered by our sensory apparatus, in shaping action goals, and in creating a powerful tool for reasoning and inferencing. This view provides an important correction on embodied accounts of language that reduce language to action, perception, emotion and mental simulation. The presence of a language system has, however, also important consequences for perception, action, emotion, and memory. Language stamps signals from perception, action, and emotional systems with rich cognitive markers that transform the role of these signals in the overall cognitive architecture of the human mind. This view does not deny that language is implemented by means of universal principles of neural organization. However, language creates the possibility to generate rich internal models of the world that are shaped and made accessible by the characteristics of a language system. This makes us less dependent on direct action-perception couplings and might even sometimes go at the expense of the veridicality of perception. In cognitive (neuro)science the pendulum has swung from language as the key to understand the organization of the human mind to the perspective that it is a byproduct of perception and action. It is time that it partly swings back again.

Learned uncertainty: The free energy principle in anxiety

Generalized anxiety disorder is among the world's most prevalent psychiatric disorders and often manifests as persistent and difficult to control apprehension. Despite its prevalence, there is no integrative, formal model of how anxiety and anxiety disorders arise. Here, we offer a perspective derived from the free energy principle; one that shares similarities with established constructs such as learned helplessness. Our account is simple: anxiety can be formalized as learned uncertainty. A biological system, having had persistent uncertainty in its past, will expect uncertainty in its future, irrespective of whether uncertainty truly persists. Despite our account's intuitive simplicity-which can be illustrated with the mere flip of a coin-it is grounded within the free energy principle and hence situates the formation of anxiety within a broader explanatory framework of biological self-organization and self-evidencing. We conclude that, through conceptualizing anxiety within a framework of working generative models, our perspective might afford novel approaches in the clinical treatment of anxiety and its key symptoms.

Visual priming and serial dependence are mediated by separate mechanisms

Perceptual history influences current perception, readily revealed by visual priming (the facilitation of responses on repeated presentations of similar stimuli) and by serial dependence (systematic biases toward the previous stimuli). We asked whether the two phenomena shared perceptual mechanisms. We modified the standard “priming of pop-out” paradigm to measure both priming and serial dependence concurrently. The stimulus comprised three grating patches, one or two red, and the other green. Participants identified the color singleton (either red or green), and reproduced its orientation. Trial sequences were designed to maximize serial dependence, and long runs of priming color and position. The results showed strong effects of priming, both on reaction times and accuracy, which accumulated steadily over time, as generally reported in the literature. The serial dependence effects were also strong, but did not depend on previous color, nor on the run length. Reaction times measured under various conditions of repetition or change of priming color or position were reliably correlated with imprecision in orientation reproduction, but reliably uncorrelated with magnitude of serial dependence. The results suggest that visual priming and serial dependence are mediated by different neural mechanisms. We propose that priming affects sensitivity, possibly via attention-like mechanisms, whereas serial dependence affects criteria, two orthogonal dimensions in the signal detection theory.

The Effects of Adding Pictorial Depth Cues to the Poggendorff Illusion

We tested if the misapplication of perceptual constancy mechanisms might explain the perceived misalignment of the oblique lines in the Poggendorff illusion. Specifically, whether these mechanisms might treat the rectangle in the middle portion of the Poggendorff stimulus as an occluder in front of one long line appearing on either side, causing an apparent decrease in the rectangle's width and an apparent increase in the misalignment of the oblique lines. The study aimed to examine these possibilities by examining the effects of adding pictorial depth cues. In experiments 1 and 2, we presented a central rectangle composed of either large or small bricks to determine if this manipulation would change the perceived alignment of the oblique lines and the perceived width of the central rectangle, respectively. The experiments demonstrated no changes that would support a misapplication of perceptual constancy in driving the illusion, despite some evidence of perceptual size rescaling of the central rectangle. In experiment 3, we presented Poggendorff stimuli in front and at the back of a corridor background rich in texture and linear perspective depth cues to determine if adding these cues would affect the Poggendorff illusion. The central rectangle was physically large and small when presented in front and at the back of the corridor, respectively. The strength of the Poggendorff illusion varied as a function of the physical size of the central rectangle, and, contrary to our predictions, the addition of pictorial depth cues in both the central rectangle and the background decreased rather than increased the strength of the illusion. The implications of these results with regards to different theories are discussed. It could be the case that the illusion depends on both low-level and cognitive mechanisms and that deleterious effects occur on the former when the latter ascribes more certainty to the oblique lines being the same line receding into the distance.

Choices by organisms: on the role of freedom in behaviour and evolution

Neo-Darwinian biology has demonstrated that it is possible to construct a theory of life that excludes the role of organisms’ free choice. In a richer theory, the latter as a possibility needs to be taken into account. For that purpose, it is necessary to introduce the biological concept of choice, analyse its structure and roles, and consider some implications for biological theory. It is argued here that the conditions for free choice emerge together with umwelt—the space of synchronous options. Basically, choice does not require purpose. This leads to the conclusion that freedom is an attribute of life.

The promise of layer-specific neuroimaging for testing predictive coding theories of psychosis

Predictive coding potentially provides an explanatory model for understanding the neurocognitive mechanisms of psychosis. It proposes that cognitive processes, such as perception and inference, are implemented by a hierarchical system, with the influence of each level being a function of the estimated precision of beliefs at that level. However, predictive coding models of psychosis are insufficiently constrained-any phenomenon can be explained in multiple ways by postulating different changes to precision at different levels of processing. One reason for the lack of constraint in these models is that the core processes are thought to be implemented by the function of specific cortical layers, and the technology to measure layer specific neural activity in humans has until recently been lacking. As a result, our ability to constrain the models with empirical data has been limited. In this review we provide a brief overview of predictive processing models of psychosis and then describe the potential for newly developed, layer specific neuroimaging techniques to test and thus constrain these models. We conclude by discussing the most promising avenues for this research as well as the technical and conceptual challenges which may limit its application.

What is neurorepresentationalism? From neural activity and predictive processing to multi-level representations and consciousness

This review provides an update on Neurorepresentationalism, a theoretical framework that defines conscious experience as multimodal, situational survey and explains its neural basis from brain systems constructing best-guess representations of sensations originating in our environment and body [1]. It posits that conscious experience is characterized by five essential hallmarks: (i) multimodal richness, (ii) situatedness and immersion, (iii) unity and integration, (iv) dynamics and stability, and (v) intentionality. Consciousness is furthermore proposed to have a biological function, framed by the contrast between reflexes and habits (not requiring consciousness) versus goal-directed, planned behavior (requiring multimodal, situational survey). Conscious experience is therefore understood as a sensorily rich, spatially encompassing representation of body and environment, while we nevertheless have the impression of experiencing external reality directly. Contributions to understanding neural mechanisms underlying consciousness are derived from models for predictive processing, which are trained in an unsupervised manner, do not necessarily require overt action, and have been extended to deep neural networks. Even with predictive processing in place, however, the question remains why this type of neural network activity would give rise to phenomenal experience. Here, I propose to tackle the Hard Problem with the concept of multi-level representations which emergently give rise to multimodal, spatially wide superinferences corresponding to phenomenal experiences. Finally, Neurorepresentationalism is compared to other neural theories of consciousness, and its implications for defining indicators of consciousness in animals, artificial intelligence devices and immobile or unresponsive patients with disorders of consciousness are discussed.

The Nature of Illusions: A New Synthesis Based on Verifiability

This overview discusses the nature of perceptual illusions with particular reference to the theory that illusions represent the operation of a sensory code for which there is no meaningful ground truth against which the illusory percepts can be compared, and therefore there are no illusions as such. This view corresponds to the Bayesian theory that “illusions” reflect unusual aspects of the core strategies of adapting to the natural world, again implying that illusions are simply an information processing characteristic. Instead, it is argued that a more meaningful approach to the field that we call illusions is the Ebbinghaus approach of comparing the illusory percept with a ground truth that is directly verifiable as aberrant by the observer in the domain of the illusory phenomenology (as opposed to relying on the authority of other experts). This concept of direct verifiability not only provides an operational definition of “illusion”; it also makes their interactive observation more effective and informative as to the perceptual processes underlying the illusory appearance. An expanded version of Gregory’s categorization of types of illusion is developed, and a range of classic and more recent illusions that illustrate the differences between these philosophical viewpoints is considered in detail. Such cases make it clear that the discrepancies from the measurable image structure cannot be simply regarded as idiosyncrasies of sensory coding, but are categorical exemplars of perceptual illusions. The widespread existence of such illusory percepts is indicative of the evolutionary limits of adaptive sensory coding.

More efficient formation of longer-term representations for word forms at birth can be linked to better language skills at 2 years

Infants are able to extract words from speech early in life. Here we show that the quality of forming longer-term representations for word forms at birth predicts expressive language ability at the age of two years. Seventy-five neonates were familiarized with two spoken disyllabic pseudowords. We then tested whether the neonate brain predicts the second syllable from the first one by presenting a familiarized pseudoword frequently, and occasionally violating the learned syllable combination by different rare pseudowords. Distinct brain responses were elicited by predicted and unpredicted word endings, suggesting that the neonates had learned the familiarized pseudowords. The difference between responses to predicted and unpredicted pseudowords indexing the quality of word-form learning during familiarization significantly correlated with expressive language scores (the mean length of utterance) at 24 months in the same infant. These findings suggest that 1) neonates can memorize disyllabic words so that a learned first syllable generates predictions for the word ending, and 2) early individual differences in the quality of word-form learning correlate with language skills. This relationship helps early identification of infants at risk for language impairment.

Music in the brain

Music is ubiquitous across human cultures - as a source of affective and pleasurable experience, moving us both physically and emotionally - and learning to play music shapes both brain structure and brain function. Music processing in the brain - namely, the perception of melody, harmony and rhythm - has traditionally been studied as an auditory phenomenon using passive listening paradigms. However, when listening to music, we actively generate predictions about what is likely to happen next. This enactive aspect has led to a more comprehensive understanding of music processing involving brain structures implicated in action, emotion and learning. Here we review the cognitive neuroscience literature of music perception. We show that music perception, action, emotion and learning all rest on the human brain's fundamental capacity for prediction - as formulated by the predictive coding of music model. This Review elucidates how this formulation of music perception and expertise in individuals can be extended to account for the dynamics and underlying brain mechanisms of collective music making. This in turn has important implications for human creativity as evinced by music improvisation. These recent advances shed new light on what makes music meaningful from a neuroscientific perspective.

Making sense of periodicity glimpses in a prediction-update-loop-A computational model of attentive voice tracking

Humans are able to follow a speaker even in challenging acoustic conditions. The perceptual mechanisms underlying this ability remain unclear. A computational model of attentive voice tracking, consisting of four computational blocks: (1) sparse periodicity-based auditory features (sPAF) extraction, (2) foreground-background segregation, (3) state estimation, and (4) top-down knowledge, is presented. The model connects the theories about auditory glimpses, foreground-background segregation, and Bayesian inference. It is implemented with the sPAF, sequential Monte Carlo sampling, and probabilistic voice models. The model is evaluated by comparing it with the human data obtained in the study by Woods and McDermott [Curr. Biol. 25(17), 2238-2246 (2015)], which measured the ability to track one of two competing voices with time-varying parameters [fundamental frequency (F0) and formants (F1,F2)]. Three model versions were tested, which differ in the type of information used for the segregation: version (a) uses the oracle F0, version (b) uses the estimated F0, and version (c) uses the spectral shape derived from the estimated F0 and oracle F1 and F2. Version (a) simulates the optimal human performance in conditions with the largest separation between the voices, version (b) simulates the conditions in which the separation in not sufficient to follow the voices, and version (c) is closest to the human performance for moderate voice separation.

EXPRESS: "Run to the hills": Specific contributions of anticipated energy expenditure during active spatial learning

Grounded views of cognition consider that space perception is shaped by the body and its potential for action. These views are substantiated by observations such as the distance-on-hill effect, described as the overestimation of visually perceived uphill distances. An interpretation of this phenomenon is that slanted distances are overestimated because of the integration of energy expenditure cues. The visual perceptual processes involved are however usually tackled through explicit estimation tasks in passive situations. The goal of this study was to consider instead more ecological active spatial processing. Using immersive virtual reality and an omnidirectional treadmill, we investigated the effect of anticipated implicit physical locomotion cost by comparing spatial learning for uphill and downhill routes, while maintaining actual physical cost and walking speed constant. In the first experiment, participants learnt city layouts by exploring uphill or downhill routes. They were then tested using a landmark positioning task on a map. In the second experiment, the same protocol was used with participants who wore loaded ankle weights. Results from the first experiment showed that walking uphill routes led to a global underestimation of distances compared to downhill routes. This inverted distance-of-hill effect was not observed in the second experiment, where an additional effort was applied. These results suggest that the underestimation of distances observed in experiment one emerged from recalibration processes whose function was to solve the transgression of proprioceptive predictions linked with uphill energy expenditure. Results are discussed in relation to constructivist approaches on spatial representations and predictive coding theories.

Multimodal Representation Learning for Place Recognition Using Deep Hebbian Predictive Coding

Recognising familiar places is a competence required in many engineering applications that interact with the real world such as robot navigation. Combining information from different sensory sources promotes robustness and accuracy of place recognition. However, mismatch in data registration, dimensionality, and timing between modalities remain challenging problems in multisensory place recognition. Spurious data generated by sensor drop-out in multisensory environments is particularly problematic and often resolved through adhoc and brittle solutions. An effective approach to these problems is demonstrated by animals as they gracefully move through the world. Therefore, we take a neuro-ethological approach by adopting self-supervised representation learning based on a neuroscientific model of visual cortex known as predictive coding. We demonstrate how this parsimonious network algorithm which is trained using a local learning rule can be extended to combine visual and tactile sensory cues from a biomimetic robot as it naturally explores a visually aliased environment. The place recognition performance obtained using joint latent representations generated by the network is significantly better than contemporary representation learning techniques. Further, we see evidence of improved robustness at place recognition in face of unimodal sensor drop-out. The proposed multimodal deep predictive coding algorithm presented is also linearly extensible to accommodate more than two sensory modalities, thereby providing an intriguing example of the value of neuro-biologically plausible representation learning for multimodal navigation.

Understanding, Explanation, and Active Inference

While machine learning techniques have been transformative in solving a range of problems, an important challenge is to understand why they arrive at the decisions they output. Some have argued that this necessitates augmenting machine intelligence with understanding such that, when queried, a machine is able to explain its behaviour (i.e., explainable AI). In this article, we address the issue of machine understanding from the perspective of active inference. This paradigm enables decision making based upon a model of how data are generated. The generative model contains those variables required to explain sensory data, and its inversion may be seen as an attempt to explain the causes of these data. Here we are interested in explanations of one's own actions. This implies a deep generative model that includes a model of the world, used to infer policies, and a higher-level model that attempts to predict which policies will be selected based upon a space of hypothetical (i.e., counterfactual) explanations-and which can subsequently be used to provide (retrospective) explanations about the policies pursued. We illustrate the construct validity of this notion of understanding in relation to human understanding by highlighting the similarities in computational architecture and the consequences of its dysfunction.

Trust as Extended Control: Human-Machine Interactions as Active Inference

In order to interact seamlessly with robots, users must infer the causes of a robot's behavior-and be confident about that inference (and its predictions). Hence, trust is a necessary condition for human-robot collaboration (HRC). However, and despite its crucial role, it is still largely unknown how trust emerges, develops, and supports human relationship to technological systems. In the following paper we review the literature on trust, human-robot interaction, HRC, and human interaction at large. Early models of trust suggest that it is a trade-off between benevolence and competence; while studies of human to human interaction emphasize the role of shared behavior and mutual knowledge in the gradual building of trust. We go on to introduce a model of trust as an agent' best explanation for reliable sensory exchange with an extended motor plant or partner. This model is based on the cognitive neuroscience of active inference and suggests that, in the context of HRC, trust can be casted in terms of virtual control over an artificial agent. Interactive feedback is a necessary condition to the extension of the trustor's perception-action cycle. This model has important implications for understanding human-robot interaction and collaboration-as it allows the traditional determinants of human trust, such as the benevolence and competence attributed to the trustee, to be defined in terms of hierarchical active inference, while vulnerability can be described in terms of information exchange and empowerment. Furthermore, this model emphasizes the role of user feedback during HRC and suggests that boredom and surprise may be used in personalized interactions as markers for under and over-reliance on the system. The description of trust as a sense of virtual control offers a crucial step toward grounding human factors in cognitive neuroscience and improving the design of human-centered technology. Furthermore, we examine the role of shared behavior in the genesis of trust, especially in the context of dyadic collaboration, suggesting important consequences for the acceptability and design of human-robot collaborative systems.

ImageNet-trained deep neural networks exhibit illusion-like response to the Scintillating grid

Deep neural network (DNN) models for computer vision are capable of human-level object recognition. Consequently, similarities between DNN and human vision are of interest. Here, we characterize DNN representations of Scintillating grid visual illusion images in which white disks are perceived to be partially black. Specifically, we use VGG-19 and ResNet-101 DNN models that were trained for image classification and consider the representational dissimilarity (\(L^1\) distance in the penultimate layer) between pairs of images: one with white Scintillating grid disks and the other with disks of decreasing luminance levels. Results showed a nonmonotonic relation, such that decreasing disk luminance led to an increase and subsequently a decrease in representational dissimilarity. That is, the Scintillating grid image with white disks was closer, in terms of the representation, to images with black disks than images with gray disks. In control nonillusion images, such nonmonotonicity was rare. These results suggest that nonmonotonicity in a deep computational representation is a potential test for illusion-like response geometry in DNN models.

A parameter-free statistical test for neuronal responsiveness

Neurophysiological studies depend on a reliable quantification of whether and when a neuron responds to stimulation. Simple methods to determine responsiveness require arbitrary parameter choices, such as binning size, while more advanced model-based methods require fitting and hyperparameter tuning. These parameter choices can change the results, which invites bad statistical practice and reduces the replicability. New recording techniques that yield increasingly large numbers of cells would benefit from a test for cell-inclusion that requires no manual curation. Here, we present the parameter-free ZETA-test, which outperforms t-tests, ANOVAs, and renewal-process-based methods by including more cells at a similar false-positive rate. We show that our procedure works across brain regions and recording techniques, including calcium imaging and Neuropixels data. Furthermore, in illustration of the method, we show in mouse visual cortex that (1) visuomotor-mismatch and spatial location are encoded by different neuronal subpopulations and (2) optogenetic stimulation of VIP cells leads to early inhibition and subsequent disinhibition.