Stimulus predictability reduces responses in primary visual cortex

Predictions enable top-down pattern separation in the macaque face-processing hierarchy

Distinguishing faces requires well distinguishable neural activity patterns. Contextual information may separate neural representations, leading to enhanced identity recognition. Here, we use functional magnetic resonance imaging to investigate how predictions derived from contextual information affect the separability of neural activity patterns in the macaque face-processing system, a 3-level processing hierarchy in ventral visual cortex. We find that in the presence of predictions, early stages of this hierarchy exhibit well separable and high-dimensional neural geometries resembling those at the top of the hierarchy. This is accompanied by a systematic shift of tuning properties from higher to lower areas, endowing lower areas with higher-order, invariant representations instead of their feedforward tuning properties. Thus, top-down signals dynamically transform neural representations of faces into separable and high-dimensional neural geometries. Our results provide evidence how predictive context transforms flexible representational spaces to optimally use the computational resources provided by cortical processing hierarchies for better and faster distinction of facial identities.

Concurrent Encoding of Sequence Predictability and Event-Evoked Prediction Error in Unfolding Auditory Patterns

Human listeners possess an innate capacity to discern patterns within rapidly unfolding sensory input. Core questions, guiding ongoing research, focus on the mechanisms through which these representations are acquired and whether the brain prioritizes or suppresses predictable sensory signals. Previous work, using fast auditory sequences (tone-pips presented at a rate of 20 Hz), revealed sustained response effects that appear to track the dynamic predictability of the sequence. Here, we extend the investigation to slower sequences (4 Hz), permitting the isolation of responses to individual tones. Stimuli were 50 ms tone-pips, ordered into random (RND) and regular (REG; a repeating pattern of 10 frequencies) sequences; Two timing profiles were created: in "fast" sequences, tone-pips were presented in direct succession (20 Hz); in "slow" sequences, tone-pips were separated by a 200 ms silent gap (4 Hz). Naive participants (N = 22; both sexes) passively listened to these sequences, while brain responses were recorded using magnetoencephalography (MEG). Results unveiled a heightened magnitude of sustained brain responses in REG when compared to RND patterns. This manifested from three tones after the onset of the pattern repetition, even in the context of slower sequences characterized by extended pattern durations (2,500 ms). This observation underscores the remarkable implicit sensitivity of the auditory brain to acoustic regularities. Importantly, brain responses evoked by single tones exhibited the opposite pattern-stronger responses to tones in RND than REG sequences. The demonstration of simultaneous but opposing sustained and evoked response effects reveals concurrent processes that shape the representation of unfolding auditory patterns.

The theta-gamma code in predictive processing and mnemonic updating

Predictive processing has become a leading theory about how the brain works. Yet, it remains an open question how predictive processes are realized in the brain. Here I discuss theta-gamma coupling as one potential neural mechanism for prediction and model updating. Building on Lisman and colleagues SOCRATIC model, theta-gamma coupling has been associated with phase precession and learning phenomena in medio-temporal lobe of rodents, where it completes and retains a sequence of places or items (i.e., predictive models). These sequences may be updated upon prediction errors (i.e., model updating), signaled by dopaminergic inputs from prefrontal networks. This framework, spanning the molecular to the network level, matches excitingly well with recent findings on predictive processing, mnemonic updating, and perceptual foraging for the theta-gamma code in human cognition. In sum, I use the case of theta-gamma coupling to link the predictive processing account, a very general concept of how the brain works, to specific neural processes which may implement predictive processing and model updating at the cognitive, network, cellular and molecular level.

Dynamic fading memory and expectancy effects in the monkey primary visual cortex

In order to investigate the involvement of the primary visual cortex (V1) in working memory (WM), parallel, multisite recordings of multi-unit activity were obtained from monkey V1 while the animals performed a delayed match-to-sample (DMS) task. During the delay period, V1 population firing rate vectors maintained a lingering trace of the sample stimulus that could be reactivated by intervening impulse stimuli that enhanced neuronal firing. This fading trace of the sample did not require active engagement of the monkeys in the DMS task and likely reflects the intrinsic dynamics of recurrent cortical networks in lower visual areas. This renders an active, attention-dependent involvement of V1 in the maintenance of WM contents unlikely. By contrast, population responses to the test stimulus depended on the probabilistic contingencies between sample and test stimuli. Responses to tests that matched expectations were reduced which agrees with concepts of predictive coding.

How previous experience shapes future affective subjective ratings: A follow-up study investigating implicit learning and cue ambiguity

People use their previous experience to predict future affective events. Since we live in ever-changing environments, affective predictions must generalize from past contexts (from which they may be implicitly learned) to new, potentially ambiguous contexts. This study investigated how past (un)certain relationships influence subjective experience following new ambiguous cues, and whether past relationships can be learned implicitly. Two S1-S2 paradigms were employed as learning and test phases in two experiments. S1s were colored circles, S2s negative or neutral affective pictures. Participants (Experiment 1 N = 121, Experiment 2 N = 116) were assigned to the certain (CG) or uncertain group (UG), and they were presented with 100% (CG) or 50% (UG) S1-S2 congruency during an uninstructed (Experiment 1) or implicit (Experiment 2) learning phase. During the test phase both groups were presented with a new 75% S1-S2 paradigm, and ambiguous (Experiment 1) or unambiguous (Experiment 2) S1s. Participants were asked to rate the expected valence of upcoming S2s (expectancy ratings), or their experienced valence and arousal (valence and arousal ratings). In Experiment 1 ambiguous cues elicited less negative expectancy ratings, and less unpleasant valence ratings, independently of prior experience. In Experiment 2, both groups showed similar expectancies, predicting upcoming pictures' valence according to the 75% contingencies of the test phase. Overall, we found that in the presence of ambiguous cues subjective affective experience is dampened, and that implicit previous experience does not emerge at the subjective level by significantly shaping reported affective experience.

Representation of event and object concepts in ventral anterior temporal lobe and angular gyrus

Semantic knowledge includes understanding of objects and their features and also understanding of the characteristics of events. The hub-and-spoke theory holds that these conceptual representations rely on multiple information sources that are integrated in a central hub in the ventral anterior temporal lobes. The dual-hub theory expands this framework with the claim that the ventral anterior temporal lobe hub is specialized for object representation, while a second hub in angular gyrus is specialized for event representation. To test these ideas, we used representational similarity analysis, univariate and psychophysiological interaction analyses of fMRI data collected while participants processed object and event concepts (e.g. "an apple," "a wedding") presented as images and written words. Representational similarity analysis showed that angular gyrus encoded event concept similarity more than object similarity, although the left angular gyrus also encoded object similarity. Bilateral ventral anterior temporal lobes encoded both object and event concept structure, and left ventral anterior temporal lobe exhibited stronger coding for events. Psychophysiological interaction analysis revealed greater connectivity between left ventral anterior temporal lobe and right pMTG, and between right angular gyrus and bilateral ITG and middle occipital gyrus, for event concepts compared to object concepts. These findings support the specialization of angular gyrus for event semantics, though with some involvement in object coding, but do not support ventral anterior temporal lobe specialization for object concepts.

Metacognitive Feelings: A Predictive-Processing Perspective

Metacognitive feelings are affective experiences that concern the subject's mental processes and capacities. Paradigmatic examples include the feeling of familiarity, the feeling of confidence, or the tip-of-the-tongue experience. In this article, we advance an account of metacognitive feelings based on the predictive-processing framework. The core tenet of predictive processing is that the brain is a hierarchical hypothesis-testing mechanism, predicting sensory input on the basis of prior experience and updating predictions on the basis of the incoming prediction error. According to the proposed account, metacognitive feelings arise out of a process in which visceral changes serve as cues to predict the error dynamics relating to a particular mental process. The expected rate of prediction-error reduction corresponds to the valence at the core of the emerging metacognitive feeling. Metacognitive feelings use prediction dynamics to model the agent's situation in a way that is both descriptive and directive. Thus, metacognitive feelings are not only an appraisal of ongoing cognitive performance but also a set of action policies. These action policies span predictive trajectories across bodily action, mental action, and interoceptive changes, which together transform the epistemic landscape within which metacognitive feelings unfold.

Temporal features of size constancy for perception and action in a real-world setting: A combined EEG-kinematics study

A stable representation of object size, in spite of continuous variations in retinal input due to changes in viewing distance, is critical for perceiving and acting in a real 3D world. In fact, our perceptual and visuo-motor systems exhibit size and grip constancies in order to compensate for the natural shrinkage of the retinal image with increased distance. The neural basis of this size-distance scaling remains largely unknown, although multiple lines of evidence suggest that size-constancy operations might take place remarkably early, already at the level of the primary visual cortex. In this study, we examined for the first time the temporal dynamics of size constancy during perception and action by using a combined measurement of event-related potentials (ERPs) and kinematics. Participants were asked to maintain their gaze steadily on a fixation point and perform either a manual estimation or a grasping task towards disks of different sizes placed at different distances. Importantly, the physical size of the target was scaled with distance to yield a constant retinal angle. Meanwhile, we recorded EEG data from 64 scalp electrodes and hand movements with a motion capture system. We focused on the first positive-going visual evoked component peaking at approximately 90 ms after stimulus onset. We found earlier latencies and greater amplitudes in response to bigger than smaller disks of matched retinal size, regardless of the task. In line with the ERP results, manual estimates and peak grip apertures were larger for the bigger targets. We also found task-related differences at later stages of processing from a cluster of central electrodes, whereby the mean amplitude of the P2 component was greater for manual estimation than grasping. Taken together, these findings provide novel evidence that size constancy for real objects at real distances occurs at the earliest cortical stages and that early visual processing does not change as a function of task demands.

Resource-rational account of sequential effects in human prediction

An abundant literature reports on 'sequential effects' observed when humans make predictions on the basis of stochastic sequences of stimuli. Such sequential effects represent departures from an optimal, Bayesian process. A prominent explanation posits that humans are adapted to changing environments, and erroneously assume non-stationarity of the environment, even if the latter is static. As a result, their predictions fluctuate over time. We propose a different explanation in which sub-optimal and fluctuating predictions result from cognitive constraints (or costs), under which humans however behave rationally. We devise a framework of costly inference, in which we develop two classes of models that differ by the nature of the constraints at play: in one case the precision of beliefs comes at a cost, resulting in an exponential forgetting of past observations, while in the other beliefs with high predictive power are favored. To compare model predictions to human behavior, we carry out a prediction task that uses binary random stimuli, with probabilities ranging from 0.05 to 0.95. Although in this task the environment is static and the Bayesian belief converges, subjects' predictions fluctuate and are biased toward the recent stimulus history. Both classes of models capture this 'attractive effect', but they depart in their characterization of higher-order effects. Only the precision-cost model reproduces a 'repulsive effect', observed in the data, in which predictions are biased away from stimuli presented in more distant trials. Our experimental results reveal systematic modulations in sequential effects, which our theoretical approach accounts for in terms of rationality under cognitive constraints.

Computational components of visual predictive coding circuitry

If a full visual percept can be said to be a 'hypothesis', so too can a neural 'prediction' - although the latter addresses one particular component of image content (such as 3-dimensional organisation, the interplay between lighting and surface colour, the future trajectory of moving objects, and so on). And, because processing is hierarchical, predictions generated at one level are conveyed in a backward direction to a lower level, seeking to predict, in fact, the neural activity at that prior stage of processing, and learning from errors signalled in the opposite direction. This is the essence of 'predictive coding', at once an algorithm for information processing and a theoretical basis for the nature of operations performed by the cerebral cortex. Neural models for the implementation of predictive coding invoke specific functional classes of neuron for generating, transmitting and receiving predictions, and for producing reciprocal error signals. Also a third general class, 'precision' neurons, tasked with regulating the magnitude of error signals contingent upon the confidence placed upon the prediction, i.e., the reliability and behavioural utility of the sensory data that it predicts. So, what is the ultimate source of a 'prediction'? The answer is multifactorial: knowledge of the current environmental context and the immediate past, allied to memory and lifetime experience of the way of the world, doubtless fine-tuned by evolutionary history too. There are, in consequence, numerous potential avenues for experimenters seeking to manipulate subjects' expectation, and examine the neural signals elicited by surprising, and less surprising visual stimuli. This review focuses upon the predictive physiology of mouse and monkey visual cortex, summarising and commenting on evidence to date, and placing it in the context of the broader field. It is concluded that predictive coding has a firm grounding in basic neuroscience and that, unsurprisingly, there remains much to learn.

Event Probabilities Have a Different Impact on Early and Late Electroencephalographic Measures Regarded as Metrics of Prediction

The oddball protocol has been used to study the neural and perceptual consequences of implicit predictions in the human brain. The protocol involves presenting a sequence of identical repeated events that are eventually broken by a novel "oddball" presentation. Oddball presentations have been linked to increased neural responding and to an exaggeration of perceived duration relative to repeated events. Because the number of repeated events in such protocols is circumscribed, as more repeats are encountered, the conditional probability of a further repeat decreases-whereas the conditional probability of an oddball increases. These facts have not been appreciated in many analyses of oddballs; repeats and oddballs have rather been treated as binary event categories. Here, we show that the human brain is sensitive to conditional event probabilities in an active, visual oddball paradigm. P300 responses (a relatively late component of visually evoked potentials measured with EEG) tended to be greater for less likely oddballs and repeats. By contrast, P1 responses (an earlier component) increased for repeats as a goal-relevant target presentation neared, but this effect occurred even when repeat probabilities were held constant, and oddball P1 responses were invariant. We also found that later, more likely oddballs seemed to last longer, and this effect was largely independent of the number of preceding repeats. These findings speak against a repetition suppression account of the temporal oddball effect. Overall, our data highlight an impact of event probability on later, rather than earlier, electroencephalographic measures previously related to predictive processes-and the importance of considering conditional probabilities in sequential presentation paradigms.

Neural mechanisms of visual motion extrapolation

Because neural processing takes time, the brain only has delayed access to sensory information. When localising moving objects this is problematic, as an object will have moved on by the time its position has been determined. Here, we consider predictive motion extrapolation as a fundamental delay-compensation strategy. From a population-coding perspective, we outline how extrapolation can be achieved by a forwards shift in the population-level activity distribution. We identify general mechanisms underlying such shifts, involving various asymmetries which facilitate the targeted 'enhancement' and/or 'dampening' of population-level activity. We classify these on the basis of their potential implementation (intra- vs inter-regional processes) and consider specific examples in different visual regions. We consider how motion extrapolation can be achieved during inter-regional signaling, and how asymmetric connectivity patterns which support extrapolation can emerge spontaneously from local synaptic learning rules. Finally, we consider how more abstract 'model-based' predictive strategies might be implemented. Overall, we present an integrative framework for understanding how the brain determines the real-time position of moving objects, despite neural delays.

Separate and overlapping mechanisms of statistical regularities and salience processing in the occipital cortex and dorsal attention network

Attention selects behaviorally relevant inputs for in-depth processing. Beside the role of traditional signals related to goal-directed and stimulus-driven control, a debate exists regarding the mechanisms governing the effect of statistical regularities on attentional selection, and how these are integrated with other control signals. Using a visuo-spatial search task under fMRI, we tested the joint effects of statistical regularities and stimulus-driven salience. We found that both types of signals modulated occipital activity in a spatially specific manner. Salience acted primarily by reducing the attention bias towards the target location when associated with irrelevant distractors, while statistical regularities reduced this attention bias when the target was presented at a low probability location, particularly at the lower levels of the visual hierarchy. In addition, we found that both statistical regularities and salience activated the dorsal frontoparietal network. Additional exploratory analyses of functional connectivity revealed that only statistical regularities modulated the inter-regional coupling between the posterior parietal cortex and the occipital cortex. These results show that statistical regularities and salience signals are both spatially represented at the occipital level, but that their integration into attentional processing priorities relies on dissociable brain mechanisms.

Top-down influence of areas 21a and 7 differently affects the surround suppression of V1 neurons in cats

Surround suppression (SS) is a phenomenon whereby a neuron's response to stimuli in its central receptive field (cRF) is suppressed by stimuli extending to its surround receptive field (sRF). Recent evidence show that top-down influence contributed to SS in the primary visual cortex (V1). However, how the top-down influence from different high-level cortical areas affects SS in V1 has not been comparatively observed. The present study applied transcranial direct current stimulation (tDCS) to modulate the neural activity in area 21a (A21a) and area 7 (A7) of cats and examined the changes in the cRF and sRF of V1 neurons. We found that anode-tDCS at A21a reduced V1 neurons' cRF size and increased their response to visual stimuli in cRF, causing an improved SS strength. By contrast, anode-tDCS at A7 increased V1 neurons' sRF size and response to stimuli in cRF, also enhancing the SS. Modeling analysis based on DoG function indicated that the increased SS of V1 neurons after anode-tDCS at A21a could be explained by a center-only mechanism, whereas the improved SS after anode-tDCS at A7 might be mediated through a combined center and surround mechanism. In conclusion, A21a and A7 may affect the SS of V1 neurons through different mechanisms.

Same but different: The latency of a shared expectation signal interacts with stimulus attributes

Predictive coding theories assert that perceptual inference is a hierarchical process of belief updating, wherein the onset of unexpected sensory data causes so-called prediction error responses that calibrate erroneous inferences. Given the functionally specialised organisation of visual cortex, it is assumed that prediction error propagation interacts with the specific visual attribute violating an expectation. We sought to test this within the temporal domain by applying time-resolved decoding methods to electroencephalography (EEG) data evoked by contextual trajectory violations of either brightness, size, or orientation within a bound stimulus. We found that following ∼170 ms post stimulus onset, responses to both size violations and orientation violations were decodable from physically identical control trials in which no attributes were violated. These two violation types were then directly compared, with attribute-specific signalling being decoded from 265 ms. Temporal generalisation suggested that this dissociation was driven by latency shifts in shared expectation signalling between the two conditions. Using a novel temporal bias method, we then found that this shared signalling occurred earlier for size violations than orientation violations. To our knowledge, we are among the first to decode expectation violations in humans using EEG and have demonstrated a temporal dissociation in attribute-specific expectancy violations.

Multisensory perception constrains the formation of object categories: a review of evidence from sensory-driven and predictive processes on categorical decisions

Although object categorization is a fundamental cognitive ability, it is also a complex process going beyond the perception and organization of sensory stimulation. Here we review existing evidence about how the human brain acquires and organizes multisensory inputs into object representations that may lead to conceptual knowledge in memory. We first focus on evidence for two processes on object perception, multisensory integration of redundant information (e.g. seeing and feeling a shape) and crossmodal, statistical learning of complementary information (e.g. the 'moo' sound of a cow and its visual shape). For both processes, the importance attributed to each sensory input in constructing a multisensory representation of an object depends on the working range of the specific sensory modality, the relative reliability or distinctiveness of the encoded information and top-down predictions. Moreover, apart from sensory-driven influences on perception, the acquisition of featural information across modalities can affect semantic memory and, in turn, influence category decisions. In sum, we argue that both multisensory processes independently constrain the formation of object categories across the lifespan, possibly through early and late integration mechanisms, respectively, to allow us to efficiently achieve the everyday, but remarkable, ability of recognizing objects. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Crossmodal visual predictions elicit spatially specific early visual cortex activity but later than real visual stimuli

Previous studies have indicated that crossmodal visual predictions are instrumental in controlling early visual cortex activity. The exact time course and spatial precision of such crossmodal top-down influences on the visual cortex have been unknown. In the present study, participants were exposed to audiovisual combinations comprising one of two sounds and a Gabor patch either in the top left or in the bottom right visual field. Event-related potentials (ERPs) were recorded to these frequent crossmodal combinations (standards) as well as to trials in which the visual stimulus was omitted (omissions) or the visual and auditory stimuli were recombined (deviants). Standards and deviants elicited an ERP between 50 and 100 ms of opposite polarity known as the C1 effect commonly associated with retinotopic processing in early visual cortex. By contrast, a C1 effect was not observed in omission trials. Spatially specific omission and mismatch effects (deviants minus standards) started only later with a latency of 230 ms and 170 ms, respectively. These results suggest that crossmodal visual predictions control visual cortex activity in a spatially specific manner. However, visual predictions do not modulate visual cortex activity with the same timing as visual stimulation activates these areas but rather seem to involve distinct neural mechanisms. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Hybrid predictive coding: Inferring, fast and slow

Predictive coding is an influential model of cortical neural activity. It proposes that perceptual beliefs are furnished by sequentially minimising "prediction errors"-the differences between predicted and observed data. Implicit in this proposal is the idea that successful perception requires multiple cycles of neural activity. This is at odds with evidence that several aspects of visual perception-including complex forms of object recognition-arise from an initial "feedforward sweep" that occurs on fast timescales which preclude substantial recurrent activity. Here, we propose that the feedforward sweep can be understood as performing amortized inference (applying a learned function that maps directly from data to beliefs) and recurrent processing can be understood as performing iterative inference (sequentially updating neural activity in order to improve the accuracy of beliefs). We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner by describing both in terms of a dual optimization of a single objective function. We show that the resulting scheme can be implemented in a biologically plausible neural architecture that approximates Bayesian inference utilising local Hebbian update rules. We demonstrate that our hybrid predictive coding model combines the benefits of both amortized and iterative inference-obtaining rapid and computationally cheap perceptual inference for familiar data while maintaining the context-sensitivity, precision, and sample efficiency of iterative inference schemes. Moreover, we show how our model is inherently sensitive to its uncertainty and adaptively balances iterative and amortized inference to obtain accurate beliefs using minimum computational expense. Hybrid predictive coding offers a new perspective on the functional relevance of the feedforward and recurrent activity observed during visual perception and offers novel insights into distinct aspects of visual phenomenology.

Prior expectation enhances sensorimotor behavior by modulating population tuning and subspace activity in sensory cortex

Prior knowledge facilitates our perception and goal-directed behaviors, particularly when sensory input is lacking or noisy. However, the neural mechanisms underlying the improvement in sensorimotor behavior by prior expectations remain unknown. In this study, we examine the neural activity in the middle temporal (MT) area of visual cortex while monkeys perform a smooth pursuit eye movement task with prior expectation of the visual target's motion direction. Prior expectations discriminately reduce the MT neural responses depending on their preferred directions, when the sensory evidence is weak. This response reduction effectively sharpens neural population direction tuning. Simulations with a realistic MT population demonstrate that sharpening the tuning can explain the biases and variabilities in smooth pursuit, suggesting that neural computations in the sensory area alone can underpin the integration of prior knowledge and sensory evidence. State-space analysis further supports this by revealing neural signals of prior expectations in the MT population activity that correlate with behavioral changes.

Predictive neural representations of naturalistic dynamic input

Adaptive behavior such as social interaction requires our brain to predict unfolding external dynamics. While theories assume such dynamic prediction, empirical evidence is limited to static snapshots and indirect consequences of predictions. We present a dynamic extension to representational similarity analysis that uses temporally variable models to capture neural representations of unfolding events. We applied this approach to source-reconstructed magnetoencephalography (MEG) data of healthy human subjects and demonstrate both lagged and predictive neural representations of observed actions. Predictive representations exhibit a hierarchical pattern, such that high-level abstract stimulus features are predicted earlier in time, while low-level visual features are predicted closer in time to the actual sensory input. By quantifying the temporal forecast window of the brain, this approach allows investigating predictive processing of our dynamic world. It can be applied to other naturalistic stimuli (e.g., film, soundscapes, music, motor planning/execution, social interaction) and any biosignal with high temporal resolution.

Backward masking reveals coarse-to-fine dynamics in human V1

Natural images exhibit luminance variations aligned across a broad spectrum of spatial frequencies (SFs). It has been proposed that, at early stages of processing, the coarse signals carried by the low SF (LSF) of the visual input are sent rapidly from primary visual cortex (V1) to ventral, dorsal and frontal regions to form a coarse representation of the input, which is later sent back to V1 to guide the processing of fine-grained high SFs (i.e., HSF). We used functional resonance imaging (fMRI) to investigate the role of human V1 in the coarse-to-fine integration of visual input. We disrupted the processing of the coarse and fine content of full-spectrum human face stimuli via backward masking of selective SF ranges (LSFs: <1.75cpd and HSFs: >1.75cpd) at specific times (50, 83, 100 or 150ms). In line with coarse-to-fine proposals, we found that (1) the selective masking of stimulus LSF disrupted V1 activity in the earliest time window, and progressively decreased in influence, while (2) an opposite trend was observed for the masking of stimulus' HSF. This pattern of activity was found in V1, as well as in ventral (i.e. the Fusiform Face area, FFA), dorsal and orbitofrontal regions. We additionally presented subjects with contrast negated stimuli. While contrast negation significantly reduced response amplitudes in the FFA, as well as coupling between FFA and V1, coarse-to-fine dynamics were not affected by this manipulation. The fact that V1 response dynamics to strictly identical stimulus sets differed depending on the masked scale adds to growing evidence that V1 role goes beyond the early and quasi-passive transmission of visual information to the rest of the brain. It instead indicates that V1 may yield a 'spatially registered common forum' or 'blackboard' that integrates top-down inferences with incoming visual signals through its recurrent interaction with high-level regions located in the inferotemporal, dorsal and frontal regions.

Naturalistic viewing increases individual identifiability based on connectivity within functional brain networks

Naturalistic viewing (NV) is currently considered a promising paradigm for studying individual differences in functional brain organization. While whole brain functional connectivity (FC) under NV has been relatively well characterized, so far little work has been done on a network level. Here, we extend current knowledge by characterizing the influence of NV on FC in fourteen meta-analytically derived brain networks considering three different movie stimuli in comparison to resting-state (RS). We show that NV increases identifiability of individuals over RS based on functional connectivity in certain, but not all networks. Furthermore, movie stimuli including a narrative appear more distinct from RS. In addition, we assess individual variability in network FC by comparing within- and between-subject similarity during NV and RS. We show that NV can evoke individually distinct NFC patterns by increasing inter-subject variability while retaining within-subject similarity. Crucially, our results highlight that this effect is not observable across all networks, but rather dependent on the network-stimulus combination. Our results confirm that NV can improve the detection of individual differences over RS and underline the importance of selecting the appropriate combination of movie and cognitive network for the research question at hand.

Statistical Learning of Distractor Suppression Downregulates Prestimulus Neural Excitability in Early Visual Cortex

Visual attention is highly influenced by past experiences. Recent behavioral research has shown that expectations about the spatial location of distractors within a search array are implicitly learned, with expected distractors becoming less interfering. Little is known about the neural mechanism supporting this form of statistical learning. Here, we used magnetoencephalography (MEG) to measure human brain activity to test whether proactive mechanisms are involved in the statistical learning of distractor locations. Specifically, we used a new technique called rapid invisible frequency tagging (RIFT) to assess neural excitability in early visual cortex during statistical learning of distractor suppression while concurrently investigating the modulation of posterior alpha band activity (8-12 Hz). Male and female human participants performed a visual search task in which a target was occasionally presented alongside a color-singleton distractor. Unbeknown to the participants, the distracting stimuli were presented with different probabilities across the two hemifields. RIFT analysis showed that early visual cortex exhibited reduced neural excitability in the prestimulus interval at retinotopic locations associated with higher distractor probabilities. In contrast, we did not find any evidence of expectation-driven distractor suppression in alpha band activity. These findings indicate that proactive mechanisms of attention are involved in predictive distractor suppression and that these mechanisms are associated with altered neural excitability in early visual cortex. Moreover, our findings indicate that RIFT and alpha band activity might subtend different and possibly independent attentional mechanisms.SIGNIFICANCE STATEMENT What we experienced in the past affects how we perceive the external world in the future. For example, an annoying flashing light might be better ignored if we know in advance where it usually appears. This ability of extracting regularities from the environment is called statistical learning. In this study, we explore the neuronal mechanisms allowing the attentional system to overlook items that are unequivocally distracting based on their spatial distribution. By recording brain activity using MEG while probing neural excitability with a novel technique called RIFT, we show that the neuronal excitability in early visual cortex is reduced in advance of stimulus presentation for locations where distracting items are more likely to occur.

Seeing and extrapolating motion trajectories share common informative activation patterns in primary visual cortex

The natural environment is dynamic and moving objects become constantly occluded, engaging the brain in a challenging completion process to estimate where and when the object might reappear. Although motion extrapolation is critical in daily life-imagine crossing the street while an approaching car is occluded by a larger standing vehicle-its neural underpinnings are still not well understood. While the engagement of low-level visual cortex during dynamic occlusion has been postulated, most of the previous group-level fMRI-studies failed to find evidence for an involvement of low-level visual areas during occlusion. In this fMRI-study, we therefore used individually defined retinotopic maps and multivariate pattern analysis to characterize the neural basis of visible and occluded changes in motion direction in humans. To this end, participants learned velocity-direction change pairings (slow motion-upwards; fast motion-downwards or vice versa) during a training phase without occlusion and judged the change in stimulus direction, based on its velocity, during a following test phase with occlusion. We find that occluded motion direction can be predicted from the activity patterns during visible motion within low-level visual areas, supporting the notion of a mental representation of motion trajectory in these regions during occlusion.

Bayesian surprise shapes neural responses in somatosensory cortical circuits

Numerous psychophysical studies show that Bayesian inference governs sensory decision-making; however, the specific neural circuitry underlying this probabilistic mechanism remains unclear. We record extracellular neural activity along the somatosensory pathway of mice while delivering sensory stimulation paradigms designed to isolate the response to the surprise generated by Bayesian inference. Our results demonstrate that laminar cortical circuits in early sensory areas encode Bayesian surprise. Systematic sensitivity to surprise is not identified in the somatosensory thalamus, rather emerging in the primary (S1) and secondary (S2) somatosensory cortices. Multiunit spiking activity and evoked potentials in layer 6 of these regions exhibit the highest sensitivity to surprise. Gamma power in S1 layer 2/3 exhibits an NMDAR-dependent scaling with surprise, as does alpha power in layers 2/3 and 6 of S2. These results show a precise spatiotemporal neural representation of Bayesian surprise and suggest that Bayesian inference is a fundamental component of cortical processing.

Multimodal processing in face-to-face interactions: A bridging link between psycholinguistics and sensory neuroscience

In face-to-face communication, humans are faced with multiple layers of discontinuous multimodal signals, such as head, face, hand gestures, speech and non-speech sounds, which need to be interpreted as coherent and unified communicative actions. This implies a fundamental computational challenge: optimally binding only signals belonging to the same communicative action while segregating signals that are not connected by the communicative content. How do we achieve such an extraordinary feat, reliably, and efficiently? To address this question, we need to further move the study of human communication beyond speech-centred perspectives and promote a multimodal approach combined with interdisciplinary cooperation. Accordingly, we seek to reconcile two explanatory frameworks recently proposed in psycholinguistics and sensory neuroscience into a neurocognitive model of multimodal face-to-face communication. First, we introduce a psycholinguistic framework that characterises face-to-face communication at three parallel processing levels: multiplex signals, multimodal gestalts and multilevel predictions. Second, we consider the recent proposal of a lateral neural visual pathway specifically dedicated to the dynamic aspects of social perception and reconceive it from a multimodal perspective ("lateral processing pathway"). Third, we reconcile the two frameworks into a neurocognitive model that proposes how multiplex signals, multimodal gestalts, and multilevel predictions may be implemented along the lateral processing pathway. Finally, we advocate a multimodal and multidisciplinary research approach, combining state-of-the-art imaging techniques, computational modelling and artificial intelligence for future empirical testing of our model.

Multivariate functional neuroimaging analyses reveal that strength-dependent face expectations are represented in higher-level face-identity areas

Perception is an active inference in which prior expectations are combined with sensory input. It is still unclear how the strength of prior expectations is represented in the human brain. The strength, or precision, of a prior could be represented with its content, potentially in higher-level sensory areas. We used multivariate analyses of functional resonance imaging data to test whether expectation strength is represented together with the expected face in high-level face-sensitive regions. Participants were trained to associate images of scenes with subsequently presented images of different faces. Each scene predicted three faces, each with either low, intermediate, or high probability. We found that anticipation enhances the similarity of response patterns in the face-sensitive anterior temporal lobe to response patterns specifically associated with the image of the expected face. In contrast, during face presentation, activity increased for unexpected faces in a typical prediction error network, containing areas such as the caudate and the insula. Our findings show that strength-dependent face expectations are represented in higher-level face-identity areas, supporting hierarchical theories of predictive processing according to which higher-level sensory regions represent weighted priors.

Perceptual Functioning

Perceptual disorders are not part of the diagnosis criteria for schizophrenia. Yet, a considerable amount of work has been conducted, especially on visual perception abnormalities, and there is little doubt that visual perception is altered in patients. There are several reasons why such perturbations are of interest in this pathology. They are observed during the prodromal phase of psychosis, they are related to the pathophysiology (clinical disorganization, disorders of the sense of self), and they are associated with neuronal connectivity disorders. Perturbations occur at different levels of processing and likely affect how patients interact and adapt to their surroundings. The literature has become very large, and here we try to summarize different models that have guided the exploration of perception in patients. We also illustrate several lines of research by showing how perception has been investigated and by discussing the interpretation of the results. In addition to discussing domains such as contrast sensitivity, masking, and visual grouping, we develop more recent fields like processing at the level of the retina, and the timing of perception.

BOLD activation on the groundside of figures: More suppression of grounds that competed more for figural status

A fundamental aspect of object detection is assigning a border to one (figure) side but not the other (ground) side. Figures are shaped; grounds appear shapeless near the figure border. Accumulating evidence supports the view that the mechanism of figure assignment is inhibitory competition with the figure perceived on the winning side. Suppression has been observed on the groundside of figure borders. One prediction is that more suppression will be observed when the groundside competes more for figural status. We tested this prediction by assessing BOLD activation on the groundside of two types of stimuli with articulated borders: AE_nov and AE_fam stimuli. In both stimulus types, multiple image-based priors (symmetry, closure, small area, enclosure by a larger region) favored the inside as the figure. In AE_fam but not AE_nov stimuli, the figural prior of familiar configuration present on the outside competes for figural status. Observers perceived the insides of both types of stimuli as novel figures and the outsides as shapeless grounds. Previously, we observed lower BOLD activation in early visual areas representing the grounds of AE_fam than AE_nov stimuli, although unexpectedly, activation was above baseline. With articulated borders, it can be difficult to exclude figure activation from ground ROIs. Here, our ground ROIs better excluded figure activation; we also added straight-edge (SE) control stimuli and increased the sample size. In early visual areas representing the grounds, we observed lower BOLD activation on the groundside of AE_fam than AE_nov stimuli and below-baseline BOLD activation on the groundside of SE and AE_fam stimuli. These results, indicating that greater suppression is applied to groundsides that competed more for figural status but lost the competition, support a Bayesian model of figure assignment in which proto-objects activated at both low and high levels where image features and familiar configurations are represented, respectively, compete for figural status.

Strong perceptual consequences of low-level visual predictions: A new illusion

Predicting information is considered to be an efficient strategy to minimise processing costs by exploiting regularities in the environment, and to allow for adaptation in case of irregularities, i.e. prediction errors. How such errors impact conscious perception is unclear, especially when predictions concern elementary visual features. Here we present results from a novel experimental approach allowing us to investigate the perceptual consequences of violated low-level predictions about moving objects. Observers were presented with two squares moving towards each other with a constant speed, and reported whether they were in contact or not before they disappeared. A compelling illusion of a gap between the squares occurred when the leading edges of those squares contacted briefly. The apparent gap was larger than a physical and stable separation of 2.6 min of arc between the squares. The illusion disappeared only when the contact did not violate extrapolations of the contrast edge between the moving object and the background. The pattern of results is consistent with an early locus of the effect and cannot be explained by decisional biases, guesses, top-down, attentional or masking effects. We suggest that violations of the contrast edge extrapolation in the direction of motion have strong perceptual consequences.

How universal is preference for visual curvature? A systematic review and meta-analysis

Evidence dating back a century shows that humans are sensitive to and exhibit a preference for visual curvature. This effect has been observed in different age groups, human cultures, and primate species, suggesting that a preference for curvature could be universal. At the same time, several studies have found that preference for curvature is modulated by contextual and individual factors, casting doubt on this hypothesis. To resolve these conflicting findings, we conducted a systematic meta-analysis of studies that have investigated the preference for visual curvature. Our meta-analysis included 61 studies which provided 106 independent samples and 309 effect sizes. The results of a three-level random effects model revealed a Hedges' g of 0.39-consistent with a medium effect size. Further analyses revealed that preference for curvature is moderated by four factors: presentation time, stimulus type, expertise, and task. Together, our results suggest that preference for visual curvature is a reliable but not universal phenomenon and is influenced by factors other than perceptual information.

Predictive coding is a consequence of energy efficiency in recurrent neural networks

Predictive coding is a promising framework for understanding brain function. It postulates that the brain continuously inhibits predictable sensory input, ensuring preferential processing of surprising elements. A central aspect of this view is its hierarchical connectivity, involving recurrent message passing between excitatory bottom-up signals and inhibitory top-down feedback. Here we use computational modeling to demonstrate that such architectural hardwiring is not necessary. Rather, predictive coding is shown to emerge as a consequence of energy efficiency. When training recurrent neural networks to minimize their energy consumption while operating in predictive environments, the networks self-organize into prediction and error units with appropriate inhibitory and excitatory interconnections and learn to inhibit predictable sensory input. Moving beyond the view of purely top-down-driven predictions, we demonstrate, via virtual lesioning experiments, that networks perform predictions on two timescales: fast lateral predictions among sensory units and slower prediction cycles that integrate evidence over time.

Updating Contextual Sensory Expectations for Adaptive Behavior

The brain has the extraordinary capacity to construct predictive models of the environment by internalizing statistical regularities in the sensory inputs. The resulting sensory expectations shape how we perceive and react to the world; at the neural level, this relates to decreased neural responses to expected than unexpected stimuli ("expectation suppression"). Crucially, expectations may need revision as context changes. However, existing research has often neglected this issue. Further, it is unclear whether contextual revisions apply selectively to expectations relevant to the task at hand, hence serving adaptive behavior. The present fMRI study examined how contextual visual expectations spread throughout the cortical hierarchy as we update our beliefs. We created a volatile environment: two alternating contexts contained different sequences of object images, thereby producing context-dependent expectations that needed revision when the context changed. Human participants of both sexes attended a training session before scanning to learn the contextual sequences. The fMRI experiment then tested for the emergence of contextual expectation suppression in two separate tasks, respectively, with task-relevant and task-irrelevant expectations. Effects of contextual expectation emerged progressively across the cortical hierarchy as participants attuned themselves to the context: expectation suppression appeared first in the insula, inferior frontal gyrus, and posterior parietal cortex, followed by the ventral visual stream, up to early visual cortex. This applied selectively to task-relevant expectations. Together, the present results suggest that an insular and frontoparietal executive control network may guide the flexible deployment of contextual sensory expectations for adaptive behavior in our complex and dynamic world.SIGNIFICANCE STATEMENT The world is structured by statistical regularities, which we use to predict the future. This is often accompanied by suppressed neural responses to expected compared with unexpected events ("expectation suppression"). Crucially, the world is also highly volatile and context-dependent: expected events may become unexpected when the context changes, thus raising the crucial need for belief updating. However, this issue has generally been neglected. By setting up a volatile environment, we show that expectation suppression emerges first in executive control regions, followed by relevant sensory areas, only when observers use their expectations to optimize behavior. This provides surprising yet clear evidence on how the brain controls the updating of sensory expectations for adaptive behavior in our ever-changing world.

Bounded rational decision-making models suggest capacity-limited concurrent motor planning in human posterior parietal and frontal cortex

While traditional theories of sensorimotor processing have often assumed a serial decision-making pipeline, more recent approaches have suggested that multiple actions may be planned concurrently and vie for execution. Evidence for the latter almost exclusively stems from electrophysiological studies in posterior parietal and premotor cortex of monkeys. Here we study concurrent prospective motor planning in humans by recording functional magnetic resonance imaging (fMRI) during a delayed response task engaging movement sequences towards multiple potential targets. We find that also in human posterior parietal and premotor cortex delay activity modulates both with sequence complexity and the number of potential targets. We tested the hypothesis that this modulation is best explained by concurrent prospective planning as opposed to the mere maintenance of potential targets in memory. We devise a bounded rationality model with information constraints that optimally assigns information resources for planning and memory for this task and determine predicted information profiles according to the two hypotheses. When regressing delay activity on these model predictions, we find that the concurrent prospective planning strategy provides a significantly better explanation of the fMRI-signal modulations. Moreover, we find that concurrent prospective planning is more costly and thus limited for most subjects, as expressed by the best fitting information capacities. We conclude that bounded rational decision-making models allow relating both behavior and neural representations to utilitarian task descriptions based on bounded optimal information-processing assumptions.

When the future is uncertain, it can be beneficial to concurrently plan several action possibilities in advance. Electrophysiological research found evidence in monkeys that brain regions in posterior parietal and promotor cortex are indeed capable of planning several actions in parallel. We now used fMRI to study brain activity in these brain regions in humans. For our analyses we applied bounded rationality models that optimally assign information resources to fMRI activity in a complex motor planning task. We find that theoretical information costs of concurrent prospective planning explained fMRI activity profiles significantly better than assuming alternative memory-based strategies. Moreover, exploiting the model allowed us to quantify the individual capacity limit for concurrent planning and to relate these individual limits to both subjects’ behavior and to their neural representations of planning.

A quantitative model reveals a frequency ordering of prediction and prediction-error signals in the human brain

The human brain is proposed to harbor a hierarchical predictive coding neuronal network underlying perception, cognition, and action. In support of this theory, feedforward signals for prediction error have been reported. However, the identification of feedback prediction signals has been elusive due to their causal entanglement with prediction-error signals. Here, we use a quantitative model to decompose these signals in electroencephalography during an auditory task, and identify their spatio-spectral-temporal signatures across two functional hierarchies. Two prediction signals are identified in the period prior to the sensory input: a low-level signal representing the tone-to-tone transition in the high beta frequency band, and a high-level signal for the multi-tone sequence structure in the low beta band. Subsequently, prediction-error signals dependent on the prior predictions are found in the gamma band. Our findings reveal a frequency ordering of prediction signals and their hierarchical interactions with prediction-error signals supporting predictive coding theory.

A computational framework can extract spatio-spectro-temporal neural signatures corresponding to hierarchical prediction and prediction errors in a local-global auditory task.

Action-based predictions affect visual perception, neural processing, and pupil size, regardless of temporal predictability

Sensory consequences of one's own action are often perceived as less intense, and lead to reduced neural responses, compared to externally generated stimuli. Presumably, such sensory attenuation is due to predictive mechanisms based on the motor command (efference copy). However, sensory attenuation has also been observed outside the context of voluntary action, namely when stimuli are temporally predictable. Here, we aimed at disentangling the effects of motor and temporal predictability-based mechanisms on the attenuation of sensory action consequences. During fMRI data acquisition, participants (N = 25) judged which of two visual stimuli was brighter. In predictable blocks, the stimuli appeared temporally aligned with their button press (active) or aligned with an automatically generated cue (passive). In unpredictable blocks, stimuli were presented with a variable delay after button press/cue, respectively. Eye tracking was performed to investigate pupil-size changes and to ensure proper fixation. Self-generated stimuli were perceived as darker and led to less neural activation in visual areas than their passive counterparts, indicating sensory attenuation for self-generated stimuli independent of temporal predictability. Pupil size was larger during self-generated stimuli, which correlated negatively with the blood oxygenation level dependent (BOLD) response: the larger the pupil, the smaller the BOLD amplitude in visual areas. Our results suggest that sensory attenuation in visual cortex is driven by action-based predictive mechanisms rather than by temporal predictability. This effect may be related to changes in pupil diameter. Altogether, these results emphasize the role of the efference copy in the processing of sensory action consequences.

Dampened sensory representations for expected input across the ventral visual stream

Expectations, derived from previous experience, can help in making perception faster, more reliable and informative. A key neural signature of perceptual expectations is expectation suppression, an attenuated neural response to expected compared with unexpected stimuli. While expectation suppression has been reported using a variety of paradigms and recording methods, it remains unclear what neural modulation underlies this response attenuation. Sharpening models propose that neural populations tuned away from an expected stimulus are particularly suppressed by expectations, thereby resulting in an attenuated, but sharper population response. In contrast, dampening models suggest that neural populations tuned toward the expected stimulus are most suppressed, thus resulting in a dampened, less redundant population response. Empirical support is divided, with some studies favoring sharpening, while others support dampening. A key limitation of previous neuroimaging studies is the ability to draw inferences about neural-level modulations based on population (e.g. voxel) level signals. Indeed, recent simulations of repetition suppression showed that opposite neural modulations can lead to comparable population-level modulations. Forward models provide one solution to this inference limitation. Here, we used forward models to implement sharpening and dampening models, mapping neural modulations to voxel-level data. We show that a feature-specific gain modulation, suppressing neurons tuned toward the expected stimulus, best explains the empirical fMRI data. Thus, our results support the dampening account of expectation suppression, suggesting that expectations reduce redundancy in sensory cortex, and thereby promote updating of internal models on the basis of surprising information.

Adaptive Cognitive Control in Prematurely Born Children: An HD-EEG Investigation

Preterm birth is a neurodevelopmental risk condition often associated with cognitive control (CC) impairment. Recent evidence showed that CC can be implicitly adapted through associative learning. In the present study we investigated the ability to flexibly adjust CC as a function of implicit stimulus-response temporal regularities in preterm (PT; N = 21; mean age 8 ± 1.3 years; gestational age 30 ± 18.5 weeks) and full-term (FT; N = 20; mean age 8 ± 1.3 years) school-age children. All children underwent an HD-EEG recording while undergoing the Dynamic Temporal Prediction (DTP) task, a simple S1–S2 detection task purposely designed to generate local-global temporal predictability of imperative stimuli. The Wisconsin card sorting test (WCST) was administered to measure explicit CC. The PT group showed more premature and slower (DTP) as well as perseverative (WCST) responses than the FT group. Moreover, pre-terms showed poor adaptive CC as revealed by less efficient global response-speed adjustment. This behavioral pattern was mirrored by a reduced and less sensitive to global manipulation anticipatory Contingent Negative Variation (CNV) and by different cortical source recruitment. These findings suggest that implicit CC may be a reliable endophenotypic marker of atypical cognitive development associated with preterm birth.

Contextual Expectations Shape Cortical Reinstatement of Sensory Representations

When making a turn at a familiar intersection, we know what items and landmarks will come into view. These perceptual expectations, or predictions, come from our knowledge of the context; however, it is unclear how memory and perceptual systems interact to support the prediction and reactivation of sensory details in cortex. To address this, human participants learned the spatial layout of animals positioned in a cross maze. During fMRI, participants of both sexes navigated between animals to reach a target, and in the process saw a predictable sequence of five animal images. Critically, to isolate activity patterns related to item predictions, rather than bottom-up inputs, one-fourth of trials ended early, with a blank screen presented instead. Using multivariate pattern similarity analysis, we reveal that activity patterns in early visual cortex, posterior medial regions, and the posterior hippocampus showed greater similarity when seeing the same item compared with different items. Further, item effects in posterior hippocampus were specific to the sequence context. Critically, activity patterns associated with seeing an item in visual cortex and posterior medial cortex, were also related to activity patterns when an item was expected, but omitted, suggesting sequence predictions were reinstated in these regions. Finally, multivariate connectivity showed that patterns in the posterior hippocampus at one position in the sequence were related to patterns in early visual cortex and posterior medial cortex at a later position. Together, our results support the idea that hippocampal representations facilitate sensory processing by modulating visual cortical activity in anticipation of expected items.SIGNIFICANCE STATEMENT Our visual world is a series of connected events, where we can predict what we might see next based on our recent past. Understanding the neural circuitry and mechanisms of the perceptual and memory systems that support these expectations is fundamental to revealing how we perceive and act in our world. Using brain imaging, we studied what happens when we expect to see specific visual items, and how such expectations relate to top-down memory signals. We find both visual and memory systems reflect item predictions, and moreover, we show that hippocampal activity supports predictions of future expected items. This demonstrates that the hippocampus acts to predict upcoming items, and reinstates such predictions in cortex.

A link between age, affect, and predictions?

The prevalence of depressive symptoms decreases from late adolescence to middle age adulthood. Furthermore, despite significant losses in motor and cognitive functioning, overall emotional well-being tends to increase with age, and a bias to positive information has been observed multiple times. Several causes have been discussed for this age-related development, such as improvement in emotion regulation, less regret, and higher socioeconomic status. Here, we explore a further explanation. Our minds host mental models that generate predictions about forthcoming events to successfully interact with our physical and social environment. To keep these models faithful, the difference between the predicted and the actual event, that is, the prediction error, is computed. We argue that prediction errors are attenuated in the middle age and older mind, which, in turn, may translate to less negative affect, lower susceptibility to affective disorders, and possibly, to a bias to positive information. Our proposal is primarily linked to perceptual inferences, but may hold as well for higher-level, cognitive, and emotional forms of error processing.

Hippocampal Contribution to Probabilistic Feedback Learning: Modeling Observation- and Reinforcement-based Processes

Simple probabilistic reinforcement learning is recognized as a striatum-based learning system, but in recent years, has also been associated with hippocampal involvement. This study examined whether such involvement may be attributed to observation-based learning (OL) processes, running in parallel to striatum-based reinforcement learning. A computational model of OL, mirroring classic models of reinforcement-based learning (RL), was constructed and applied to the neuroimaging data set of Palombo, Hayes, Reid, and Verfaellie (2019). Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia. Cognitive, Affective & Behavioral Neuroscience, 19(3), 523-536. Results suggested that OL processes may indeed take place concomitantly to reinforcement learning and involve activation of the hippocampus and central orbitofrontal cortex. However, rather than independent mechanisms running in parallel, the brain correlates of the OL and RL prediction errors indicated collaboration between systems, with direct implication of the hippocampus in computations of the discrepancy between the expected and actual reinforcing values of actions. These findings are consistent with previous accounts of a role for the hippocampus in encoding the strength of observed stimulus-outcome associations, with updating of such associations through striatal reinforcement-based computations. In addition, enhanced negative RL prediction error signaling was found in the anterior insula with greater use of OL over RL processes. This result may suggest an additional mode of collaboration between the OL and RL systems, implicating the error monitoring network.

From the inside out: Interoceptive feedback facilitates the integration of visceral signals for efficient sensory processing

Neuroscientific studies have mainly focused on the way humans perceive and interact with the external world. Recent work in the interoceptive domain indicates that the brain predictively models information from inside the body such as the heartbeat and that the efficiency with which this is executed can have implications for exteroceptive processing. However, to date direct evidence underpinning these hypotheses is lacking. Here, we show how the brain predictively refines neural resources to process afferent cardiac feedback and uses these interoceptive cues to enable more efficient processing of external sensory information. Participants completed a repetition-suppression paradigm consisting of a neutral repeating face. During the first face presentation, they heard auditory feedback of their heartbeat which either coincided with the systole of the cardiac cycle, the time at which cardiac events are registered by the brain or the diastole during which the brain receives no internal cardiac feedback. We used electroencephalography to measure the heartbeat evoked potential (HEP) as well as auditory (AEP) and visual evoked potentials (VEP). Exteroceptive cardiac feedback which coincided with the systole produced significantly higher HEP amplitudes relative to feedback timed to the diastole. Elevation of the HEP in this condition was followed by significant suppression of the VEP in response to the repeated neutral face and a stepwise decrease of AEP amplitude to repeated heartbeat feedback. Our results hereby show that exteroceptive heartbeat feedback coinciding with interoceptive signals at systole enhanced interoceptive cardiac processing. Furthermore, the same cue facilitating interoceptive integration enabled efficient suppression of a visual stimulus, as well as repetition suppression of the AEP across successive auditory heartbeat feedback. Our findings provide evidence that the alignment of external to internal signals can enhance the efficiency of interoceptive processing and that cues facilitating this process in either domain have beneficial effects for internal as well as external sensory processing.

Spatial and Temporal Context Jointly Modulate the Sensory Response within the Ventral Visual Stream

Both spatial and temporal context play an important role in visual perception and behavior. Humans can extract statistical regularities from both forms of context to help process the present and to construct expectations about the future. Numerous studies have found reduced neural responses to expected stimuli compared with unexpected stimuli, for both spatial and temporal regularities. However, it is largely unclear whether and how these forms of context interact. In the current fMRI study, 33 human volunteers were exposed to pairs of object stimuli that could be expected or surprising in terms of their spatial and temporal context. We found reliable independent contributions of both spatial and temporal context in modulating the neural response. Specifically, neural responses to stimuli in expected compared with unexpected contexts were suppressed throughout the ventral visual stream. These results suggest that both spatial and temporal context may aid sensory processing in a similar fashion, providing evidence on how different types of context jointly modulate perceptual processing.

Predictive Coding, Variational Autoencoders, and Biological Connections

We present a review of predictive coding, from theoretical neuroscience, and variational autoencoders, from machine learning, identifying the common origin and mathematical framework underlying both areas. As each area is prominent within its respective field, more firmly connecting these areas could prove useful in the dialogue between neuroscience and machine learning. After reviewing each area, we discuss two possible correspondences implied by this perspective: cortical pyramidal dendrites as analogous to (nonlinear) deep networks and lateral inhibition as analogous to normalizing flows. These connections may provide new directions for further investigations in each field.

Special treatment of prediction errors in autism spectrum disorder

For autistic individuals, sensory stimulation can be experienced as overwhelming. Models of predictive coding postulate that cortical mechanisms disamplify predictable information and amplify prediction errors that surpass a defined precision level. In autism, the neuronal processing is putting an inflexibly high precision on prediction errors according to the HIPPEA theory (High, Inflexible Precision of Prediction Errors in Autism). We used an apparent motion paradigm to test this prediction. In apparent motion paradigms, the illusory motion of an object creates a prediction about where and when an internally generated token would be moving along the apparent motion trace. This illusion facilitates the perception of a flashing stimulus (target) appearing in-time with the apparent motion token and is perceived as a predictable event (predictable target). In contrast, a flashing stimulus appearing out-of-time with the apparent motion illusion is an unpredictable target that is less often detected even though it produces a prediction error signal. If a prediction error does not surpass a given precision threshold the stimulation event is discounted and therefore less often detected than predictable tokens. In autism, the precision threshold is lower and the same prediction errors (unpredictable target) triggers a detection similar to that of a predictable flash stimulus. To test this hypothesis, we recruited 11 autistic males and 9 neurotypical matched controls. The participants were tasked to detect flashing stimuli placed on an apparent motion trace either in-time or out-of-time with the apparent motion illusion. Descriptively, 66% (6/9) of neurotypical and 64% (7/11) of autistic participants were better at detecting predictable targets. The prediction established by illusory motion appears to assist autistic and neurotypical individuals equally in the detection of predictable over unpredictable targets. Importantly, 55% (6/11) of autistic participants had faster responses for unpredictable targets, whereas only 22% (2/9) of neurotypicals had faster responses to unpredictable compared to predictable targets. Hence, these tentative results suggest that for autistic participants, unpredictable targets produce an above threshold prediction error, which leads to faster response. This difference in unpredictable target detection can be encapsulated under the HIPPEA theory, suggesting that precision setting could be aberrant in autistic individuals with respect to prediction errors. These tentative results should be considered in light of the small sample. For this reason, we provide the full set of materials necessary to replicate and extend the results.

Prediction, Suppression of Visual Response, and Modulation of Visual Perception: Insights From Visual Evoked Potentials and Representational Momentum

When a visual object changes its position along with certain sequential regularities, the visual system rapidly and automatically forms a prediction regarding the future position of the object based on the regularities. Such prediction can drastically alter visual perception. A phenomenon called representational momentum (RM: a predictive displacement of the perceived final position of a visual object along its recent regular pattern) has provided extensive evidence for the predictive modulation of visual perception. The purpose of the present study was to identify neural effects that could explain individual differences in the strength of the predictive modulation of visual perception as measured by RM. For this purpose, in two experiments with a conventional RM paradigm where a bar was discretely presented in a regular rotation manner (with a step of 18° in Experiment 1 and a step of 20° in Experiment 2), visual evoked potentials (VEPs) in response to the regularly rotated bar were measured, and correlations between the magnitudes of RM and VEPs were examined. The results showed that the magnitudes of RM and central P2 were negatively correlated, consistently in both experiments; participants who showed a smaller central P2 tended to exhibit greater RM. Together with a previous proposal that central P2 would represent delayed reactivation of lower visual areas around the striate and prestriate cortices via reentrant feedback projections from higher areas, the present results suggest that greater suppression of delayed reactivation of lower visual areas (as indicated by smaller central P2) may underlie stronger predictive modulation of visual perception (as indicated by greater RM).

Predictive processing models and affective neuroscience

The neural bases of affective experience remain elusive. Early neuroscience models of affect searched for specific brain regions that uniquely carried out the computations that underlie dimensions of valence and arousal. However, a growing body of work has failed to identify these circuits. Research turned to multivariate analyses, but these strategies, too, have made limited progress. Predictive processing models offer exciting new directions to address this problem. Here, we use predictive processing models as a lens to critique prevailing functional neuroimaging research practices in affective neuroscience. Our review highlights how much work relies on rigid assumptions that are inconsistent with a predictive processing approach. We outline the central aspects of a predictive processing model and draw out their implications for research in affective and cognitive neuroscience. Predictive models motivate a reformulation of "reverse inference" in cognitive neuroscience, and placing a greater emphasis on external validity in experimental design.

Resolving visual motion through perceptual gaps

Perceptual gaps can be caused by objects in the foreground temporarily occluding objects in the background or by eyeblinks, which briefly but frequently interrupt visual information. Resolving visual motion across perceptual gaps is particularly challenging, as object position changes during the gap. We examine how visual motion is maintained and updated through externally driven (occlusion) and internally driven (eyeblinks) perceptual gaps. Focusing on both phenomenology and potential mechanisms such as suppression, extrapolation, and integration, we present a framework for how perceptual gaps are resolved over space and time. We finish by highlighting critical questions and directions for future work.