Repetition probability effects depend on prior experiences

Adaptation in the visual system: Networked fatigue or suppressed prediction error signalling?

Our brains are constantly adapting to changes in our visual environments. Neural adaptation exerts a persistent influence on the activity of sensory neurons and our perceptual experience, however there is a lack of consensus regarding how adaptation is implemented in the visual system. One account describes fatigue-based mechanisms embedded within local networks of stimulus-selective neurons (networked fatigue models). Another depicts adaptation as a product of stimulus expectations (predictive coding models). In this review, I evaluate neuroimaging and psychophysical evidence that poses fundamental problems for predictive coding models of neural adaptation. Specifically, I discuss observations of distinct repetition and expectation effects, as well as incorrect predictions of repulsive adaptation aftereffects made by predictive coding accounts. Based on this evidence, I argue that networked fatigue models provide a more parsimonious account of adaptation effects in the visual system. Although stimulus expectations can be formed based on recent stimulation history, any consequences of these expectations are likely to co-occur (or interact) with effects of fatigue-based adaptation. I conclude by proposing novel, testable hypotheses relating to interactions between fatigue-based adaptation and other predictive processes, focusing on stimulus feature extrapolation phenomena.

Recent visual experience reshapes V4 neuronal activity and improves perceptual performance

Recent visual experience heavily influences our visual perception, but how this is mediated by the reshaping of neuronal activity to alter and improve perceptual discrimination remains unknown. We recorded from populations of neurons in visual cortical area V4 while monkeys performed a natural image change detection task under different experience conditions. We found that maximizing the recent experience with a particular image led to an improvement in the ability to detect a change in that image. This improvement was associated with decreased neural responses to the image, consistent with neuronal changes previously seen in studies of adaptation and expectation. We found that the magnitude of behavioral improvement was correlated with the magnitude of response suppression. Furthermore, this suppression of activity led to an increase in signal separation, providing evidence that a reduction in activity can improve stimulus encoding. Within populations of neurons, greater recent experience was associated with decreased trial-to-trial shared variability, indicating that a reduction in variability is a key means by which experience influences perception. Taken together, the results of our study contribute to an understanding of how recent visual experience can shape our perception and behavior through modulating activity patterns in mid-level visual cortex.

Expectation Modulates Repetition Suppression at Late But Not Early Stages during Visual Word Recognition: Evidence from Event-related Potentials

Visual word recognition is commonly rapid and efficient, incorporating top-down predictive processing mechanisms. Neuroimaging studies with face stimuli suggest that repetition suppression (RS) reflects predictive processing at the neural level, as this effect is larger when repetitions are more frequent, that is, more expected. It remains unclear, however, at the temporal level whether and how RS and its modulation by expectation occur in visual word recognition. To address this gap, the present study aimed to investigate the presence and time course of these effects during visual word recognition using EEG. Thirty-six native Cantonese speakers were presented with pairs of Chinese written words and performed a nonlinguistic oddball task. The second word of a pair was either a repetition of the first or a different word (alternation). In repetition blocks, 75% of trials were repetitions and 25% were alternations, whereas the reverse was true in alternation blocks. Topographic analysis of variance of EEG at each time point showed robust RS effects in three time windows (141-227 msec, 242-445 msec, and 467-513 msec) reflecting facilitation of visual word recognition. Importantly, the modulation of RS by expectation was observed at the late rather than early intervals (334-387 msec, 465-550 msec, and 559-632 msec) and more than 100 msec after the first RS effects. In the predictive coding view of RS, only late repetition effects are modulated by expectation, whereas early RS effects may be mediated by lower-level predictions. Taken together, our findings provide the first EEG evidence revealing distinct temporal dynamics of RS effects and repetition probability on RS effects in visual processing of Chinese words.

Effects of expectation on face perception and its association with expertise

Perceptual decisions are derived from the combination of priors and sensorial input. While priors are broadly understood to reflect experience/expertise developed over one's lifetime, the role of perceptual expertise at the individual level has seldom been directly explored. Here, we manipulate probabilistic information associated with a high and low expertise category (faces and cars respectively), while assessing individual level of expertise with each category. 67 participants learned the probabilistic association between a color cue and each target category (face/car) in a behavioural categorization task. Neural activity (EEG) was then recorded in a similar paradigm in the same participants featuring the previously learned contingencies without the explicit task. Behaviourally, perception of the higher expertise category (faces) was modulated by expectation. Specifically, we observed facilitatory and interference effects when targets were correctly or incorrectly expected, which were also associated with independently measured individual levels of face expertise. Multivariate pattern analysis of the EEG signal revealed clear effects of expectation from 100 ms post stimulus, with significant decoding of the neural response to expected vs. not stimuli, when viewing identical images. Latency of peak decoding when participants saw faces was directly associated with individual level facilitation effects in the behavioural task. The current results not only provide time sensitive evidence of expectation effects on early perception but highlight the role of higher-level expertise on forming priors.

The empirical status of predictive coding and active inference

Research on predictive processing models has focused largely on two specific algorithmic theories: Predictive Coding for perception and Active Inference for decision-making. While these interconnected theories possess broad explanatory potential, they have only recently begun to receive direct empirical evaluation. Here, we review recent studies of Predictive Coding and Active Inference with a focus on evaluating the degree to which they are empirically supported. For Predictive Coding, we find that existing empirical evidence offers modest support. However, some positive results can also be explained by alternative feedforward (e.g., feature detection-based) models. For Active Inference, most empirical studies have focused on fitting these models to behavior as a means of identifying and explaining individual or group differences. While Active Inference models tend to explain behavioral data reasonably well, there has not been a focus on testing empirical validity of active inference theory per se, which would require formal comparison to other models (e.g., non-Bayesian or model-free reinforcement learning models). This review suggests that, while promising, a number of specific research directions are still necessary to evaluate the empirical adequacy and explanatory power of these algorithms.

Bridging the gap between EEG and DCNNs reveals a fatigue mechanism of facial repetition suppression

Facial repetition suppression, a well-studied phenomenon characterized by decreased neural responses to repeated faces in visual cortices, remains a subject of ongoing debate regarding its underlying neural mechanisms. Our research harnesses advanced multivariate analysis techniques and the prowess of deep convolutional neural networks (DCNNs) in face recognition to bridge the gap between human electroencephalogram (EEG) data and DCNNs, especially in the context of facial repetition suppression. Our innovative reverse engineering approach, manipulating the neuronal activity in DCNNs and conducted representational comparisons between brain activations derived from human EEG and manipulated DCNN activations, provided insights into the underlying facial repetition suppression. Significantly, our findings advocate the fatigue mechanism as the dominant force behind the facial repetition suppression effect. Broadly, this integrative framework, bridging the human brain and DCNNs, offers a promising tool for simulating brain activity and making inferences regarding the neural mechanisms underpinning complex human behaviors.

Is increased activation in the fusiform face area to Greebles a result of appropriate expertise training or caused by Greebles' face likeness?

Background

In 2011, Brants et al. trained eight individuals to become Greeble experts and found neuronal inversion effects [NIEs; i.e., higher fusiform face area (FFA) activity for upright, rather than inverted Greebles]. These effects were also found for faces, both before and after training. By claiming to have replicated the seminal Greeble training study by Gauthier and colleagues in 1999, Brants et al. interpreted these results as participants viewing Greebles as faces throughout training, contrary to the original argument of subjects becoming Greeble experts only after training. However, Brants et al.'s claim presents two issues. First, their behavioral training results did not replicate those of Gauthier and Tarr conducted in 1997 and 1998, raising concerns of whether the right training regime had been adopted. Second, both a literature review and meta-analysis of NIEs in the FFA suggest its impotency as an index of the face(-like) processing.

Objectives

To empirically evaluate these issues, the present study compared two documented training paradigms Gauthier and colleagues in 1997 and 1998, and compared their impact on the brain.

Methods

Sixteen NCKU undergraduate and graduate students (nine girls) were recruited. Sixty Greeble exemplars were categorized by two genders, five families, and six individual levels. The participants were randomly divided into two groups (one for Greeble classification at all three levels and the other for gender- and individual-level training). Several fMRI tasks were administered at various time points, specifically, before training (1st), during training (2nd), and typically no <24 h after reaching expertise criterion (3rd).

Results

The ROI analysis results showed significant increases in the FFA for Greebles, and a clear neural "adaptation," both only in the Gauthier97 group and only after training, reflecting clear modulation of extensive experiences following an "appropriate" training regime. In both groups, no clear NIEs for faces nor Greebles were found, which was also in line with the review of extant studies bearing this comparison.

Conclusion

Collectively, these results invalidate the assumptions behind Brants et al.'s findings.

Repetition Probability Effects for Chinese Characters and German Words in the Visual Word Form Area

The magnitude of repetition suppression (RS), measured by fMRI, is modulated by the probability of repetitions (P(rep)) for various sensory stimulus categories. It has been suggested that for visually presented simple letters this P(rep) effect depends on the prior practices of the participants with the stimuli. Here we tested further if previous experiences affect the neural mechanisms of RS, leading to the modulatory effects of stimulus P(rep), for more complex lexical stimuli as well. We measured the BOLD signal in the Visual Word Form Area (VWFA) of native Chinese and German participants and estimated the P(rep) effects for Chinese characters and German words. The results showed a significant P(rep) effect for stimuli of the mother tongue in both participant groups. Interestingly, Chinese participants, learning German as a second language, also showed a significant P(rep) modulation of RS for German words while the German participants who had no prior experiences with the Chinese characters showed no such effects. Our findings suggest that P(rep) effects on RS are manifest for visual word processing as well, but only for words of a language with which participants are highly familiar. These results support further the idea that predictive processes, estimated by P(rep) modulations of RS, require prior experiences.

Visual short-term memory load modulates repetition related fMRI signal adaptation

While several computational models have suggested how predictive coding could be implemented on an algorithmic level, reference to cognitive processes remains rather sparse. A crucial process might be elevating relevant prior information from long-term memory to render it highly accessible for subsequent comparison with sensory input. In many models, visual short-term memory (VSTM) is considered as information from long-term memory in a state of elevated activity. We measured the BOLD signal in face-specific cortical areas using repetition suppression (RS) paradigm. RS has been associated with predictive processing in previous studies. We show that RS within the fusiform face area is significantly attenuated when VSTM is loaded with other, non-facial visual information. Although an unequivocal inference is not possible, the data indicate a role of VSTM for predictive processes as indexed by expectation-related RS.

Remembering the Past to See the Future

In addition to the role that our visual system plays in determining what we are seeing right now, visual computations contribute in important ways to predicting what we will see next. While the role of memory in creating future predictions is often overlooked, efficient predictive computation requires the use of information about the past to estimate future events. In this article, we introduce a framework for understanding the relationship between memory and visual prediction and review the two classes of mechanisms that the visual system relies on to create future predictions. We also discuss the principles that define the mapping from predictive computations to predictive mechanisms and how downstream brain areas interpret the predictive signals computed by the visual system. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Predictive brain: Addressing the level of representation by reviewing perceptual hysteresis

In recent years, the idea that the prediction of sensory input is one of the major computational goals of the nervous system led to the development of several large-scale theories of brain functioning, such as different versions of the Bayesian approach to brain functions, predictive coding theories of cognition and the Free-energy principle. During the years, various empirical phenomena have been re-interpreted within such frames, and have been considered as consequences of predictive processing. Here we focus on perceptual hysteresis, or serial dependence, as an exemplary case. We unravel a potential gap in the predictive frameworks and raise the idea that alternative explanations of this effect can solve this issue, as they address the type of cognitive and neural representations involved.

Pinpointing the neural signatures of single-exposure visual recognition memory

Memories of the images that we have seen are thought to be reflected in the reduction of neural responses in high-level visual areas such as inferotemporal (IT) cortex, a phenomenon known as repetition suppression (RS). We challenged this hypothesis with a task that required rhesus monkeys to report whether images were novel or repeated while ignoring variations in contrast, a stimulus attribute that is also known to modulate the overall IT response. The monkeys' behavior was largely contrast invariant, contrary to the predictions of an RS-inspired decoder, which could not distinguish responses to images that are repeated from those that are of lower contrast. However, the monkeys' behavioral patterns were well predicted by a linearly decodable variant in which the total spike count was corrected for contrast modulation. These results suggest that the IT neural activity pattern that best aligns with single-exposure visual recognition memory behavior is not RS but rather sensory referenced suppression: reductions in IT population response magnitude, corrected for sensory modulation.

Evaluating the evidence for expectation suppression in the visual system

Reports of expectation suppression have shaped the development of influential predictive coding-based theories of visual perception. However recent work has highlighted confounding factors that may mimic or inflate expectation suppression effects. In this review, we describe four confounds that are prevalent across experiments that tested for expectation suppression: effects of surprise, attention, stimulus repetition and adaptation, and stimulus novelty. With these confounds in mind we then critically review the evidence for expectation suppression across probabilistic cueing, statistical learning, oddball, action-outcome learning and apparent motion designs. We found evidence for expectation suppression within a specific subset of statistical learning designs that involved weeks of sequence learning prior to neural activity measurement. Across other experimental contexts, whereby stimulus appearance probabilities were learned within one or two testing sessions, there was inconsistent evidence for genuine expectation suppression. We discuss how an absence of expectation suppression could inform models of predictive processing, repetition suppression and perceptual decision-making. We also provide suggestions for designing experiments that may better test for expectation suppression in future work.

Active listening

This paper introduces active listening, as a unified framework for synthesising and recognising speech. The notion of active listening inherits from active inference, which considers perception and action under one universal imperative: to maximise the evidence for our (generative) models of the world. First, we describe a generative model of spoken words that simulates (i) how discrete lexical, prosodic, and speaker attributes give rise to continuous acoustic signals; and conversely (ii) how continuous acoustic signals are recognised as words. The 'active' aspect involves (covertly) segmenting spoken sentences and borrows ideas from active vision. It casts speech segmentation as the selection of internal actions, corresponding to the placement of word boundaries. Practically, word boundaries are selected that maximise the evidence for an internal model of how individual words are generated. We establish face validity by simulating speech recognition and showing how the inferred content of a sentence depends on prior beliefs and background noise. Finally, we consider predictive validity by associating neuronal or physiological responses, such as the mismatch negativity and P300, with belief updating under active listening, which is greatest in the absence of accurate prior beliefs about what will be heard next.

Novelty Manipulations, Memory Performance, and Predictive Coding: the Role of Unexpectedness

Novelty is central to the study of memory, but the wide range of experimental manipulations aimed to reveal its effects on learning produced inconsistent results. The novelty/encoding hypothesis suggests that novel information undergoes enhanced encoding and thus leads to benefits in memory, especially in recognition performance; however, recent studies cast doubts on this assumption. On the other hand, data from animal studies provided evidence on the robust effects of novelty manipulations on the neurophysiological correlates of memory processes. Conceptualizations and operationalizations of novelty are remarkably variable and were categorized into different subtypes, such as stimulus, context, associative or spatial novelty. Here, we summarize previous findings about the effects of novelty on memory and suggest that predictive coding theories provide a framework that could shed light on the differential influence of novelty manipulations on memory performance. In line with predictive coding theories, we emphasize the role of unexpectedness as a crucial property mediating the behavioral and neural effects of novelty manipulations.

Expectations about word stress modulate neural activity in speech-sensitive cortical areas

A recent dual-stream model of language processing proposed that the postero-dorsal stream performs predictive sequential processing of linguistic information via hierarchically organized internal models. However, it remains unexplored whether the prosodic segmentation of linguistic information involves predictive processes. Here, we addressed this question by investigating the processing of word stress, a major component of speech segmentation, using probabilistic repetition suppression (RS) modulation as a marker of predictive processing. In an event-related acoustic fMRI RS paradigm, we presented pairs of pseudowords having the same (Rep) or different (Alt) stress patterns, in blocks with varying Rep and Alt trial probabilities. We found that the BOLD signal was significantly lower for Rep than for Alt trials, indicating RS in the posterior and middle superior temporal gyrus (STG) bilaterally, and in the anterior STG in the left hemisphere. Importantly, the magnitude of RS was modulated by repetition probability in the posterior and middle STG. These results reveal the predictive processing of word stress in the STG areas and raise the possibility that words stress processing is related to the dorsal "where" auditory stream.

Evaluating the neurophysiological evidence for predictive processing as a model of perception

For many years, the dominant theoretical framework guiding research into the neural origins of perceptual experience has been provided by hierarchical feedforward models, in which sensory inputs are passed through a series of increasingly complex feature detectors. However, the long-standing orthodoxy of these accounts has recently been challenged by a radically different set of theories that contend that perception arises from a purely inferential process supported by two distinct classes of neurons: those that transmit predictions about sensory states and those that signal sensory information that deviates from those predictions. Although these predictive processing (PP) models have become increasingly influential in cognitive neuroscience, they are also criticized for lacking the empirical support to justify their status. This limited evidence base partly reflects the considerable methodological challenges that are presented when trying to test the unique predictions of these models. However, a confluence of technological and theoretical advances has prompted a recent surge in human and nonhuman neurophysiological research seeking to fill this empirical gap. Here, we will review this new research and evaluate the degree to which its findings support the key claims of PP.

Similar Expectation Effects for Immediate and Delayed Stimulus Repetitions

A prior cue or stimulus allows prediction of the future occurrence of an event and therefore reduces the associated neural activity in several cortical areas. This phenomenon is labeled expectation suppression (ES) and has recently been shown to be independent of the generally observed effects of stimulus repetitions (repetition suppression, RS: reduced neuronal response after the repetition of a given stimulus). While it has been shown that attentional cueing is strongly affected by the length of the cue-target delay, we have no information on the temporal dynamics of expectation effects, as in most prior studies of ES the delay between the predictive cue and the target (i.e., the inter-stimulus interval, ISI) was in the range of a few hundred milliseconds. Hence, we presented participants with pairs of faces where the first face could be used to build expectations regarding the second one, in the sense that one gender indicated repetition of the same face while the other gender predicted the occurrence of novel faces. In addition, we presented the stimulus pairs with two different ISIs (0.5 s for Immediate and 1.75 or 3.75 s for Delayed ISIs). We found significant RS as well as a reduced response for correctly predicted when compared to surprising trials in the fusiform face area. Importantly, the effects of repetition and expectation were both independent of the length of the ISI period. This implies that Immediate and Delayed cue-target stimulus arrangements lead to similar expectation effects in the face sensitive-visual cortex.

Intentionality and temporal binding: Do causality beliefs increase the perceived temporal attraction between events?

Intentional motor actions and their effects are bound together in temporal perception, resulting in the so-called intentional binding effect. In the current study, we address an alternative explanatory mechanism for the emergence of temporal binding by excluding the role of motor action. Employing a sensory-based Libet clock paradigm, we examined temporal perception of two different auditory stimuli, and tested the influence of beliefs about the causal relationship between the two auditory stimuli, thus simulating a crucial feature of intentional action. In two experiments, we found a robust temporal repulsion effect, indicating that instead of being attracted to each other, the auditory stimuli were shifted away from each other in temporal perception. Interestingly, repulsion was attenuated by causal beliefs, but this effect was fragile. Furthermore, temporal repulsion was unaffected by the intensity of prior learning. Findings are discussed in the context of intentional action awareness research and multisensory integration.

Visual novelty, curiosity, and intrinsic reward in machine learning and the brain

A strong preference for novelty emerges in infancy and is prevalent across the animal kingdom. When incorporated into reinforcement-based machine learning algorithms, visual novelty can act as an intrinsic reward signal that vastly increases the efficiency of exploration and expedites learning, particularly in situations where external rewards are difficult to obtain. Here we review parallels between recent developments in novelty-driven machine learning algorithms and our understanding of how visual novelty is computed and signaled in the primate brain. We propose that in the visual system, novelty representations are not configured with the principal goal of detecting novel objects, but rather with the broader goal of flexibly generalizing novelty information across different states in the service of driving novelty-based learning.

Significant repetition probability effects in schizophrenia

BACKGROUND

A growing body of evidence suggests that the comparison of expected and incoming sensory stimuli (the prediction-error (ε) processing) is impaired in schizophrenia patients (SZ). For example, in studies of mismatch negativity, an ERP component that signals ε, SZ patients show deficits in the auditory and visual modalities. To test the role of impaired ε processing further in SZ, using neuroimaging methods, we applied a repetition-suppression (RS) paradigm.

METHODS

Patients diagnosed with SZ (n = 17) as well as age- and sex- matched healthy control subjects (HC, n = 17) were presented with pairs of faces, which could either repeat or alternate. Additionally, the likelihood of repetition/alternation trials was modulated in individual blocks of fMRI recordings, testing the effects of repetition probability (P(rep)) on RS.

RESULTS

We found a significant RS in the fusiform and occipital face areas as well as in the lateral occipital cortex that was similar in healthy controls and SZ patients SZ. More importantly, we observed similar P(rep) effects (larger RS in blocks with high frequency of repetitions than in blocks with low repetition likelihood) in both the control and the patient group.

CONCLUSION

Our findings suggest that repetition_probability modulations affect the neural responses in schizophrenia patients and healthy participants similarly. This suggests that the neural mechanisms determining perceptual inferences based on stimulus probabilities remain unimpaired in schizophrenia.

Repetition Suppression for Noisy and Intact Faces in the Occipito-Temporal Cortex

Repetition suppression (RS), the relative lower neural response magnitude to repeated as compared to non-repeated stimuli, is often explained within the predictive coding framework. According to this theory, precise predictions (priors) together with less precise sensory evidences lead to decisions that are determined largely by the predictions and the other way around. In other words, the prediction error, namely the magnitude of RS, should depend on the precision of predictions and sensory inputs. In the current study, we aimed at testing this idea by manipulating the clarity and thereby the precision of the incoming sensory data by adding noise to the images. This resulted in an fMRI adaptation design with repeated or alternating trials showing clear or noisy face stimuli. Our results show a noise effect on the activity in the fusiform face area (FFA), namely less activation for noisy than for clear trials, which supports previous findings. No such effects could be found in OFA or LO. Data also showed reliable RS in the FFA (bilateral) and unilaterally in OFA (right) and LO (left). Interestingly, the noise added to the stimuli did not affect the magnitude of RS in any of the tested cortical areas. This suggests that the clarity of the sensory input is not crucial in determining the magnitude of RS.

Context-Based Facilitation in Visual Word Recognition: Evidence for Visual and Lexical But Not Pre-Lexical Contributions

Word familiarity and predictive context facilitate visual word processing, leading to faster recognition times and reduced neuronal responses. Previously, models with and without top-down connections, including lexical-semantic, pre-lexical (e.g., orthographic/phonological), and visual processing levels were successful in accounting for these facilitation effects. Here we systematically assessed context-based facilitation with a repetition priming task and explicitly dissociated pre-lexical and lexical processing levels using a pseudoword (PW) familiarization procedure. Experiment 1 investigated the temporal dynamics of neuronal facilitation effects with magnetoencephalography (MEG; N = 38 human participants), while experiment 2 assessed behavioral facilitation effects (N = 24 human participants). Across all stimulus conditions, MEG demonstrated context-based facilitation across multiple time windows starting at 100 ms, in occipital brain areas. This finding indicates context-based facilitation at an early visual processing level. In both experiments, we furthermore found an interaction of context and lexical familiarity, such that stimuli with associated meaning showed the strongest context-dependent facilitation in brain activation and behavior. Using MEG, this facilitation effect could be localized to the left anterior temporal lobe at around 400 ms, indicating within-level (i.e., exclusively lexical-semantic) facilitation but no top-down effects on earlier processing stages. Increased pre-lexical familiarity (in PWs familiarized utilizing training) did not enhance or reduce context effects significantly. We conclude that context-based facilitation is achieved within visual and lexical processing levels. Finally, by testing alternative hypotheses derived from mechanistic accounts of repetition suppression, we suggest that the facilitatory context effects found here are implemented using a predictive coding mechanism.

Prior Precision Modulates the Minimization of Auditory Prediction Error

The predictive coding model of perception proposes that successful representation of the perceptual world depends upon canceling out the discrepancy between prediction and sensory input (i.e., prediction error). Recent studies further suggest a distinction to be made between prediction error triggered by non-predicted stimuli of different prior precision (i.e., inverse variance). However, it is not fully understood how prediction error with different precision levels is minimized in the predictive process. Here, we conducted a magnetoencephalography (MEG) experiment which orthogonally manipulated prime-probe relation (for contextual precision) and stimulus repetition (for perceptual learning which decreases prediction error). We presented participants with cycles of tone quartets which consisted of three prime tones and one probe tone of randomly selected frequencies. Within each cycle, the three prime tones remained identical while the probe tones changed once at some point (e.g., from repetition of 123X to repetition of 123Y). Therefore, the repetition of probe tones can reveal the development of perceptual inferences in low and high precision contexts depending on their position within the cycle. We found that the two conditions resemble each other in terms of N1m modulation (as both were associated with N1m suppression) but differ in terms of N2m modulation. While repeated probe tones in low precision context did not exhibit any modulatory effect, repeated probe tones in high precision context elicited a suppression and rebound of the N2m source power. The differentiation suggested that the minimization of prediction error in low and high precision contexts likely involves distinct mechanisms.

Brain signatures of a multiscale process of sequence learning in humans

Extracting the temporal structure of sequences of events is crucial for perception, decision-making, and language processing. Here, we investigate the mechanisms by which the brain acquires knowledge of sequences and the possibility that successive brain responses reflect the progressive extraction of sequence statistics at different timescales. We measured brain activity using magnetoencephalography in humans exposed to auditory sequences with various statistical regularities, and we modeled this activity as theoretical surprise levels using several learning models. Successive brain waves related to different types of statistical inferences. Early post-stimulus brain waves denoted a sensitivity to a simple statistic, the frequency of items estimated over a long timescale (habituation). Mid-latency and late brain waves conformed qualitatively and quantitatively to the computational properties of a more complex inference: the learning of recent transition probabilities. Our findings thus support the existence of multiple computational systems for sequence processing involving statistical inferences at multiple scales.

Prediction error and repetition suppression have distinct effects on neural representations of visual information

Predictive coding theories argue that recent experience establishes expectations in the brain that generate prediction errors when violated. Prediction errors provide a possible explanation for repetition suppression, where evoked neural activity is attenuated across repeated presentations of the same stimulus. The predictive coding account argues repetition suppression arises because repeated stimuli are expected, whereas non-repeated stimuli are unexpected and thus elicit larger neural responses. Here, we employed electroencephalography in humans to test the predictive coding account of repetition suppression by presenting sequences of visual gratings with orientations that were expected either to repeat or change in separate blocks of trials. We applied multivariate forward modelling to determine how orientation selectivity was affected by repetition and prediction. Unexpected stimuli were associated with significantly enhanced orientation selectivity, whereas selectivity was unaffected for repeated stimuli. Our results suggest that repetition suppression and expectation have separable effects on neural representations of visual feature information.

Affective interoceptive inference: Evidence from heart-beat evoked brain potentials

The perception of internal bodily signals (interoception) is central to many theories of emotion and embodied cognition. According to recent theoretical views, the sensory processing of visceral signals such as one's own heartbeat is determined by top-down predictions about the expected interoceptive state of the body (interoceptive inference). In this EEG study we examined neural responses to heartbeats following expected and unexpected emotional stimuli. We used a modified stimulus repetition task in which pairs of facial expressions were presented with repeating or alternating emotional content, and we manipulated the emotional valence and the likelihood of stimulus repetition. We found that affective predictions of external socially relevant information modulated the heartbeat-evoked potential, a marker of cardiac interoception. Crucially, the HEP changes highly relied on the expected emotional content of the facial expression. Thus, expected negative faces led to a decreased HEP amplitude, whereas such an effect was not observed after an expected neutral face. These results suggest that valence-specific affective predictions, and their uniquely associated predicted bodily sensory state, can reduce or amplify cardiac interoceptive responses. In addition, the affective repetition effects were dependent on repetition probability, highlighting the influence of top-down exteroceptive predictions on interoception. Our results are in line with recent models of interoception supporting the idea that predicted bodily states influence sensory processing of salient external information.

Perceptual Expectations of Object Stimuli Modulate Repetition Suppression in a Delayed Repetition Design

Several fMRI and EEG/MEG studies show that repetition suppression (RS) effects are stronger when a stimulus repetition is expected compared to when a stimulus repetition is less expected. To date, the prevalent way to assess the influence of expectations on RS is via immediate stimulus repetition designs, that is, no intervening stimuli appear between the initial and repeated presentation of a stimulus. Since there is evidence that repetition lag may alter RS effects in a qualitative manner, the current study investigated how perceptual expectations modify RS effects on object stimuli when repetition lag is relatively long. Region of interest analyses in the left occipital cortex revealed a similar activation pattern as identified in previous studies on immediate lag: RS effects were strongest when repetitions were expected compared to decreased RS effects when repetitions were less expected. Therefore, the current study expands previous research in two ways: First, we replicate prior studies showing that perceptual expectation effects can be observed in object-sensitive occipital areas. Second, the finding that expectation effects can be found even for several-minute lags proposes that Bayesian inference processes are a relatively robust component in visual stimulus processing.

Face Repetition Probability Does Not Affect Repetition Suppression in Macaque Inferotemporal Cortex

Repetition suppression, which refers to reduced neural activity for repeated stimuli, is typically explained by bottom-up or local adaptation mechanisms. However, recent theories have emphasized the role of top-down processes, suggesting that this response reduction reflects the fulfillment of perceptual expectations. To support this, an influential human fMRI study showed that the magnitude of suppression is modulated by the probability of a repetition. No such repetition probability effect was found in macaque inferior temporal (IT) cortex for spiking activity despite the presence of repetition suppression. Contrary to the human fMRI studies that showed an effect of repetition probability, the macaque single-unit study used a large variety of unfamiliar stimuli and the monkeys were not required to attend the stimuli. Here, as in the human fMRI studies, we used faces as stimuli and made the monkeys attend to the stimulus content. We simultaneously recorded spiking activity and local field potentials (LFPs) in the middle lateral face patch (ML) of one monkey (male) and a face-responsive region of another (female). Although we observed significant repetition suppression of spiking activity and high gamma-band LFPs in both animals, there were no effects of repetition probability even when repetitions were task relevant and repetition probability affected behavioral decisions. In conclusion, despite the use of face stimuli and a stimulus-related task, no neural signature of repetition probability was present for faces in a face responsive patch of macaque IT. This further challenges a general perceptual expectation account of repetition suppression.SIGNIFICANCE STATEMENT Repetition suppression is a reduced brain activity for repeated stimuli commonly observed across species. In the predictive coding framework, such suppression is thought to reflect fulfilled perceptual expectations. Although this hypothesis is supported by several human fMRI studies reporting an effect of repetition probability on repetition suppression, this could not be replicated in single-cell recordings in monkey inferior temporal (IT) cortex. Subsequent studies narrowed down the conditions for the effect to requiring attention and being limited to particular stimulus categories such as faces. Here, we show that, even under these conditions, repetition suppression in monkey IT neurons is still unaffected by repetition probability, even in a task with a behavioral effect, challenging the perceptual expectation account of repetition suppression.

Visual mismatch response and fMRI signal adaptation correlate in the occipital-temporal cortex

Several electrophysiological studies found response differences to a given stimulus when it is repeated frequently as compared to when it occurs rarely in oddball sequences. Initially defined in acoustic perception, such difference also exists in the visual modality and is referred to as visual mismatch negativity (vMMN). However, the repetition of a stimulus also leads to the reduction of the blood oxygen-level dependent (BOLD) signal (fMRI adaptation, fMRIa) when compared to alternating stimuli in fMRI experiments. So far no study compared the vMMN to fMRIa within the same paradigm and participants. Here we tested the possible connection between fMRIa and vMMN in a visual oddball paradigm in two separate sessions, acquiring electrophysiological and neuroimaging data for real and false characters from the same participants. We found significant visual mismatch response (vMM) as well as fMRIa for both character types. Importantly, the magnitude of the vMM over the CP1 electrode cluster showed a significant correlation with the fMRIa within the letter form area, for real characters. This finding suggests that similar neural mechanisms are responsible for the two phenomena.

Neuroimaging results suggest the role of prediction in cross-domain priming

The repetition of a stimulus leads to shorter reaction times as well as to the reduction of neural activity. Previous encounters with closely related stimuli (primes) also lead to faster and often to more accurate processing of subsequent stimuli (targets). For instance, if the prime is a name, and the target is a face, the recognition of a persons’ face is facilitated by prior presentation of his/her name. A possible explanation for this phenomenon is that the prime allows predicting the occurrence of the target. To the best of our knowledge, so far, no study tested the neural correlates of such cross-domain priming with fMRI. To fill this gap, here we used names of famous persons as primes, and congruent or incongruent faces as targets. We found that congruent primes not only reduced RT, but also lowered the BOLD signal in bilateral fusiform (FFA) and occipital (OFA) face areas. This suggests that semantic information affects not only behavioral performance, but also neural responses in relatively early processing stages of the occipito-temporal cortex. We interpret our results in the framework of predictive coding theories.

A preference for mathematical processing outweighs the selectivity for Arabic numbers in the inferior temporal gyrus

A region in the posterior inferior temporal gyrus (ITG), referred to as the number form area (NFA, here ITG-numbers) has been implicated in the visual processing of Arabic numbers. However, it is unknown if this region is specifically involved in the visual encoding of Arabic numbers per se or in mathematical processing more broadly. Using functional magnetic resonance imaging (fMRI) during experiments that systematically vary tasks and stimuli, we find that mathematical processing, not preference to Arabic numbers, consistently drives both mean and distributed responses in the posterior ITG. While we replicated findings of higher responses in ITG-numbers to numbers than other visual stimuli during a 1-back task, this preference to numbers was abolished when participants engaged in mathematical processing. In contrast, an ITG region (ITG-math) that showed higher responses during an adding task vs. other tasks maintained this preference for mathematical processing across a wide range of stimuli including numbers, number/letter morphs, hands, and dice. Analysis of distributed responses across an anatomically-defined posterior ITG expanse further revealed that mathematical task but not Arabic number form can be successfully and consistently decoded from these distributed responses. Together, our findings suggest that the function of neuronal regions in the posterior ITG goes beyond the specific visual processing of Arabic numbers. We hypothesize that they ascribe numerical content to the visual input, irrespective of the format of the stimulus.

Prior Expectation Modulates Repetition Suppression without Perceptual Awareness

Stimulus repetition induces attenuated brain responses. This phenomenon, termed repetition suppression (RS), is classically held to stem from bottom-up neuronal adaptation. However, recent studies suggest that RS is driven by top-down predictive mechanisms. It remains controversial whether these top-down mechanisms of RS rely on conscious strategies, or if they represent a more fundamental aspect of perception, coding for physical properties of the repeated feature. The presence of top-down effects in the absence of perceptual awareness would indicate that conscious strategies are not sufficient to explain top-down mechanisms of RS. We combined an unconscious priming paradigm with EEG recordings and tested whether RS can be modulated by the probability of encountering a repetition, even in the absence of awareness. Our results show that both behavioural priming and RS near occipital areas are modulated by repetition probability, regardless of prime awareness. This contradicts previous findings that have argued that RS modulation is a by-product of conscious strategies. In contrast, we found that the increase in theta-band power following unrepeated trials – an index of conflict detection – is modulated only by expectations during conscious primes, implicating the use of conscious strategies. Together, our results suggest that the influence of predictions on RS can be either automatic in sensory brain regions or dependent on conscious strategies.

Further Evidence That the Effects of Repetition on Subjective Time Depend on Repetition Probability

Repeated stimuli typically have shorter apparent duration than novel stimuli. Most explanations for this effect have attributed it to the repeated stimuli being more expected or predictable than the novel items, but an emerging body of work suggests that repetition and expectation exert distinct effects on time perception. The present experiment replicated a recent study in which the probability of repetition was varied between blocks of trials. As in the previous work, the repetition effect was smaller when repeats were common (and therefore more expected) than when they were rare. These results add to growing evidence that, contrary to traditional accounts, expectation increases apparent duration whereas repetition compresses subjective time, perhaps via a low-level process like adaptation. These opposing processes can be seen as instances of a more general “processing principle,” according to which subjective time is a function of the perceptual strength of the stimulus representation, and therefore depends on a confluence of “bottom-up” and “top-down” variables.

Repetition suppression to objects is modulated by stimulus-specific expectations

Repeated exposure to the same stimulus results in an attenuated brain response in cortical regions that are activated during the processing of that stimulus. This phenomenon, called repetition suppression (RS), has been shown to be modulated by expectation. Typically, this is achieved by varying the probability of stimulus repetitions (P_rep) between blocks of an experiment, generating an abstract expectation that ‘things will repeat’. Here, we examined whether stimulus-specific expectations also modulate RS. We designed a task where expectation and repetition are manipulated independently, using stimulus-specific expectations. We investigated to which extent such stimulus-specific expectations modulated the visual evoked response to objects in lateral occipital cortex (LOC) and primary visual cortex (V1), using functional magnetic resonance imaging (fMRI). In LOC, we found that RS interacted with expectation, such that repetition suppression was more pronounced for unexpected relative to expected stimuli. Additionally, we found that the response of stimulus-preferring voxels in V1 was generally decreased when stimuli were expected. These results suggest that stimulus-specific expectations about objects modulate LOC and propagate back to the earliest cortical station processing visual input.

Expectation modulates repetition priming under high stimulus variability

Neural responses to stimuli are often attenuated by repeated presentation. When observed in blood oxygen level-dependent signals, this attenuation is known as fMRI adaptation (fMRIa) or fMRI repetition suppression. According to a prominent account, fMRIa reflects the fulfillment of perceptual expectations during recognition of repeated items (Summerfield, Trittschuh, Monti, Mesulam, & Egner, 2008). Supporting this idea, expectation has been shown to modulate fMRIa under some circumstances; however, it is not currently known whether expectation similarly modulates recognition performance. To address this lacuna, we measured behavioral and fMRI responses to faces while varying the extent to which each stimulus was informative about its successor. Behavioral priming was greater when repetitions were more likely, suggesting that recognition was facilitated by the expectation than an item would repeat. Notably, this effect was only observed when stimuli were drawn from a broad set of faces including many ethnicities and both genders, but not when stimuli were drawn from a narrower face set, thus making repetitions less informative. Moreover, expectation did not modulate fMRIa in face-selective cortex, contrary to previous studies, although an exploratory analysis indicated that it did so in a medial frontal region. These results support the idea that expectation modulates recognition efficiency, but insofar as behavioral effects of expectation were not accompanied by fMRI effects in visual cortex, they suggest that fMRIa cannot be entirely explained in terms of fulfilled expectations.

Unsuppressible Repetition Suppression and exemplar-specific Expectation Suppression in the Fusiform Face Area

Recent work casts Repetition Suppression (RS), i.e. the reduced neural response to repeated stimuli, as the consequence of reduced surprise for repeated inputs. This research, along with other studies documenting Expectation Suppression, i.e. reduced responses to expected stimuli, emphasizes the role of expectations and predictive codes in perception. Here, we use fMRI to further characterize the nature of predictive signals in the human brain. Prior to scanning, participants were implicitly exposed to associations within face pairs. Critically, we found that this resulted in exemplar-specific Expectation Suppression in the fusiform face-sensitive area (FFA): individual faces that could be predicted from the associations elicited reduced FFA responses, as compared to unpredictable faces. Thus, predictive signals in the FFA are specific to face exemplars, and not only generic to the category of face stimuli. In addition, we show that under such circumstances, the occurrence of surprising repetitions did not trigger enhanced brain responses, as had been recently hypothesized, but still suppressed responses, suggesting that repetition suppression might be partly 'unsuppressible'. Repetition effects cannot be fully modulated by expectations, which supports the recent view that expectation and repetition effects rest on partially independent mechanisms. Altogether, our study sheds light on the nature of expectation signals along the perceptual system.

Repetition Suppression and Memory for Faces is Reduced in Adults with Autism Spectrum Conditions

Autism spectrum conditions (ASC) are associated with a number of atypicalities in face processing, including difficulties in face memory. However, the neural mechanisms underlying this difficulty are unclear. In neurotypical individuals, repeated presentation of the same face is associated with a reduction in activity, known as repetition suppression (RS), in the fusiform face area (FFA). However, to date, no studies have investigated RS to faces in individuals with ASC, or the relationship between RS and face memory. Here, we measured RS to faces and geometric shapes in individuals with a clinical diagnosis of an ASC and in age and IQ matched controls. Relative to controls, the ASC group showed reduced RS to faces in bilateral FFA and reduced performance on a standardized test of face memory. By contrast, RS to shapes in object-selective regions and object memory did not differ between groups. Individual variation in face-memory performance was positively correlated with RS in regions of left parietal and prefrontal cortex. These findings suggest difficulties in face memory in ASC may be a consequence of differences in the way faces are stored and/or maintained across a network of regions involved in both visual perception and short-term/working memory.

Local expectation violations result in global activity gain in primary visual cortex

During natural perception, we often form expectations about upcoming input. These expectations are usually multifaceted - we expect a particular object at a particular location. However, expectations about spatial location and stimulus features have mostly been studied in isolation, and it is unclear whether feature-based expectation can be spatially specific. Interestingly, feature-based attention automatically spreads to unattended locations. It is still an open question whether the neural mechanisms underlying feature-based expectation differ from those underlying feature-based attention. Therefore, establishing whether the effects of feature-based expectation are spatially specific may inform this debate. Here, we investigated this by inducing expectations of a specific stimulus feature at a specific location, and probing the effects on sensory processing across the visual field using fMRI. We found an enhanced sensory response for unexpected stimuli, which was elicited only when there was a violation of expectation at the specific location where participants formed a stimulus expectation. The neural consequences of this expectation violation, however, spread to cortical locations processing the stimulus in the opposite hemifield. This suggests that an expectation violation at one location in the visual world can lead to a spatially non-specific gain increase across the visual field.

Stimulus context alters neural representations of faces in inferotemporal cortex

One goal of our nervous system is to form predictions about the world around us to facilitate our responses to upcoming events. One basis for such predictions could be the recently encountered visual stimuli, or the recent statistics of the visual environment. We examined the effect of recently experienced stimulus statistics on the visual representation of face stimuli by recording the responses of face-responsive neurons in the final stage of visual object recognition, the inferotemporal (IT) cortex, during blocks in which the probability of seeing a particular face was either 100% or only 12%. During the block with only face images, ∼30% of IT neurons exhibit enhanced anticipatory activity before the evoked visual response. This anticipatory modulation is followed by greater activity, broader view tuning, more distributed processing, and more reliable responses of IT neurons to the face stimuli. These changes in the visual response were sufficient to improve the ability of IT neurons to represent a variable property of the predictable face images (viewing angle), as measured by the performance of a simple linear classifier. These results demonstrate that the recent statistics of the visual environment can facilitate processing of stimulus information in the population neuronal representation.

NEW & NOTEWORTHY

Neurons in inferotemporal (IT) cortex anticipate the arrival of a predictable stimulus, and visual responses to an expected stimulus are more distributed throughout the population of IT neurons, providing an enhanced representation of second-order stimulus information (in this case, viewing angle). The findings reveal a potential neural basis for the behavioral benefits of contextual expectation.