Stimulus predictability reduces responses in primary visual cortex

Expectation Suppression in Early Visual Cortex Depends on Task Set

Stimulus expectation can modulate neural responses in early sensory cortical regions, with expected stimuli often leading to a reduced neural response. However, it is unclear whether this expectation suppression is an automatic phenomenon or is instead dependent on the type of task a subject is engaged in. To investigate this, human subjects were presented with visual grating stimuli in the periphery that were either predictable or non-predictable while they performed three tasks that differently engaged cognitive resources. In two of the tasks, the predictable stimulus was task-irrelevant and spatial attention was engaged at fixation, with a high load on either perceptual or working memory resources. In the third task, the predictable stimulus was task-relevant, and therefore spatially attended. We observed that expectation suppression is dependent on the cognitive resources engaged by a subjects’ current task. When the grating was task-irrelevant, expectation suppression for predictable items was visible in retinotopically specific areas of early visual cortex (V1-V3) during the perceptual task, but it was abolished when working memory was loaded. When the grating was task-relevant and spatially attended, there was no significant effect of expectation in early visual cortex. These results suggest that expectation suppression is not an automatic phenomenon, but dependent on attentional state and type of available cognitive resources.

Cognitive workload modulation through degraded visual stimuli: a single-trial EEG study

OBJECTIVE

Our experiments explored the effect of visual stimuli degradation on cognitive workload.

APPROACH

We investigated the subjective assessment, event-related potentials (ERPs) as well as electroencephalogram (EEG) as measures of cognitive workload.

MAIN RESULTS

These experiments confirm that degradation of visual stimuli increases cognitive workload as assessed by subjective NASA task load index and confirmed by the observed P300 amplitude attenuation. Furthermore, the single-trial multi-level classification using features extracted from ERPs and EEG is found to be promising. Specifically, the adopted single-trial oscillatory EEG/ERP detection method achieved an average accuracy of 85% for discriminating 4 workload levels. Additionally, we found from the spatial patterns obtained from EEG signals that the frontal parts carry information that can be used for differentiating workload levels.

SIGNIFICANCE

Our results show that visual stimuli can modulate cognitive workload, and the modulation can be measured by the single trial EEG/ERP detection method.

Neuronal oscillations as a mechanistic substrate of auditory temporal prediction

Neuronal oscillations are comprised of rhythmic fluctuations of excitability that are synchronized in ensembles of neurons and thus function as temporal filters that dynamically organize sensory processing. When perception relies on anticipatory mechanisms, ongoing oscillations also provide a neurophysiological substrate for temporal prediction. In this article, we review evidence for this account with a focus on auditory perception. We argue that such "oscillatory temporal predictions" can selectively amplify neuronal sensitivity to inputs that occur in a predicted, task-relevant rhythm and optimize temporal selection. We elaborate this argument for a prototypic example, speech processing, where information is present at multiple time scales, with delta, theta, and low-gamma oscillations being specifically and simultaneously engaged, enabling multiplexing. We then consider the origin of temporal predictions, specifically the idea that the motor system is involved in the generation of such prior information. Finally, we place temporal predictions in the general context of internal models, discussing how they interact with feature-based or spatial predictions. We propose that complementary predictions interact synergistically according to a dominance hierarchy, shaping perception in the form of a multidimensional filter mechanism.

Predictions to motion stimuli in human early visual cortex: Effects of motion displacement on motion predictability

Recently, several studies showed that fMRI BOLD responses to moving random dot stimuli are enhanced at the location of dot appearance, i.e., the motion trailing edge. Possibly, BOLD activity in human visual cortex reflects predictability of visual motion input. In the current study, we investigate to what extent fMRI BOLD responses reflect estimated predictions to visual motion. We varied motion displacement parameters (duration and velocity), while measuring BOLD amplitudes as a function of distance from the trailing edge. We have found that for all stimulus configurations, BOLD signals decrease with increasing distance from the trailing edge. This finding indicates that neural activity directly reflects the predictability of moving dots, rather than their appearance within classical receptive fields. However, different motion displacement parameters exerted only marginal effects on predictability, suggesting that early visual cortex does not literally predict motion trajectories. Rather, the results reveal a heuristic mechanism of motion suppression from trailing to leading edge, plausibly mediated through short-range horizontal connections. Simple heuristic suppression allows the visual system to recognize novel input among many motion signals, while being most energy efficient.

Activity of somatosensory-responsive neurons in high subdivisions of SI cortex during locomotion

Responses of neurons in the primary somatosensory cortex during movements are poorly understood, even during such simple tasks as walking on a flat surface. In this study, we analyzed spike discharges of neurons in the rostral bank of the ansate sulcus (areas 1-2) in 2 cats while the cats walked on a flat surface or on a horizontal ladder, a complex task requiring accurate stepping. All neurons (n = 82) that had receptive fields (RFs) on the contralateral forelimb exhibited frequency modulation of their activity that was phase locked to the stride cycle during simple locomotion. Neurons with proximal RFs (upper arm/shoulder) and pyramidal tract-projecting neurons (PTNs) with fast-conducting axons tended to fire at peak rates in the middle of the swing phase, whereas neurons with RFs on the distal limb (wrist/paw) and slow-conducting PTNs typically showed peak firing at the transition between swing and stance phases. Eleven of 12 neurons with tactile RFs on the volar forepaw began firing toward the end of swing, with peak activity occurring at the moment of foot contact with floor, thereby preceding the evoked sensory volley from touch receptors. Requirement to step accurately on the ladder affected 91% of the neurons, suggesting their involvement in control of accuracy of stepping. During both tasks, neurons exhibited a wide variety of spike distributions within the stride cycle, suggesting that, during either simple or ladder locomotion, they represent the cycling somatosensory events in their activity both predictively before and reflectively after these events take place.

Reappraising abstract paintings after exposure to background information

Can knowledge help viewers when they appreciate an artwork? Experts’ judgments of the aesthetic value of a painting often differ from the estimates of naïve viewers, and this phenomenon is especially pronounced in the aesthetic judgment of abstract paintings. We compared the changes in aesthetic judgments of naïve viewers while they were progressively exposed to five pieces of background information. The participants were asked to report their aesthetic judgments of a given painting after each piece of information was presented. We found that commentaries by the artist and a critic significantly increased the subjective aesthetic ratings. Does knowledge enable experts to attend to the visual features in a painting and to link it to the evaluative conventions, thus potentially causing different aesthetic judgments? To investigate whether a specific pattern of attention is essential for the knowledge-based appreciation, we tracked the eye movements of subjects while viewing a painting with a commentary by the artist and with a commentary by a critic. We observed that critics’ commentaries directed the viewers’ attention to the visual components that were highly relevant to the presented commentary. However, attention to specific features of a painting was not necessary for increasing the subjective aesthetic judgment when the artists’ commentary was presented. Our results suggest that at least two different cognitive mechanisms may be involved in knowledge- guided aesthetic judgments while viewers reappraise a painting.

Behavioral oscillation in priming: competing perceptual predictions conveyed in alternating theta-band rhythms

The brain constantly creates perceptual predictions about forthcoming stimuli to guide perception efficiently. Abundant studies have demonstrated that perceptual predictions modulate sensory activities depending on whether the actual inputs are consistent with one particular prediction. In real-life contexts, however, multiple and even conflicting predictions might concurrently exist to be tested, requiring a multiprediction coordination process. It remains largely unknown how multiple hypotheses are conveyed and harmonized to guide moment-by-moment perception. Based on recent findings revealing that multiple locations are sampled alternatively in various phase of attentional rhythms, we hypothesize that this oscillation-based temporal organization mechanism may also underlie the multiprediction coordination process. To address the issue, we used well established priming paradigms in combination with a time-resolved behavioral approach to investigate the fine temporal dynamics of the multiprediction harmonization course in human subjects. We first replicate classical priming effects in slowly developing trends of priming time courses. Second, after removing the typical priming patterns, we reveal a new theta-band (∼4 Hz) oscillatory component in the priming behavioral data regardless of whether the prime was masked. Third, we show that these theta-band priming oscillations triggered by congruent and incongruent primes are in an out-of-phase relationship. These findings suggest that perceptual predictions return to low-sensory areas not continuously but recurrently in a theta-band rhythm (every 200-300 ms) and that multiple predictions are dynamically coordinated in time by being conveyed in different phases of the theta-band oscillations to achieve dissociated but temporally organized neural representations.

Predictive motor control of sensory dynamics in auditory active sensing

Neuronal oscillations present potential physiological substrates for brain operations that require temporal prediction. We review this idea in the context of auditory perception. Using speech as an exemplar, we illustrate how hierarchically organized oscillations can be used to parse and encode complex input streams. We then consider the motor system as a major source of rhythms (temporal priors) in auditory processing, that act in concert with attention to sharpen sensory representations and link them across areas. We discuss the circuits that could mediate this audio-motor interaction, notably the potential role of the somatosensory system. Finally, we reposition temporal predictions in the context of internal models, discussing how they interact with feature-based or spatial predictions. We argue that complementary predictions interact synergistically according to the organizational principles of each sensory system, forming multidimensional filters crucial to perception.

Temporal expectation and attention jointly modulate auditory oscillatory activity in the beta band

The neural response to a stimulus is influenced by endogenous factors such as expectation and attention. Current research suggests that expectation and attention exert their effects in opposite directions, where expectation decreases neural activity in sensory areas, while attention increases it. However, expectation and attention are usually studied either in isolation or confounded with each other. A recent study suggests that expectation and attention may act jointly on sensory processing, by increasing the neural response to expected events when they are attended, but decreasing it when they are unattended. Here we test this hypothesis in an auditory temporal cueing paradigm using magnetoencephalography in humans. In our study participants attended to, or away from, tones that could arrive at expected or unexpected moments. We found a decrease in auditory beta band synchrony to expected (versus unexpected) tones if they were unattended, but no difference if they were attended. Modulations in beta power were already evident prior to the expected onset times of the tones. These findings suggest that expectation and attention jointly modulate sensory processing.

Decoding the future from past experience: learning shapes predictions in early visual cortex

Learning the structure of the environment is critical for interpreting the current scene and predicting upcoming events. However, the brain mechanisms that support our ability to translate knowledge about scene statistics to sensory predictions remain largely unknown. Here we provide evidence that learning of temporal regularities shapes representations in early visual cortex that relate to our ability to predict sensory events. We tested the participants' ability to predict the orientation of a test stimulus after exposure to sequences of leftward- or rightward-oriented gratings. Using fMRI decoding, we identified brain patterns related to the observers' visual predictions rather than stimulus-driven activity. Decoding of predicted orientations following structured sequences was enhanced after training, while decoding of cued orientations following exposure to random sequences did not change. These predictive representations appear to be driven by the same large-scale neural populations that encode actual stimulus orientation and to be specific to the learned sequence structure. Thus our findings provide evidence that learning temporal structures supports our ability to predict future events by reactivating selective sensory representations as early as in primary visual cortex.

Shape perception simultaneously up- and downregulates neural activity in the primary visual cortex

An essential part of visual perception is the grouping of local elements (such as edges and lines) into coherent shapes. Previous studies have shown that this grouping process modulates neural activity in the primary visual cortex (V1) that is signaling the local elements [1-4]. However, the nature of this modulation is controversial. Some studies find that shape perception reduces neural activity in V1 [2, 5, 6], while others report increased V1 activity during shape perception [1, 3, 4, 7-10]. Neurocomputational theories that cast perception as a generative process [11-13] propose that feedback connections carry predictions (i.e., the generative model), while feedforward connections signal the mismatch between top-down predictions and bottom-up inputs. Within this framework, the effect of feedback on early visual cortex may be either enhancing or suppressive, depending on whether the feedback signal is met by congruent bottom-up input. Here, we tested this hypothesis by quantifying the spatial profile of neural activity in V1 during the perception of illusory shapes using population receptive field mapping. We find that shape perception concurrently increases neural activity in regions of V1 that have a receptive field on the shape but do not receive bottom-up input and suppresses activity in regions of V1 that receive bottom-up input that is predicted by the shape. These effects were not modulated by task requirements. Together, these findings suggest that shape perception changes lower-order sensory representations in a highly specific and automatic manner, in line with theories that cast perception in terms of hierarchical generative models.

A DCM study of spectral asymmetries in feedforward and feedback connections between visual areas V1 and V4 in the monkey

This paper reports a dynamic causal modeling study of electrocorticographic (ECoG) data that addresses functional asymmetries between forward and backward connections in the visual cortical hierarchy. Specifically, we ask whether forward connections employ gamma-band frequencies, while backward connections preferentially use lower (beta-band) frequencies. We addressed this question by modeling empirical cross spectra using a neural mass model equipped with superficial and deep pyramidal cell populations-that model the source of forward and backward connections, respectively. This enabled us to reconstruct the transfer functions and associated spectra of specific subpopulations within cortical sources. We first established that Bayesian model comparison was able to discriminate between forward and backward connections, defined in terms of their cells of origin. We then confirmed that model selection was able to identify extrastriate (V4) sources as being hierarchically higher than early visual (V1) sources. Finally, an examination of the auto spectra and transfer functions associated with superficial and deep pyramidal cells confirmed that forward connections employed predominantly higher (gamma) frequencies, while backward connections were mediated by lower (alpha/beta) frequencies. We discuss these findings in relation to current views about alpha, beta, and gamma oscillations and predictive coding in the brain.

Filling gaps in visual motion for target capture

A remarkable challenge our brain must face constantly when interacting with the environment is represented by ambiguous and, at times, even missing sensory information. This is particularly compelling for visual information, being the main sensory system we rely upon to gather cues about the external world. It is not uncommon, for example, that objects catching our attention may disappear temporarily from view, occluded by visual obstacles in the foreground. Nevertheless, we are often able to keep our gaze on them throughout the occlusion or even catch them on the fly in the face of the transient lack of visual motion information. This implies that the brain can fill the gaps of missing sensory information by extrapolating the object motion through the occlusion. In recent years, much experimental evidence has been accumulated that both perceptual and motor processes exploit visual motion extrapolation mechanisms. Moreover, neurophysiological and neuroimaging studies have identified brain regions potentially involved in the predictive representation of the occluded target motion. Within this framework, ocular pursuit and manual interceptive behavior have proven to be useful experimental models for investigating visual extrapolation mechanisms. Studies in these fields have pointed out that visual motion extrapolation processes depend on manifold information related to short-term memory representations of the target motion before the occlusion, as well as to longer term representations derived from previous experience with the environment. We will review recent oculomotor and manual interception literature to provide up-to-date views on the neurophysiological underpinnings of visual motion extrapolation.

Automatic prediction regarding the next state of a visual object: Electrophysiological indicators of prediction match and mismatch

Behavioral phenomena such as representational momentum suggest that the brain can automatically predict the next state of a visual object, based on sequential rules embedded in its preceding spatiotemporal context. To identify electrophysiological indicators of automatic visual prediction in terms of prediction match and mismatch, we recorded event-related brain potentials (ERPs) while participants passively viewed three types of task-irrelevant sequences of a bar stimulus: (1) an oddball sequence, which contained a sequential rule defined by stimulus repetition, providing repetition-rule-conforming (standard) and -violating (deviant) stimuli; (2) a rotating-oddball sequence, which contained a sequential rule defined by stimulus change (i.e., rotation), providing change-rule-conforming (regular) and -violating (irregular) stimuli; and (3) a random sequence, which did not contain a sequential rule, providing a neutral (control) stimulus. This protocol allowed us to expect that (1) an ERP effect that reflects a prediction-mismatch process should be exclusively observed in both the deviant-minus-control and irregular-minus-control comparisons and (2) an ERP effect that reflects a prediction-match process should be exclusively observed in both the standard-minus-control and regular-minus-control comparisons. The results showed that the ERP effect that met the criterion for prediction mismatch was an occipito-temporal negative deflection at around 170-300ms (visual mismatch negativity), while the ERP effect that met the criterion for prediction match was a frontal/central negative deflection at around 150-270ms (probably, the reduction of P2). These two contrasting ERP effects support a hypothetical view that automatic visual prediction would involve both an increase in the neural response to prediction-incongruent (i.e., novel) events and a decrease in the neural response to prediction-congruent (i.e., redundant) events. This article is part of a Special Issue entitled SI: Prediction and Attention.

The role of early stages of cortical visual processing in size and distance judgment: a transcranial direct current stimulation study

Recent research suggests that V1 plays an active role in the judgment of size and distance. Nevertheless, no research has been performed using direct brain stimulation to address this issue. We used transcranial direct-current stimulation (tDCS) to directly modulate the early stages of cortical visual processing while measuring size and distance perception with a psychophysical scaling method of magnitude estimation in a repeated-measures design. The subjects randomly received anodal, cathodal, and sham tDCS in separate sessions starting with size or distance judgment tasks. Power functions were fit to the size judgment data, whereas logarithmic functions were fit to distance judgment data. Slopes and R(2) were compared with separate repeated-measures analyses of variance with two factors: task (size vs. distance) and tDCS (anodal vs. cathodal vs. sham). Anodal tDCS significantly decreased slopes, apparently interfering with size perception. No effects were found for distance perception. Consistent with previous studies, the results of the size task appeared to reflect a prothetic continuum, whereas the results of the distance task seemed to reflect a metathetic continuum. The differential effects of tDCS on these tasks may support the hypothesis that different physiological mechanisms underlie judgments on these two continua. The results further suggest the complex involvement of the early visual cortex in size judgment tasks that go beyond the simple representation of low-level stimulus properties. This supports predictive coding models and experimental findings that suggest that higher-order visual areas may inhibit incoming information from the early visual cortex through feedback connections when complex tasks are performed.

Parallel processing streams for motor output and sensory prediction during action preparation

Sensory consequences of one's own actions are perceived as less intense than identical, externally generated stimuli. This is generally taken as evidence for sensory prediction of action consequences. Accordingly, recent theoretical models explain this attenuation by an anticipatory modulation of sensory processing prior to stimulus onset (Roussel et al. 2013) or even action execution (Brown et al. 2013). Experimentally, prestimulus changes that occur in anticipation of self-generated sensations are difficult to disentangle from more general effects of stimulus expectation, attention and task load (performing an action). Here, we show that an established manipulation of subjective agency over a stimulus leads to a predictive modulation in sensory cortex that is independent of these factors. We recorded magnetoencephalography while subjects performed a simple action with either hand and judged the loudness of a tone caused by the action. Effector selection was manipulated by subliminal motor priming. Compatible priming is known to enhance a subjective experience of agency over a consequent stimulus (Chambon and Haggard 2012). In line with this effect on subjective agency, we found stronger sensory attenuation when the action that caused the tone was compatibly primed. This perceptual effect was reflected in a transient phase-locked signal in auditory cortex before stimulus onset and motor execution. Interestingly, this sensory signal emerged at a time when the hemispheric lateralization of motor signals in M1 indicated ongoing effector selection. Our findings confirm theoretical predictions of a sensory modulation prior to self-generated sensations and support the idea that a sensory prediction is generated in parallel to motor output (Walsh and Haggard 2010), before an efference copy becomes available.

Effects of prediction and contextual support on lexical processing: prediction takes precedence

Readers may use contextual information to anticipate and pre-activate specific lexical items during reading. However, prior studies have not clearly dissociated the effects of accurate lexical prediction from other forms of contextual facilitation such as plausibility or semantic priming. In this study, we measured electrophysiological responses to predicted and unpredicted target words in passages providing varying levels of contextual support. This method was used to isolate the neural effects of prediction from other potential contextual influences on lexical processing. While both prediction and discourse context influenced ERP amplitudes within the time range of the N400, the effects of prediction occurred much more rapidly, preceding contextual facilitation by approximately 100 ms. In addition, a frontal, post-N400 positivity (PNP) was modulated by both prediction accuracy and the overall plausibility of the preceding passage. These results suggest a unique temporal primacy for prediction in facilitating lexical access. They also suggest that the frontal PNP may index the costs of revising discourse representations following an incorrect lexical prediction.

The effects of prediction on the perception for own-race and other-race faces

Human beings do not passively perceive important social features about others such as race and age in social interactions. Instead, it is proposed that humans might continuously generate predictions about these social features based on prior similar experiences. Pre-awareness of racial information conveyed by others' faces enables individuals to act in “culturally appropriate” ways, which is useful for interpersonal relations in different ethnicity groups. However, little is known about the effects of prediction on the perception for own-race and other-race faces. Here, we addressed this issue using high temporal resolution event-related potential techniques. In total, data from 24 participants (13 women and 11 men) were analyzed. It was found that the N170 amplitudes elicited by other-race faces, but not own-race faces, were significantly smaller in the predictable condition compared to the unpredictable condition, reflecting a switch to holistic processing of other-race faces when those faces were predictable. In this respect, top-down prediction about face race might contribute to the elimination of the other-race effect (one face recognition impairment). Furthermore, smaller P300 amplitudes were observed for the predictable than for unpredictable conditions, which suggested that the prediction of race reduced the neural responses of human brains.

Contributions of cortical feedback to sensory processing in primary visual cortex

Closing the structure-function divide is more challenging in the brain than in any other organ (Lichtman and Denk, 2011). For example, in early visual cortex, feedback projections to V1 can be quantified (e.g., Budd, 1998) but the understanding of feedback function is comparatively rudimentary (Muckli and Petro, 2013). Focusing on the function of feedback, we discuss how textbook descriptions mask the complexity of V1 responses, and how feedback and local activity reflects not only sensory processing but internal brain states.

Enhanced Alpha-oscillations in Visual Cortex during Anticipation of Self-generated Visual Stimulation

The perceived intensity of sensory stimuli is reduced when these stimuli are caused by the observer's actions. This phenomenon is traditionally explained by forward models of sensory action–outcome, which arise from motor processing. Although these forward models critically predict anticipatory modulation of sensory neural processing, neurophysiological evidence for anticipatory modulation is sparse and has not been linked to perceptual data showing sensory attenuation. By combining a psychophysical task involving contrast discrimination with source-level time–frequency analysis of MEG data, we demonstrate that the amplitude of alpha-oscillations in visual cortex is enhanced before the onset of a visual stimulus when the identity and onset of the stimulus are controlled by participants' motor actions. Critically, this prestimulus enhancement of alpha-amplitude is paralleled by psychophysical judgments of a reduced contrast for this stimulus. We suggest that alpha-oscillations in visual cortex preceding self-generated visual stimulation are a likely neurophysiological signature of motor-induced sensory anticipation and mediate sensory attenuation. We discuss our results in relation to proposals that attribute generic inhibitory functions to alpha-oscillations in prioritizing and gating sensory information via top–down control.

How and when predictability interacts with accentuation in temporally selective attention during speech comprehension

The present study used EEG to investigate how and when top-down prediction interacts with bottom-up acoustic signals in temporally selective attention during speech comprehension. Mandarin Chinese spoken sentences were used as stimuli. We systematically manipulated the predictability and de/accentuation of the critical words in the sentence context. Meanwhile, a linguistic attention probe 'ba' was presented concurrently with the critical words or not. The results showed that, first, words with a linguistic attention probe elicited a larger N1 than those without a probe. The latency of this N1 effect was shortened for accented or lowly predictable words, indicating more attentional resources allocated to these words. Importantly, prediction and accentuation showed a complementary interplay on the latency of this N1 effect, demonstrating that when the words had already attracted attention due to low predictability or due to the presence of pitch accent, the other factor did not modulate attention allocation anymore. Second, relative to the lowly predictable words, the highly predictable words elicited a reduced N400 and enhanced gamma-band power increases, especially under the accented conditions; moreover, under the accented conditions, shorter N1 peak-latency was found to correlate with larger gamma-band power enhancement, which indicates that a close relationship might exist between early selective attention and later semantic integration. Finally, the interaction between top-down selective attention (driven by prediction) and bottom-up selective attention (driven by accentuation) occurred before lexical-semantic processing, namely before the N400 effect evoked by predictability, which was discussed with regard to the language comprehension models.

Higher brain functions served by the lowly rodent primary visual cortex

It has been more than 50 years since the first description of ocular dominance plasticity--the profound modification of primary visual cortex (V1) following temporary monocular deprivation. This discovery immediately attracted the intense interest of neurobiologists focused on the general question of how experience and deprivation modify the brain as a potential substrate for learning and memory. The pace of discovery has quickened considerably in recent years as mice have become the preferred species to study visual cortical plasticity, and new studies have overturned the dogma that primary sensory cortex is immutable after a developmental critical period. Recent work has shown that, in addition to ocular dominance plasticity, adult visual cortex exhibits several forms of response modification previously considered the exclusive province of higher cortical areas. These "higher brain functions" include neural reports of stimulus familiarity, reward-timing prediction, and spatiotemporal sequence learning. Primary visual cortex can no longer be viewed as a simple visual feature detector with static properties determined during early development. Rodent V1 is a rich and dynamic cortical area in which functions normally associated only with "higher" brain regions can be studied at the mechanistic level.

Statistical learning of serial visual transitions by neurons in monkey inferotemporal cortex

If monkeys repeatedly, over the course of weeks, view displays in which two images appear in fixed sequence, then neurons of inferotemporal cortex (ITC) come to exhibit prediction suppression. The response to the trailing image is weaker if it follows the leading image with which it was paired during training than if it follows some other leading image. Prediction suppression is a plausible neural mechanism for statistical learning of visual transitions such as has been demonstrated in behavioral studies of human infants and adults. However, in the human studies, subjects are exposed to continuous sequences in which the same image can be both predicted and predicting and statistical dependency can exist between nonadjacent items. The aim of the present study was to investigate whether prediction suppression in ITC develops under such circumstances. To resolve this issue, we exposed monkeys repeatedly to triplets of images presented in fixed order. The results indicate that prediction suppression can be induced by training not only with pairs of images but also with longer sequences.

Tracing path-guided apparent motion in human primary visual cortex V1

Vision is a constructive process. For example, a square, flashed at two distinct locations one after the other, appears to move smoothly between the two locations rather than as two separate flashes (apparent motion). Apparent motion is usually perceived along the shortest path between locations. Previous studies have shown that retinotopic activity in V1 correlates well with the subjective filling-in in apparent motion. If V1 activity truly reflects illusory motion, it should flexibly reflect filling-in of any path, subjectively perceived. Here, we used a path-guided apparent motion paradigm in which a faint cue, presented in addition to the squares, leads to a curved illusory motion path. We found retinotopic activity in V1 to reflect the illusory filling-in of the curved path, similarly to filling-in with linear, shortest paths. Moreover, our results show that activity along the linear path was less selective to stimulus conditions than the activity along the curved path. This finding may be interpreted as V1 activity representing a small subset of infinitely many possible solutions to ambiguous stimuli, whilst giving more weight to the shortest path/energy solution.

Face processing regions are sensitive to distinct aspects of temporal sequence in facial dynamics

Facial movement conveys important information for social interactions, yet its neural processing is poorly understood. Computational models propose that shape- and temporal sequence sensitive mechanisms interact in processing dynamic faces. While face processing regions are known to respond to facial movement, their sensitivity to particular temporal sequences has barely been studied. Here we used fMRI to examine the sensitivity of human face-processing regions to two aspects of directionality in facial movement trajectories. We presented genuine movie recordings of increasing and decreasing fear expressions, each of which were played in natural or reversed frame order. This two-by-two factorial design matched low-level visual properties, static content and motion energy within each factor, emotion-direction (increasing or decreasing emotion) and timeline (natural versus artificial). The results showed sensitivity for emotion-direction in FFA, which was timeline-dependent as it only occurred within the natural frame order, and sensitivity to timeline in the STS, which was emotion-direction-dependent as it only occurred for decreased fear. The occipital face area (OFA) was sensitive to the factor timeline. These findings reveal interacting temporal sequence sensitive mechanisms that are responsive to both ecological meaning and to prototypical unfolding of facial dynamics. These mechanisms are temporally directional, provide socially relevant information regarding emotional state or naturalness of behavior, and agree with predictions from modeling and predictive coding theory.

The influence of spontaneous brain oscillations on apparent motion perception

A good example of inferential processes in perception is long-range apparent motion (AM), the illusory percept of visual motion that occurs when two spatially distinct stationary visual objects are presented in alternating sequence. The AM illusion is strongest at presentation frequencies around 3 Hz. At lower presentation frequencies, the percept varies from trial to trial between AM and sequential alternation, while at higher frequencies perception varies between AM and two simultaneously flickering objects. Previous studies have demonstrated that prestimulus alpha oscillations explain trial-to-trial variability in detection performance for visual stimuli presented at threshold. In the present study, we investigated whether fluctuations of prestimulus alpha oscillations can also account for variations in AM perception. Prestimulus alpha power was stronger when observers reported AM perception in subsequent trials with low presentation frequencies, while at high presentation frequencies there were no significant differences in alpha power preceding AM and veridical flicker perception. Moreover, when observers perceived AM the prestimulus functional connectivity between frontal and occipital channels was increased in the alpha band, as revealed by the imaginary part of coherency, which is insensitive to artefacts from volume conduction. Dynamic causal modelling of steady-state responses revealed that the most likely direction of this fronto-occipital connectivity was from frontal to occipital sources. These results point to a role of ongoing alpha oscillations in the inferential process that gives rise to the perception of AM and suggest that fronto-occipital interactions bias perception towards internally generated predictions.

Prior probability and feature predictability interactively bias perceptual decisions

Anticipating a forthcoming sensory experience facilitates perception for expected stimuli but also hinders perception for less likely alternatives. Recent neuroimaging studies suggest that expectation biases arise from feature-level predictions that enhance early sensory representations and facilitate evidence accumulation for contextually probable stimuli while suppressing alternatives. Reasonably then, the extent to which prior knowledge biases subsequent sensory processing should depend on the precision of expectations at the feature level as well as the degree to which expected features match those of an observed stimulus. In the present study we investigated how these two sources of uncertainty modulated pre- and post-stimulus bias mechanisms in the drift-diffusion model during a probabilistic face/house discrimination task. We tested several plausible models of choice bias, concluding that predictive cues led to a bias in both the starting-point and rate of evidence accumulation favoring the more probable stimulus category. We further tested the hypotheses that prior bias in the starting-point was conditional on the feature-level uncertainty of category expectations and that dynamic bias in the drift-rate was modulated by the match between expected and observed stimulus features. Starting-point estimates suggested that subjects formed a constant prior bias in favor of the face category, which exhibits less feature-level variability, that was strengthened or weakened by trial-wise predictive cues. Furthermore, we found that the gain on face/house evidence was increased for stimuli with less ambiguous features and that this relationship was enhanced by valid category expectations. These findings offer new evidence that bridges psychological models of decision-making with recent predictive coding theories of perception.

Prior Expectations Evoke Stimulus Templates in the Primary Visual Cortex

Sensory processing is strongly influenced by prior expectations. Valid expectations have been shown to lead to improvements in perception as well as in the quality of sensory representations in primary visual cortex. However, very little is known about the neural correlates of the expectations themselves. Previous studies have demonstrated increased activity in sensory cortex following the omission of an expected stimulus, yet it is unclear whether this increased activity constitutes a general surprise signal or rather has representational content. One intriguing possibility is that top–down expectation leads to the formation of a template of the expected stimulus in visual cortex, which can then be compared with subsequent bottom–up input. To test this hypothesis, we used fMRI to noninvasively measure neural activity patterns in early visual cortex of human participants during expected but omitted visual stimuli. Our results show that prior expectation of a specific visual stimulus evokes a feature-specific pattern of activity in the primary visual cortex (V1) similar to that evoked by the corresponding actual stimulus. These results are in line with the notion that prior expectation triggers the formation of specific stimulus templates to efficiently process expected sensory inputs.

Surprise beyond prediction error

Surprise drives learning. Various neural “prediction error” signals are believed to underpin surprise‐based reinforcement learning. Here, we report a surprise signal that reflects reinforcement learning but is neither un/signed reward prediction error (RPE) nor un/signed state prediction error (SPE). To exclude these alternatives, we measured surprise responses in the absence of RPE and accounted for a host of potential SPE confounds. This new surprise signal was evident in ventral striatum, primary sensory cortex, frontal poles, and amygdala. We interpret these findings via a normative model of surprise. Hum Brain Mapp 35:4805–4814, 2014. © 2014 The Authors. Human Brain Mapping Published by Wiley Periodicals, Inc.

Effects of Anticipation on Perception of Facial Expressions

Human beings do not passively perceive the facial expressions of other people, but predict observed facial expressions by employing past experiences. The aim of the current study was to investigate whether and how anticipation affected the perception of facial expressions. A 3-way repeated-measures ANOVA on anticipation, orientation, and facial expression was performed on RTs and recognition accuracy in Experiments 1 and 2. The results showed that anticipation reduced susceptibility to negative facial expressions. In this regard, anticipation might be considered as an effective emotion-regulation strategy. In addition, a decreased inversion effect for positive facial expressions was found in the predictable condition, which might reflect a switch from feature-based to holistic processing.

Short-term memory affects color perception in context

Color-based object selection - for instance, looking for ripe tomatoes in the market - places demands on both perceptual and memory processes: it is necessary to form a stable perceptual estimate of surface color from a variable visual signal, as well as to retain multiple perceptual estimates in memory while comparing objects. Nevertheless, perceptual and memory processes in the color domain are generally studied in separate research programs with the assumption that they are independent. Here, we demonstrate a strong failure of independence between color perception and memory: the effect of context on color appearance is substantially weakened by a short retention interval between a reference and test stimulus. This somewhat counterintuitive result is consistent with Bayesian estimation: as the precision of the representation of the reference surface and its context decays in memory, prior information gains more weight, causing the retained percepts to be drawn toward prior information about surface and context color. This interaction implies that to fully understand information processing in real-world color tasks, perception and memory need to be considered jointly.

Attention sharpens the distinction between expected and unexpected percepts in the visual brain

Attention, the prioritization of goal-relevant stimuli, and expectation, the modulation of stimulus processing by probabilistic context, represent the two main endogenous determinants of visual cognition. Neural selectivity in visual cortex is enhanced for both attended and expected stimuli, but the functional relationship between these mechanisms is poorly understood. Here, we adjudicated between two current hypotheses of how attention relates to predictive processing, namely, that attention either enhances or filters out perceptual prediction errors (PEs), the PE-promotion model versus the PE-suppression model. We acquired fMRI data from category-selective visual regions while human subjects viewed expected and unexpected stimuli that were either attended or unattended. Then, we trained multivariate neural pattern classifiers to discriminate expected from unexpected stimuli, depending on whether these stimuli had been attended or unattended. If attention promotes PEs, then this should increase the disparity of neural patterns associated with expected and unexpected stimuli, thus enhancing the classifier's ability to distinguish between the two. In contrast, if attention suppresses PEs, then this should reduce the disparity between neural signals for expected and unexpected percepts, thus impairing classifier performance. We demonstrate that attention greatly enhances a neural pattern classifier's ability to discriminate between expected and unexpected stimuli in a region- and stimulus category-specific fashion. These findings are incompatible with the PE-suppression model, but they strongly support the PE-promotion model, whereby attention increases the precision of prediction errors. Our results clarify the relationship between attention and expectation, casting attention as a mechanism for accelerating online error correction in predicting task-relevant visual inputs.

Dissociation of Prediction from Conscious Perception

The framework of predictive coding offers a parsimonious explanation for many perceptual phenomena. According to this framework, perception of the outer world is created by the comparison of incoming sensory information with an internal predictive model based on previous experience and context. However, it is unclear whether the predicted percept needs to enter conscious awareness for the internal predictive model to be effective. Here we used an apparent motion paradigm to show that while prediction and conscious awareness of a predicted percept may coincide, a dissociation can be observed. When sensory information provides reliable input for the internal predictive model, the predicted percept does not have to be consciously perceived for successful prediction. However, when sensory input is ambiguous, conscious awareness helps the prediction to take effect.

Prior expectations bias sensory representations in visual cortex

Perception is strongly influenced by expectations. Accordingly, perception has sometimes been cast as a process of inference, whereby sensory inputs are combined with prior knowledge. However, despite a wealth of behavioral literature supporting an account of perception as probabilistic inference, the neural mechanisms underlying this process remain largely unknown. One important question is whether top-down expectation biases stimulus representations in early sensory cortex, i.e., whether the integration of prior knowledge and bottom-up inputs is already observable at the earliest levels of sensory processing. Alternatively, early sensory processing may be unaffected by top-down expectations, and integration of prior knowledge and bottom-up input may take place in downstream association areas that are proposed to be involved in perceptual decision-making. Here, we implicitly manipulated human subjects' prior expectations about visual motion stimuli, and probed the effects on both perception and sensory representations in visual cortex. To this end, we measured neural activity noninvasively using functional magnetic resonance imaging, and applied a forward modeling approach to reconstruct the motion direction of the perceived stimuli from the signal in visual cortex. Our results show that top-down expectations bias representations in visual cortex, demonstrating that the integration of prior information and sensory input is reflected at the earliest stages of sensory processing.

Frontoparietal cortex mediates perceptual transitions in bistable perception

During bistable vision, perception oscillates between two mutually exclusive percepts despite constant sensory input. Greater BOLD responses in frontoparietal cortex have been shown to be associated with endogenous perceptual transitions compared with "replay" transitions designed to closely match bistability in both perceptual quality and timing. It has remained controversial, however, whether this enhanced activity reflects causal influences of these regions on processing at the sensory level or, alternatively, an effect of stimulus differences that result in, for example, longer durations of perceptual transitions in bistable perception compared with replay conditions. Using a rotating Lissajous figure in an fMRI experiment on 15 human participants, we controlled for potential confounds of differences in transition duration and confirmed previous findings of greater activity in frontoparietal areas for transitions during bistable perception. In addition, we applied dynamic causal modeling to identify the neural model that best explains the observed BOLD signals in terms of effective connectivity. We found that enhanced activity for perceptual transitions is associated with a modulation of top-down connectivity from frontal to visual cortex, thus arguing for a crucial role of frontoparietal cortex in perceptual transitions during bistable perception.

The dynamics of contour integration: A simultaneous EEG-fMRI study

To study the dynamics of contour integration in the human brain, we simultaneously acquired EEG and fMRI data while participants were engaged in a passive viewing task. The stimuli were Gabor arrays with some Gabor elements positioned on the contour of an embedded shape, in three conditions: with local and global structure (perfect contour alignment), with global structure only (orthogonal orientations interrupting the alignment), or without contour. By applying JointICA to the EEG and fMRI responses of the subjects, new insights could be obtained that cannot be derived from unimodal recordings. In particular, only in the global structure condition, an ERP peak around 300ms was identified that involved a loop from LOC to the early visual areas. This component can be interpreted as being related to the verification of the consistency of the different local elements with the globally defined shape, which is necessary when perfect local-to-global alignment is absent. By modifying JointICA, a quantitative comparison of brain regions and the time-course of their interplay were obtained between different conditions. More generally, we provide additional support for the presence of feedback loops from higher areas to lower level sensory regions.

Theory of mind: a neural prediction problem

Predictive coding posits that neural systems make forward-looking predictions about incoming information. Neural signals contain information not about the currently perceived stimulus, but about the difference between the observed and the predicted stimulus. We propose to extend the predictive coding framework from high-level sensory processing to the more abstract domain of theory of mind; that is, to inferences about others' goals, thoughts, and personalities. We review evidence that, across brain regions, neural responses to depictions of human behavior, from biological motion to trait descriptions, exhibit a key signature of predictive coding: reduced activity to predictable stimuli. We discuss how future experiments could distinguish predictive coding from alternative explanations of this response profile. This framework may provide an important new window on the neural computations underlying theory of mind.

TMS over V5 disrupts motion prediction

Given the vast amount of sensory information the brain has to deal with, predicting some of this information based on the current context is a resource-efficient strategy. The framework of predictive coding states that higher-level brain areas generate a predictive model to be communicated via feedback connections to early sensory areas. Here, we directly tested the necessity of a higher-level visual area, V5, in this predictive processing in the context of an apparent motion paradigm. We flashed targets on the apparent motion trace in-time or out-of-time with the predicted illusory motion token. As in previous studies, we found that predictable in-time targets were better detected than unpredictable out-of-time targets. However, when we applied functional magnetic resonance imaging-guided, double-pulse transcranial magnetic stimulation (TMS) over left V5 at 13–53 ms before target onset, the detection advantage of in-time targets was eliminated; this was not the case when TMS was applied over the vertex. Our results are causal evidence that V5 is necessary for a prediction effect, which has been shown to modulate V1 activity (Alink et al. 2010). Thus, our findings suggest that information processing between V5 and V1 is crucial for visual motion prediction, providing experimental support for the predictive coding framework.

Predicting language: MEG evidence for lexical preactivation

It is widely assumed that prediction plays a substantial role in language processing. However, despite numerous studies demonstrating that contextual information facilitates both syntactic and lexical-semantic processing, there exists no direct evidence pertaining to the neural correlates of the prediction process itself. Using magnetoencephalography (MEG), this study found that brain activity was modulated by whether or not a specific noun could be predicted, given a picture prime. Specifically, before the noun was presented, predictive contexts triggered enhanced activation in left mid-temporal cortex (implicated in lexical access), ventro-medial prefrontal cortex (previously associated with top-down processing), and visual cortex (hypothesized to index the preactivation of predicted form features), successively. This finding suggests that predictive language processing recruits a top-down network where predicted words are activated at different levels of representation, from more 'abstract' lexical-semantic representations in temporal cortex, all the way down to visual word form features. The same brain regions that exhibited enhanced activation for predictive contexts before the onset of the noun showed effects of congruence during the target word. To our knowledge, this study is one of the first to directly investigate the anticipatory stage of predictive language processing.

Perceptual distortions in pitch and time reveal active prediction and support for an auditory pitch-motion hypothesis

A number of accounts of human auditory perception assume that listeners use prior stimulus context to generate predictions about future stimulation. Here, we tested an auditory pitch-motion hypothesis that was developed from this perspective. Listeners judged either the time change (i.e., duration) or pitch change of a comparison frequency glide relative to a standard (referent) glide. Under a constant-velocity assumption, listeners were hypothesized to use the pitch velocity (Δf/Δt) of the standard glide to generate predictions about the pitch velocity of the comparison glide, leading to perceptual distortions along the to-be-judged dimension when the velocities of the two glides differed. These predictions were borne out in the pattern of relative points of subjective equality by a significant three-way interaction between the velocities of the two glides and task. In general, listeners' judgments along the task-relevant dimension (pitch or time) were affected by expectations generated by the constant-velocity standard, but in an opposite manner for the two stimulus dimensions. When the comparison glide velocity was faster than the standard, listeners overestimated time change, but underestimated pitch change, whereas when the comparison glide velocity was slower than the standard, listeners underestimated time change, but overestimated pitch change. Perceptual distortions were least evident when the velocities of the standard and comparison glides were matched. Fits of an imputed velocity model further revealed increasingly larger distortions at faster velocities. The present findings provide support for the auditory pitch-motion hypothesis and add to a larger body of work revealing a role for active prediction in human auditory perception.

The effects of alerting signals in masked priming

Alerting signals often serve to reduce temporal uncertainty by predicting the time of stimulus onset. The resulting response time benefits have often been explained by facilitated translation of stimulus codes into response codes on the basis of established stimulus-response (S-R) links. In paradigms of masked S-R priming alerting signals also modulate response activation processes triggered by subliminally presented prime stimuli. In the present study we tested whether facilitation of visuo-motor translation processes due to alerting signals critically depends on established S-R links. Alerting signals resulted in significantly enhanced masked priming effects for masked prime stimuli that included and that did not include established S-R links (i.e., target vs. novel primes). Yet, the alerting-priming interaction was more pronounced for target than for novel primes. These results suggest that effects of alerting signals on masked priming are especially evident when S-R links between prime and target exist. At the same time, an alerting-priming interaction also for novel primes suggests that alerting signals also facilitate stimulus-response translation processes when masked prime stimuli provide action-trigger conditions in terms of programmed S-R links.