Face Repetition Probability Does Not Affect Repetition Suppression in Macaque Inferotemporal Cortex

Recent Visual Experience Reshapes V4 Neuronal Activity and Improves Perceptual Performance

Recent visual experience heavily influences our visual perception, but how neuronal activity is reshaped to alter and improve perceptual discrimination remains unknown. We recorded from populations of neurons in visual cortical area V4 while two male rhesus macaque monkeys performed a natural image change detection task under different experience conditions. We found that maximizing the recent experience with a particular image led to an improvement in the ability to detect a change in that image. This improvement was associated with decreased neural responses to the image, consistent with neuronal changes previously seen in studies of adaptation and expectation. We found that the magnitude of behavioral improvement was correlated with the magnitude of response suppression. Furthermore, this suppression of activity led to an increase in signal separation, providing evidence that a reduction in activity can improve stimulus encoding. Within populations of neurons, greater recent experience was associated with decreased trial-to-trial shared variability, indicating that a reduction in variability is a key means by which experience influences perception. Taken together, the results of our study contribute to an understanding of how recent visual experience can shape our perception and behavior through modulating activity patterns in the mid-level visual cortex.

Adaptation in the visual system: Networked fatigue or suppressed prediction error signalling?

Our brains are constantly adapting to changes in our visual environments. Neural adaptation exerts a persistent influence on the activity of sensory neurons and our perceptual experience, however there is a lack of consensus regarding how adaptation is implemented in the visual system. One account describes fatigue-based mechanisms embedded within local networks of stimulus-selective neurons (networked fatigue models). Another depicts adaptation as a product of stimulus expectations (predictive coding models). In this review, I evaluate neuroimaging and psychophysical evidence that poses fundamental problems for predictive coding models of neural adaptation. Specifically, I discuss observations of distinct repetition and expectation effects, as well as incorrect predictions of repulsive adaptation aftereffects made by predictive coding accounts. Based on this evidence, I argue that networked fatigue models provide a more parsimonious account of adaptation effects in the visual system. Although stimulus expectations can be formed based on recent stimulation history, any consequences of these expectations are likely to co-occur (or interact) with effects of fatigue-based adaptation. I conclude by proposing novel, testable hypotheses relating to interactions between fatigue-based adaptation and other predictive processes, focusing on stimulus feature extrapolation phenomena.

Adaptation of the inferior temporal neurons and efficient visual processing

Numerous studies examining the responses of individual neurons in the inferior temporal (IT) cortex have revealed their characteristics such as two-dimensional or three-dimensional shape tuning, objects, or category selectivity. While these basic selectivities have been studied assuming that their response to stimuli is relatively stable, physiological experiments have revealed that the responsiveness of IT neurons also depends on visual experience. The activity changes of IT neurons occur over various time ranges; among these, repetition suppression (RS), in particular, is robustly observed in IT neurons without any behavioral or task constraints. I observed a similar phenomenon in the ventral visual neurons in macaque monkeys while they engaged in free viewing and actively fixated on one consistent object multiple times. This observation indicates that the phenomenon also occurs in natural situations during which the subject actively views stimuli without forced fixation, suggesting that this phenomenon is an everyday occurrence and widespread across regions of the visual system, making it a default process for visual neurons. Such short-term activity modulation may be a key to understanding the visual system; however, the circuit mechanism and the biological significance of RS remain unclear. Thus, in this review, I summarize the observed modulation types in IT neurons and the known properties of RS. Subsequently, I discuss adaptation in vision, including concepts such as efficient and predictive coding, as well as the relationship between adaptation and psychophysical aftereffects. Finally, I discuss some conceptual implications of this phenomenon as well as the circuit mechanisms and the models that may explain adaptation as a fundamental aspect of visual processing.

Recent visual experience reshapes V4 neuronal activity and improves perceptual performance

Recent visual experience heavily influences our visual perception, but how this is mediated by the reshaping of neuronal activity to alter and improve perceptual discrimination remains unknown. We recorded from populations of neurons in visual cortical area V4 while monkeys performed a natural image change detection task under different experience conditions. We found that maximizing the recent experience with a particular image led to an improvement in the ability to detect a change in that image. This improvement was associated with decreased neural responses to the image, consistent with neuronal changes previously seen in studies of adaptation and expectation. We found that the magnitude of behavioral improvement was correlated with the magnitude of response suppression. Furthermore, this suppression of activity led to an increase in signal separation, providing evidence that a reduction in activity can improve stimulus encoding. Within populations of neurons, greater recent experience was associated with decreased trial-to-trial shared variability, indicating that a reduction in variability is a key means by which experience influences perception. Taken together, the results of our study contribute to an understanding of how recent visual experience can shape our perception and behavior through modulating activity patterns in mid-level visual cortex.

Expectation Modulates Repetition Suppression at Late But Not Early Stages during Visual Word Recognition: Evidence from Event-related Potentials

Visual word recognition is commonly rapid and efficient, incorporating top-down predictive processing mechanisms. Neuroimaging studies with face stimuli suggest that repetition suppression (RS) reflects predictive processing at the neural level, as this effect is larger when repetitions are more frequent, that is, more expected. It remains unclear, however, at the temporal level whether and how RS and its modulation by expectation occur in visual word recognition. To address this gap, the present study aimed to investigate the presence and time course of these effects during visual word recognition using EEG. Thirty-six native Cantonese speakers were presented with pairs of Chinese written words and performed a nonlinguistic oddball task. The second word of a pair was either a repetition of the first or a different word (alternation). In repetition blocks, 75% of trials were repetitions and 25% were alternations, whereas the reverse was true in alternation blocks. Topographic analysis of variance of EEG at each time point showed robust RS effects in three time windows (141-227 msec, 242-445 msec, and 467-513 msec) reflecting facilitation of visual word recognition. Importantly, the modulation of RS by expectation was observed at the late rather than early intervals (334-387 msec, 465-550 msec, and 559-632 msec) and more than 100 msec after the first RS effects. In the predictive coding view of RS, only late repetition effects are modulated by expectation, whereas early RS effects may be mediated by lower-level predictions. Taken together, our findings provide the first EEG evidence revealing distinct temporal dynamics of RS effects and repetition probability on RS effects in visual processing of Chinese words.

Prediction error processing and sharpening of expected information across the face-processing hierarchy

The perception and neural processing of sensory information are strongly influenced by prior expectations. The integration of prior and sensory information can manifest through distinct underlying mechanisms: focusing on unexpected input, denoted as prediction error (PE) processing, or amplifying anticipated information via sharpened representation. In this study, we employed computational modeling using deep neural networks combined with representational similarity analyses of fMRI data to investigate these two processes during face perception. Participants were cued to see face images, some generated by morphing two faces, leading to ambiguity in face identity. We show that expected faces were identified faster and perception of ambiguous faces was shifted towards priors. Multivariate analyses uncovered evidence for PE processing across and beyond the face-processing hierarchy from the occipital face area (OFA), via the fusiform face area, to the anterior temporal lobe, and suggest sharpened representations in the OFA. Our findings support the proposition that the brain represents faces grounded in prior expectations.

Computational components of visual predictive coding circuitry

If a full visual percept can be said to be a 'hypothesis', so too can a neural 'prediction' - although the latter addresses one particular component of image content (such as 3-dimensional organisation, the interplay between lighting and surface colour, the future trajectory of moving objects, and so on). And, because processing is hierarchical, predictions generated at one level are conveyed in a backward direction to a lower level, seeking to predict, in fact, the neural activity at that prior stage of processing, and learning from errors signalled in the opposite direction. This is the essence of 'predictive coding', at once an algorithm for information processing and a theoretical basis for the nature of operations performed by the cerebral cortex. Neural models for the implementation of predictive coding invoke specific functional classes of neuron for generating, transmitting and receiving predictions, and for producing reciprocal error signals. Also a third general class, 'precision' neurons, tasked with regulating the magnitude of error signals contingent upon the confidence placed upon the prediction, i.e., the reliability and behavioural utility of the sensory data that it predicts. So, what is the ultimate source of a 'prediction'? The answer is multifactorial: knowledge of the current environmental context and the immediate past, allied to memory and lifetime experience of the way of the world, doubtless fine-tuned by evolutionary history too. There are, in consequence, numerous potential avenues for experimenters seeking to manipulate subjects' expectation, and examine the neural signals elicited by surprising, and less surprising visual stimuli. This review focuses upon the predictive physiology of mouse and monkey visual cortex, summarising and commenting on evidence to date, and placing it in the context of the broader field. It is concluded that predictive coding has a firm grounding in basic neuroscience and that, unsurprisingly, there remains much to learn.

Modeling face recognition in the predictive coding framework: A combined computational modeling and functional imaging study

The learning of new facial identities and the recognition of familiar faces are crucial processes for social interactions. Recently, a combined computational modeling and functional magnetic resonance imaging (fMRI) study used predictive coding as a biologically plausible framework to model face identity learning and to relate specific model parameters with brain activity (Apps and Tsakiris, Nat Commun 4, 2698, 2013). On the one hand, it was shown that behavioral responses on a two-option face recognition task could be predicted by the level of contextual and facial familiarity in a computational model derived from predictive-coding principles. On the other hand, brain activity in specific brain regions was associated with these parameters. More specifically, brain activity in the superior temporal sulcus (STS) varied with contextual familiarity, whereas activity in the fusiform face area (FFA) covaried with the prediction error parameter that updated facial familiarity. Literature combining fMRI assessments and computational modeling in humans still needs to be expanded. Furthermore, prior results are largely not replicated. The present study was, therefore, specifically set up to replicate these previous findings. Our results support the original findings in two critical aspects. First, on a group level, the behavioral responses were modeled best by the same computational model reported by the original authors. Second, we showed that estimates of these model parameters covary with brain activity in specific, face-sensitive brain regions. Our results thus provide further evidence that the functional properties of the face perception network conform to central principles of predictive coding. However, our study yielded diverging findings on specific computational model parameters reflected in brain activity. On the one hand, we did not find any evidence of a computational involvement of the STS. On the other hand, our results showed that activity in the right FFA was associated with multiple computational model parameters. Our data do not provide evidence for functional segregation between particular face-sensitive brain regions, as previously proposed.

Stimulus expectations do not modulate visual event-related potentials in probabilistic cueing designs

Humans and other animals can learn and exploit repeating patterns that occur within their environments. These learned patterns can be used to form expectations about future sensory events. Several influential predictive coding models have been proposed to explain how learned expectations influence the activity of stimulus-selective neurons in the visual system. These models specify reductions in neural response measures when expectations are fulfilled (termed expectation suppression) and increases following surprising sensory events. However, there is currently scant evidence for expectation suppression in the visual system when confounding factors are taken into account. Effects of surprise have been observed in blood oxygen level dependent (BOLD) signals, but not when using electrophysiological measures. To provide a strong test for expectation suppression and surprise effects we performed a predictive cueing experiment while recording electroencephalographic (EEG) data. Participants (n=48) learned cue-face associations during a training session and were then exposed to these cue-face pairs in a subsequent experiment. Using univariate analyses of face-evoked event-related potentials (ERPs) we did not observe any differences across expected (90% probability), neutral (50%) and surprising (10%) face conditions. Across these comparisons, Bayes factors consistently favoured the null hypothesis throughout the time-course of the stimulus-evoked response. When using multivariate pattern analysis we did not observe above-chance classification of expected and surprising face-evoked ERPs. By contrast, we found robust within- and across-trial stimulus repetition effects. Our findings do not support predictive coding-based accounts that specify reduced prediction error signalling when perceptual expectations are fulfilled. They instead highlight the utility of other types of predictive processing models that describe expectation-related phenomena in the visual system without recourse to prediction error signalling.

Temporal continuity shapes visual responses of macaque face patch neurons

Macaque inferior temporal cortex neurons respond selectively to complex visual images, with recent work showing that they are also entrained reliably by the evolving content of natural movies. To what extent does temporal continuity itself shape the responses of high-level visual neurons? We addressed this question by measuring how cells in face-selective regions of the macaque visual cortex were affected by the manipulation of a movie's temporal structure. Sampling a 5-min movie at 1 s intervals, we measured neural responses to randomized, brief stimuli of different lengths, ranging from 800 ms dynamic movie snippets to 100 ms static frames. We found that the disruption of temporal continuity strongly altered neural response profiles, particularly in the early response period after stimulus onset. The results suggest that models of visual system function based on discrete and randomized visual presentations may not translate well to the brain's natural modes of operation.

Inferior temporal cortex leads prefrontal cortex in response to a violation of a learned sequence

Primates learn statistical regularities that are embedded in visual sequences, a form of statistical learning. Single-unit recordings in macaques showed that inferior temporal (IT) neurons are sensitive to statistical regularities in visual sequences. Here, we asked whether ventrolateral prefrontal cortex (VLPFC), which is connected to IT, is also sensitive to the transition probabilities in visual sequences and whether the statistical learning signal in IT originates in VLPFC. We recorded simultaneously multiunit activity (MUA) and local field potentials (LFPs) in IT and VLPFC after monkeys were exposed to triplets of images with a fixed presentation order. In both areas, the MUA was stronger to images that violated the learned sequence (deviants) compared to the same images presented in the learned triplets. The high-gamma and beta LFP power showed an enhanced and suppressed response, respectively, to the deviants in both areas. The enhanced response was present also for the image following the deviant, suggesting a sensitivity for temporal adjacent dependencies in IT and VLPFC. The increased response to the deviant occurred later in VLPFC than in IT, suggesting that the deviant response in IT was not inherited from VLPFC. These data support predictive coding theories that propose a feedforward flow of prediction errors.

Independent repetition suppression in macaque area V2 and inferotemporal cortex

When a complexly structured natural image is presented twice in succession, first as adapter and then as test, neurons in area TE of macaque inferotemporal cortex exhibit repetition suppression, responding less strongly to the second presentation than to the first. This phenomenon, which has been studied primarily in TE, might plausibly be argued to arise in TE because TE neurons respond selectively to complex images and thus carry information adequate for determining whether an image is or is not a repeat. However, the idea has never been put to a direct test. To resolve this issue, we monitored neuronal responses to sequences of complex natural images under identical conditions in areas V2 and TE. We found that repetition suppression occurs in both areas. Moreover, in each area, suppression takes the form of a dynamic alteration whereby the initial peak of excitation is followed by a trough and then a rebound of firing rate. To assess whether repetition suppression in either area is transmitted from the other area, we analyzed the timing of the phenomenon and its degree of spatial generalization. Suppression occurs at shorter latency in V2 than in TE. Therefore it is not simply fed back from TE. Suppression occurs in TE but not in V2 under conditions in which the test and adapter are presented in different visual field quadrants. Therefore it is not simply fed forward from V2. We conclude that repetition suppression occurs independently in V2 and TE.NEW & NOTEWORTHY When a complexly structured natural image is presented twice in rapid succession, neurons in inferotemporal area TE exhibit repetition suppression, responding less strongly to the second than to the first presentation. We have explored whether this phenomenon is confined to high-order areas where neurons respond selectively to such images and thus carry information relevant to recognizing a repeat. We have found surprisingly that repetition suppression occurs even in low-order visual area V2.

Contribution of Individual Features to Repetition Suppression in Macaque Inferotemporal Cortex

When an image is presented twice in succession, neurons in area TE of macaque inferotemporal cortex exhibit repetition suppression, responding less strongly to the second presentation than to the first. Suppression is known to occur if the adapter and the test image are subtly different from each other. However, it is not known whether cross-suppression occurs between images that are radically different from each other but that share a subset of features. To explore this issue, we measured repetition suppression using colored shapes. On interleaved trials, the test image might be identical to the adapter, might share its shape or color alone or might differ from it totally. At the level of the neuronal population as a whole, suppression was especially deep when adapter and test were identical, intermediate when they shared only one attribute and minimal when they shared neither attribute. At the level of the individual neuron, the degree of suppression depended not only on the properties of the two images but also on the preferences of the neuron. Suppression was deeper when the repeated color or shape was preferred by the neuron than when it was not. This effect might arise from feature-specific adaptation or alternatively from adapter-induced fatigue. Both mechanisms conform to the principle that the degree of suppression is determined by the preferences of the neuron.

Remembering the Past to See the Future

In addition to the role that our visual system plays in determining what we are seeing right now, visual computations contribute in important ways to predicting what we will see next. While the role of memory in creating future predictions is often overlooked, efficient predictive computation requires the use of information about the past to estimate future events. In this article, we introduce a framework for understanding the relationship between memory and visual prediction and review the two classes of mechanisms that the visual system relies on to create future predictions. We also discuss the principles that define the mapping from predictive computations to predictive mechanisms and how downstream brain areas interpret the predictive signals computed by the visual system. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Oscillatory signatures of Repetition Suppression and Novelty Detection reveal altered induced visual responses in early deafness

The ability to differentiate between repeated and novel events represents a fundamental property of the visual system. Neural responses are typically reduced upon stimulus repetition, a phenomenon called Repetition Suppression (RS). On the contrary, following a novel visual stimulus, the neural response is generally enhanced, a phenomenon referred to as Novelty Detection (ND). Here, we aimed to investigate the impact of early deafness on the oscillatory signatures of RS and ND brain responses. To this aim, electrophysiological data were acquired in early deaf and hearing control individuals during processing of repeated and novel visual events unattended by participants. By studying evoked and induced oscillatory brain activities, as well as inter-trial phase coherence, we linked response modulations to feedback and/or feedforward processes. Results revealed selective experience-dependent changes on both RS and ND mechanisms. Compared to hearing controls, early deaf individuals displayed: (i) greater attenuation of the response following stimulus repetition, selectively in the induced theta-band (4-7 Hz); (ii) reduced desynchronization following the onset of novel visual stimuli, in the induced alpha and beta bands (8-12 and 13-25 Hz); (iii) comparable modulation of evoked responses and inter-trial phase coherence. The selectivity of the effects in the induced responses parallels findings observed in the auditory cortex of deaf animal models following intracochlear electric stimulation. The present results support the idea that early deafness alters induced oscillatory activity and the functional tuning of basic visual processing.

Moving a Shape behind a Slit: Partial Shape Representations in Inferior Temporal Cortex

Current models of object recognition are based on spatial representations build from object features that are simultaneously present in the retinal image. However, one can recognize an object when it moves behind a static occlude, and only a small fragment of its shape is visible through a slit at a given moment in time. Such anorthoscopic perception requires spatiotemporal integration of the successively presented shape parts during slit-viewing. Human fMRI studies suggested that ventral visual stream areas represent whole shapes formed through temporal integration during anorthoscopic perception. To examine the time course of shape-selective responses during slit-viewing, we recorded the responses of single inferior temporal (IT) neurons of rhesus monkeys to moving shapes that were only partially visible through a static narrow slit. The IT neurons signaled shape identity by their response when that was cumulated across the duration of the shape presentation. Their shape preference during slit-viewing equaled that for static, whole-shape presentations. However, when analyzing their responses at a finer time scale, we showed that the IT neurons responded to particular shape fragments that were revealed by the slit. We found no evidence for temporal integration of slit-views that result in a whole-shape representation, even when the monkey was matching slit-views of a shape to static whole-shape presentations. These data suggest that, although the temporally integrated response of macaque IT neurons can signal shape identity in slit-viewing conditions, the spatiotemporal integration needed for the formation of a whole-shape percept occurs in other areas, perhaps downstream to IT.SIGNIFICANCE STATEMENT One recognizes an object when it moves behind a static occluder and only a small fragment of its shape is visible through a static slit at a given moment in time. Such anorthoscopic perception requires spatiotemporal integration of the successively presented partial shape parts. Human fMRI studies suggested that ventral visual stream areas represent shapes formed through temporal integration. We recorded the responses of inferior temporal (IT) cortical neurons of macaques during slit-viewing conditions. Although the temporally summated response of macaque IT neurons could signal shape identity under slit-viewing conditions, we found no evidence for a whole-shape representation using analyses at a finer time scale. Thus, the spatiotemporal integration needed for anorthoscopic perception does not occur within IT.

Long term fMRI adaptation depends on adapter response in face-selective cortex

Repetition suppression (RS) reflects a neural attenuation during repeated stimulation. We used fMRI and the subsequent memory paradigm to test the predictive coding hypothesis for RS during visual memory processing by investigating the interaction between RS and differences due to memory in category-selective cortex (FFA, pSTS, PPA, and RSC). Fifty-six participants encoded face and house stimuli twice, followed by an immediate and delayed (48 h) recognition memory assessment. Linear Mixed Model analyses with repetition, subsequent recognition performance, and their interaction as fixed effects revealed that absolute RS during encoding interacts with probability of future remembrance in face-selective cortex. This effect was not observed for relative RS, i.e. when controlled for adapter-response. The findings also reveal an association between adapter response and RS, both for short and long term (48h) intervals, after controlling for the mathematical dependence between both measures. These combined findings are challenging for predictive coding models of visual memory and are more compatible with adapter-related and familiarity accounts.

Pinpointing the neural signatures of single-exposure visual recognition memory

Memories of the images that we have seen are thought to be reflected in the reduction of neural responses in high-level visual areas such as inferotemporal (IT) cortex, a phenomenon known as repetition suppression (RS). We challenged this hypothesis with a task that required rhesus monkeys to report whether images were novel or repeated while ignoring variations in contrast, a stimulus attribute that is also known to modulate the overall IT response. The monkeys' behavior was largely contrast invariant, contrary to the predictions of an RS-inspired decoder, which could not distinguish responses to images that are repeated from those that are of lower contrast. However, the monkeys' behavioral patterns were well predicted by a linearly decodable variant in which the total spike count was corrected for contrast modulation. These results suggest that the IT neural activity pattern that best aligns with single-exposure visual recognition memory behavior is not RS but rather sensory referenced suppression: reductions in IT population response magnitude, corrected for sensory modulation.

Evaluating the evidence for expectation suppression in the visual system

Reports of expectation suppression have shaped the development of influential predictive coding-based theories of visual perception. However recent work has highlighted confounding factors that may mimic or inflate expectation suppression effects. In this review, we describe four confounds that are prevalent across experiments that tested for expectation suppression: effects of surprise, attention, stimulus repetition and adaptation, and stimulus novelty. With these confounds in mind we then critically review the evidence for expectation suppression across probabilistic cueing, statistical learning, oddball, action-outcome learning and apparent motion designs. We found evidence for expectation suppression within a specific subset of statistical learning designs that involved weeks of sequence learning prior to neural activity measurement. Across other experimental contexts, whereby stimulus appearance probabilities were learned within one or two testing sessions, there was inconsistent evidence for genuine expectation suppression. We discuss how an absence of expectation suppression could inform models of predictive processing, repetition suppression and perceptual decision-making. We also provide suggestions for designing experiments that may better test for expectation suppression in future work.

Examining cross-modal fMRI adaptation for observed and executed actions in the monkey brain

While mirror neurons have been found in several monkey brain regions, their existence in the human brain is still largely inferred from indirect non-invasive measurements like functional MRI. It has been proposed that, beyond showing overlapping brain responses during action observation and execution tasks, candidate mirror neuron regions should demonstrate cross-modal action specificity, in line with a defining physiological characteristic of these neurons in the monkey brain. Although cross-modal fMRI adaptation has been put forward as a suited method to test this key feature of cross-modal action specificity in the human brain, so far, the overall usefulness of this technique to demonstrate mirror neuron activity remains unclear. To date, it has never been tested to what extent monkey brain regions known to house mirror neurons, would yield uni- and/or cross-modal fMRI adaptation effects. We therefore performed an fMRI adaptation experiment while male rhesus macaques either performed or observed two different goal-directed hand actions. Executing grasp/lift or touch/press actions in the dark, as well as observing videos of these monkey hand actions, yielded robust responses throughout the brain, including overlapping fMRI responses in parietal and premotor mirror neuron regions. Uni-modal adaptation effects were mostly restricted to the visual modality and the early visual cortices. Both frequentist and Bayesian statistical analyses however suggested no evidence for cross-modal fMRI adaptation effects in monkey parietal and premotor mirror neuron regions. Overall, these findings suggest monkey mirror neuron activity does not readily translate into cross-modal repetition suppression effects that can be detected by fMRI.

Limited Evidence for Sensory Prediction Error Responses in Visual Cortex of Macaques and Humans

A recent formulation of predictive coding theory proposes that a subset of neurons in each cortical area encodes sensory prediction errors, the difference between predictions relayed from higher cortex and the sensory input. Here, we test for evidence of prediction error responses in spiking responses and local field potentials (LFP) recorded in primary visual cortex and area V4 of macaque monkeys, and in complementary electroencephalographic (EEG) scalp recordings in human participants. We presented a fixed sequence of visual stimuli on most trials, and violated the expected ordering on a small subset of trials. Under predictive coding theory, pattern-violating stimuli should trigger robust prediction errors, but we found that spiking, LFP and EEG responses to expected and pattern-violating stimuli were nearly identical. Our results challenge the assertion that a fundamental computational motif in sensory cortex is to signal prediction errors, at least those based on predictions derived from temporal patterns of visual stimulation.

Dynamic Suppression of Average Facial Structure Shapes Neural Tuning in Three Macaque Face Patches

The visual perception of identity in humans and other primates is thought to draw upon cortical areas specialized for the analysis of facial structure. A prominent theory of face recognition holds that the brain computes and stores average facial structure, which it then uses to efficiently determine individual identity, though the neural mechanisms underlying this process are controversial. Here, we demonstrate that the dynamic suppression of average facial structure plays a prominent role in the responses of neurons in three fMRI-defined face patches of the macaque. Using photorealistic face stimuli that systematically varied in identity level according to a psychophysically based face space, we found that single units in the AF, AM, and ML face patches exhibited robust tuning around average facial structure. This tuning emerged after the initial excitatory response to the face and was expressed as the selective suppression of sustained responses to low-identity faces. The coincidence of this suppression with increased spike timing synchrony across the population suggests a mechanism of active inhibition underlying this effect. Control experiments confirmed that the diminished responses to low-identity faces were not due to short-term adaptation processes. We propose that the brain's neural suppression of average facial structure facilitates recognition by promoting the extraction of distinctive facial characteristics and suppressing redundant or irrelevant responses across the population.

Within- and between-hemifield generalization of repetition suppression in inferior temporal cortex

The decrease in response with stimulus repetition is a common property observed in many sensory brain areas. This repetition suppression (RS) is ubiquitous in neurons of macaque inferior temporal (IT) cortex, the end-stage of the ventral visual pathway. The neural mechanisms of RS in IT are still unclear, and one possibility is that it is inherited from areas upstream to IT that show also RS. Since neurons in IT have larger receptive fields compared with earlier visual areas, we examined the inheritance hypothesis by presenting adapter and test stimuli at widely different spatial locations along both vertical and horizontal meridians and across hemifields. RS was present for distances between adapter and test stimuli up to 22° and when the two stimuli were presented in different hemifields. Also, we examined the position tolerance of the stimulus selectivity of adaptation by comparing the responses to a test stimulus following the same (repetition trial) or a different (alternation trial) adapter at a position different from the test stimulus. Stimulus-selective adaptation was still present and consistently stronger in the later phase of the response for distances up to 18°. Finally, we observed stimulus-selective adaptation in repetition trials even without a measurable excitatory response to the adapter stimulus. To accommodate these and previous data, we propose that at least part of the stimulus-selective adaptation in IT is based on short-term plasticity mechanisms within IT and/or reflects top-down activity from areas downstream to IT.NEW & NOTEWORTHY Neurons in inferior temporal cortex reduce their response when stimuli are repeated. To assess whether this repetition suppression is inherited from upstream visual areas, we examined the extent of its spatial generalization. We observed stimulus-selective adaptation when adapter and test stimuli were presented at widely different spatial positions and in different hemifields. These data suggest that at least part of the repetition suppression originates within inferior temporal cortex and/or reflects feedback from downstream areas.

Incorporating intrinsic suppression in deep neural networks captures dynamics of adaptation in neurophysiology and perception

Adaptation is a fundamental property of sensory systems that can change subjective experiences in the context of recent information. Adaptation has been postulated to arise from recurrent circuit mechanisms or as a consequence of neuronally intrinsic suppression. However, it is unclear whether intrinsic suppression by itself can account for effects beyond reduced responses. Here, we test the hypothesis that complex adaptation phenomena can emerge from intrinsic suppression cascading through a feedforward model of visual processing. A deep convolutional neural network with intrinsic suppression captured neural signatures of adaptation including novelty detection, enhancement, and tuning curve shifts, while producing aftereffects consistent with human perception. When adaptation was trained in a task where repeated input affects recognition performance, an intrinsic mechanism generalized better than a recurrent neural network. Our results demonstrate that feedforward propagation of intrinsic suppression changes the functional state of the network, reproducing key neurophysiological and perceptual properties of adaptation.

External (Versus Internal) Facial Features Contribute Most to Repetition Priming in Facial Recognition: ERP Evidence

Previous event-related potential (ERP) research demonstrated four successive ERP components in the repetition priming of human face recognition: P100, N170, N250r, and N400. While these components correspond, respectively, to the four stages proposed by the interactive activation and competition (IAC) model, there has been no emphasis in past research on how internal and external facial features affect repetition priming and the sensitivity of these ERP components to item interval. This study was designed to address these issues. We used faces of celebrities as targets, including completely familiar faces, familiar internal feature faces, and familiar external feature faces. We displayed a target face either immediately following its prime (immediate repetition) or after a delay with interference from a presentation of two other faces (delayed repetition). ERP differences at P100 and N170 were nearly statistically non-significant; familiar faces and familiar external feature faces were associated with reliable ERP signals of N250r and N400 in the immediate repetition condition. For delayed repetition, however, N250r and N400 signals were only preserved for the familiar external feature faces. The differences of these ERP components suggest that, compared with internal facial features, external features of a previously presented face contribute more to brain-based facial repetition priming, particularly during the last two stages of the IAC model.

Diverse Temporal Dynamics of Repetition Suppression Revealed by Intracranial Recordings in the Human Ventral Temporal Cortex

Repeated stimulus presentations commonly produce decreased neural responses-a phenomenon known as repetition suppression (RS) or adaptation-in ventral temporal cortex (VTC) of humans and nonhuman primates. However, the temporal features of RS in human VTC are not well understood. To fill this gap in knowledge, we utilized the precise spatial localization and high temporal resolution of electrocorticography (ECoG) from nine human subjects implanted with intracranial electrodes in the VTC. The subjects viewed nonrepeated and repeated images of faces with long-lagged intervals and many intervening stimuli between repeats. We report three main findings: 1) robust RS occurs in VTC for activity in high-frequency broadband (HFB), but not lower-frequency bands; 2) RS of the HFB signal is associated with lower peak magnitude (PM), lower total responses, and earlier peak responses; and 3) RS effects occur early within initial stages of stimulus processing and persist for the entire stimulus duration. We discuss these findings in the context of early and late components of visual perception, as well as theoretical models of repetition suppression.

Evaluating the neurophysiological evidence for predictive processing as a model of perception

For many years, the dominant theoretical framework guiding research into the neural origins of perceptual experience has been provided by hierarchical feedforward models, in which sensory inputs are passed through a series of increasingly complex feature detectors. However, the long-standing orthodoxy of these accounts has recently been challenged by a radically different set of theories that contend that perception arises from a purely inferential process supported by two distinct classes of neurons: those that transmit predictions about sensory states and those that signal sensory information that deviates from those predictions. Although these predictive processing (PP) models have become increasingly influential in cognitive neuroscience, they are also criticized for lacking the empirical support to justify their status. This limited evidence base partly reflects the considerable methodological challenges that are presented when trying to test the unique predictions of these models. However, a confluence of technological and theoretical advances has prompted a recent surge in human and nonhuman neurophysiological research seeking to fill this empirical gap. Here, we will review this new research and evaluate the degree to which its findings support the key claims of PP.

Similar Expectation Effects for Immediate and Delayed Stimulus Repetitions

A prior cue or stimulus allows prediction of the future occurrence of an event and therefore reduces the associated neural activity in several cortical areas. This phenomenon is labeled expectation suppression (ES) and has recently been shown to be independent of the generally observed effects of stimulus repetitions (repetition suppression, RS: reduced neuronal response after the repetition of a given stimulus). While it has been shown that attentional cueing is strongly affected by the length of the cue-target delay, we have no information on the temporal dynamics of expectation effects, as in most prior studies of ES the delay between the predictive cue and the target (i.e., the inter-stimulus interval, ISI) was in the range of a few hundred milliseconds. Hence, we presented participants with pairs of faces where the first face could be used to build expectations regarding the second one, in the sense that one gender indicated repetition of the same face while the other gender predicted the occurrence of novel faces. In addition, we presented the stimulus pairs with two different ISIs (0.5 s for Immediate and 1.75 or 3.75 s for Delayed ISIs). We found significant RS as well as a reduced response for correctly predicted when compared to surprising trials in the fusiform face area. Importantly, the effects of repetition and expectation were both independent of the length of the ISI period. This implies that Immediate and Delayed cue-target stimulus arrangements lead to similar expectation effects in the face sensitive-visual cortex.

Interactions between Conscious and Subconscious Signals: Selective Attention under Feature-Based Competition Increases Neural Selectivity during Brain Adaptation

Efficient perception in natural environments depends on neural interactions between voluntary processes within cognitive control, such as attention, and those that are automatic and subconscious, such as brain adaptation to predictable input (also called repetition suppression). Although both attention and adaptation have been studied separately and there is considerable knowledge of the neurobiology involved in each of these processes, how attention interacts with adaptation remains equivocal. We examined how attention interacts with visual and auditory adaptation by measuring neuroimaging effects consistent with changes in either neural gain or selectivity. Male and female human participants were scanned with functional magnetic resonance imaging (fMRI) first while they discriminated repetition of morphed faces or voices and either directed their attention to stimulus identity or spatial location. Attention to face or voice identity, while ignoring stimulus location, solely increased the gain of respectively face- or voice-sensitive cortex. The results were strikingly different in an experiment when participants attended to voice identity versus stimulus loudness. In this case, attention to voice while ignoring sound loudness increased neural selectivity. The combined results show that how attention affects adaptation depends on the level of feature-based competition, reconciling prior conflicting observations. The findings are theoretically important and are discussed in relation to neurobiological interactions between attention and different types of predictive signals.SIGNIFICANCE STATEMENT Adaptation to repeated environmental events is ubiquitous in the animal brain, an automatic typically subconscious, predictive signal. Cognitive influences, such as by attention, powerfully affect sensory processing and can overcome brain adaptation. However, how neural interactions occur between adaptation and attention remains controversial. We conducted fMRI experiments regulating the focus of attention during adaptation to repeated stimuli with perceptually balanced stimulus expectancy. We observed an interaction between attention and adaptation consistent with increased neural selectivity, but only under conditions of feature-based competition, challenging the notion that attention interacts with brain adaptation by only affecting response gain. This demonstrates that attention retains its full complement of mechanistic influences on sensory cortex even as it interacts with more automatic or subconscious predictive processes.

Repetition Suppression for Noisy and Intact Faces in the Occipito-Temporal Cortex

Repetition suppression (RS), the relative lower neural response magnitude to repeated as compared to non-repeated stimuli, is often explained within the predictive coding framework. According to this theory, precise predictions (priors) together with less precise sensory evidences lead to decisions that are determined largely by the predictions and the other way around. In other words, the prediction error, namely the magnitude of RS, should depend on the precision of predictions and sensory inputs. In the current study, we aimed at testing this idea by manipulating the clarity and thereby the precision of the incoming sensory data by adding noise to the images. This resulted in an fMRI adaptation design with repeated or alternating trials showing clear or noisy face stimuli. Our results show a noise effect on the activity in the fusiform face area (FFA), namely less activation for noisy than for clear trials, which supports previous findings. No such effects could be found in OFA or LO. Data also showed reliable RS in the FFA (bilateral) and unilaterally in OFA (right) and LO (left). Interestingly, the noise added to the stimuli did not affect the magnitude of RS in any of the tested cortical areas. This suggests that the clarity of the sensory input is not crucial in determining the magnitude of RS.

Probing the Mechanisms of Repetition Suppression in Inferior Temporal Cortex with Optogenetics

Neurons in macaque inferior temporal (IT) cortex show a decrease in the response with stimulus repetition, known as repetition suppression (RS). Several mechanisms may contribute to RS in IT, such as firing rate-dependent fatigue and transsynaptic mechanisms, like synaptic depression or reduced input from neurons within the same area or from up- or downstream areas. We examined the role of firing rate fatigue and transsynaptic mechanisms by stimulating directly IT neurons using optogenetics and measured the effect of photo-stimulation on their responses using timing parameters that resulted in RS for visual stimuli. Photo-stimulation of IT neurons resulted in a marginally decreased probability of spiking activity to a subsequent photo-stimulation or to a subsequent low-contrast visual stimulus. This response reduction was small relative to that for repeated visual stimuli and was related to post-stimulation inhibition of the activity during the interval between adapter and test stimuli. Presentation of a visual adapter did not change the response to subsequent photo-stimulation. In neurons whose response to the visual adapter was inhibited by simultaneous photo-stimulation, RS to visual stimuli was unaffected. Overall, these data imply that RS in IT has a transsynaptic origin, with little or no contribution of intrinsic firing rate fatigue. In addition, they suggest a limited contribution of both local synaptic depression and reduced input from nearby IT neurons, whose responses were postulated to be decreased by firing rate fatigue, to RS in IT.