Effect of adapter duration on repetition suppression in inferior temporal cortex

Adaptation in the visual system: Networked fatigue or suppressed prediction error signalling?

Our brains are constantly adapting to changes in our visual environments. Neural adaptation exerts a persistent influence on the activity of sensory neurons and our perceptual experience, however there is a lack of consensus regarding how adaptation is implemented in the visual system. One account describes fatigue-based mechanisms embedded within local networks of stimulus-selective neurons (networked fatigue models). Another depicts adaptation as a product of stimulus expectations (predictive coding models). In this review, I evaluate neuroimaging and psychophysical evidence that poses fundamental problems for predictive coding models of neural adaptation. Specifically, I discuss observations of distinct repetition and expectation effects, as well as incorrect predictions of repulsive adaptation aftereffects made by predictive coding accounts. Based on this evidence, I argue that networked fatigue models provide a more parsimonious account of adaptation effects in the visual system. Although stimulus expectations can be formed based on recent stimulation history, any consequences of these expectations are likely to co-occur (or interact) with effects of fatigue-based adaptation. I conclude by proposing novel, testable hypotheses relating to interactions between fatigue-based adaptation and other predictive processes, focusing on stimulus feature extrapolation phenomena.

Temporal continuity shapes visual responses of macaque face patch neurons

Macaque inferior temporal cortex neurons respond selectively to complex visual images, with recent work showing that they are also entrained reliably by the evolving content of natural movies. To what extent does temporal continuity itself shape the responses of high-level visual neurons? We addressed this question by measuring how cells in face-selective regions of the macaque visual cortex were affected by the manipulation of a movie's temporal structure. Sampling a 5-min movie at 1 s intervals, we measured neural responses to randomized, brief stimuli of different lengths, ranging from 800 ms dynamic movie snippets to 100 ms static frames. We found that the disruption of temporal continuity strongly altered neural response profiles, particularly in the early response period after stimulus onset. The results suggest that models of visual system function based on discrete and randomized visual presentations may not translate well to the brain's natural modes of operation.

Independent repetition suppression in macaque area V2 and inferotemporal cortex

When a complexly structured natural image is presented twice in succession, first as adapter and then as test, neurons in area TE of macaque inferotemporal cortex exhibit repetition suppression, responding less strongly to the second presentation than to the first. This phenomenon, which has been studied primarily in TE, might plausibly be argued to arise in TE because TE neurons respond selectively to complex images and thus carry information adequate for determining whether an image is or is not a repeat. However, the idea has never been put to a direct test. To resolve this issue, we monitored neuronal responses to sequences of complex natural images under identical conditions in areas V2 and TE. We found that repetition suppression occurs in both areas. Moreover, in each area, suppression takes the form of a dynamic alteration whereby the initial peak of excitation is followed by a trough and then a rebound of firing rate. To assess whether repetition suppression in either area is transmitted from the other area, we analyzed the timing of the phenomenon and its degree of spatial generalization. Suppression occurs at shorter latency in V2 than in TE. Therefore it is not simply fed back from TE. Suppression occurs in TE but not in V2 under conditions in which the test and adapter are presented in different visual field quadrants. Therefore it is not simply fed forward from V2. We conclude that repetition suppression occurs independently in V2 and TE.NEW & NOTEWORTHY When a complexly structured natural image is presented twice in rapid succession, neurons in inferotemporal area TE exhibit repetition suppression, responding less strongly to the second than to the first presentation. We have explored whether this phenomenon is confined to high-order areas where neurons respond selectively to such images and thus carry information relevant to recognizing a repeat. We have found surprisingly that repetition suppression occurs even in low-order visual area V2.

Contribution of Individual Features to Repetition Suppression in Macaque Inferotemporal Cortex

When an image is presented twice in succession, neurons in area TE of macaque inferotemporal cortex exhibit repetition suppression, responding less strongly to the second presentation than to the first. Suppression is known to occur if the adapter and the test image are subtly different from each other. However, it is not known whether cross-suppression occurs between images that are radically different from each other but that share a subset of features. To explore this issue, we measured repetition suppression using colored shapes. On interleaved trials, the test image might be identical to the adapter, might share its shape or color alone or might differ from it totally. At the level of the neuronal population as a whole, suppression was especially deep when adapter and test were identical, intermediate when they shared only one attribute and minimal when they shared neither attribute. At the level of the individual neuron, the degree of suppression depended not only on the properties of the two images but also on the preferences of the neuron. Suppression was deeper when the repeated color or shape was preferred by the neuron than when it was not. This effect might arise from feature-specific adaptation or alternatively from adapter-induced fatigue. Both mechanisms conform to the principle that the degree of suppression is determined by the preferences of the neuron.

Stimulus-specific plasticity of macaque V1 spike rates and gamma

When a visual stimulus is repeated, average neuronal responses typically decrease, yet they might maintain or even increase their impact through increased synchronization. Previous work has found that many repetitions of a grating lead to increasing gamma-band synchronization. Here, we show in awake macaque area V1 that both repetition-related reductions in firing rate and increases in gamma are specific to the repeated stimulus. These effects show some persistence on the timescale of minutes. Gamma increases are specific to the presented stimulus location. Further, repetition effects on gamma and on firing rates generalize to images of natural objects. These findings support the notion that gamma-band synchronization subserves the adaptive processing of repeated stimulus encounters.

Within- and between-hemifield generalization of repetition suppression in inferior temporal cortex

The decrease in response with stimulus repetition is a common property observed in many sensory brain areas. This repetition suppression (RS) is ubiquitous in neurons of macaque inferior temporal (IT) cortex, the end-stage of the ventral visual pathway. The neural mechanisms of RS in IT are still unclear, and one possibility is that it is inherited from areas upstream to IT that show also RS. Since neurons in IT have larger receptive fields compared with earlier visual areas, we examined the inheritance hypothesis by presenting adapter and test stimuli at widely different spatial locations along both vertical and horizontal meridians and across hemifields. RS was present for distances between adapter and test stimuli up to 22° and when the two stimuli were presented in different hemifields. Also, we examined the position tolerance of the stimulus selectivity of adaptation by comparing the responses to a test stimulus following the same (repetition trial) or a different (alternation trial) adapter at a position different from the test stimulus. Stimulus-selective adaptation was still present and consistently stronger in the later phase of the response for distances up to 18°. Finally, we observed stimulus-selective adaptation in repetition trials even without a measurable excitatory response to the adapter stimulus. To accommodate these and previous data, we propose that at least part of the stimulus-selective adaptation in IT is based on short-term plasticity mechanisms within IT and/or reflects top-down activity from areas downstream to IT.NEW & NOTEWORTHY Neurons in inferior temporal cortex reduce their response when stimuli are repeated. To assess whether this repetition suppression is inherited from upstream visual areas, we examined the extent of its spatial generalization. We observed stimulus-selective adaptation when adapter and test stimuli were presented at widely different spatial positions and in different hemifields. These data suggest that at least part of the repetition suppression originates within inferior temporal cortex and/or reflects feedback from downstream areas.

Diverse Temporal Dynamics of Repetition Suppression Revealed by Intracranial Recordings in the Human Ventral Temporal Cortex

Repeated stimulus presentations commonly produce decreased neural responses-a phenomenon known as repetition suppression (RS) or adaptation-in ventral temporal cortex (VTC) of humans and nonhuman primates. However, the temporal features of RS in human VTC are not well understood. To fill this gap in knowledge, we utilized the precise spatial localization and high temporal resolution of electrocorticography (ECoG) from nine human subjects implanted with intracranial electrodes in the VTC. The subjects viewed nonrepeated and repeated images of faces with long-lagged intervals and many intervening stimuli between repeats. We report three main findings: 1) robust RS occurs in VTC for activity in high-frequency broadband (HFB), but not lower-frequency bands; 2) RS of the HFB signal is associated with lower peak magnitude (PM), lower total responses, and earlier peak responses; and 3) RS effects occur early within initial stages of stimulus processing and persist for the entire stimulus duration. We discuss these findings in the context of early and late components of visual perception, as well as theoretical models of repetition suppression.

Forgotten but not gone: FMRI evidence of implicit memory for negative stimuli 24 hours after the initial study episode

Endel Tulving conducted pioneering work on the explicit and implicit memory systems and demonstrated that priming effects can be long-lasting. It is also well-established that emotion can amplify explicit and implicit memory. Prior work has utilized repetition suppression (RS) of the fMRI-BOLD signal-a reduction in the magnitude of activity over repeated presentations of stimuli-to index implicit memory. Using an explicit recognition memory paradigm, we examined emotional modulation of long-term implicit memory effects as revealed by repetition suppression (i.e., comparing second-exposure forgotten items to first-exposure correct rejections). Forty-seven participants incidentally encoded line-drawings of negative, positive, and neutral scenes followed by the full color image. Twenty-four hours later, participants underwent fMRI during a recognition memory test in which old and new line-drawings were presented. Implicit and explicit memory effects were defined by the contrasts of New-Correct Rejections > Old-Misses and Old-Hits > New-Correct Rejections, respectively. Wide-spread Negative RS was found in frontal and occipito-temporal cortex that was greater than Neutral RS in the right orbito-frontal cortex and inferior frontal gyri. Valence-specific Negative RS, compared to Positive RS, was observed in the left inferior occipital gyrus. There was no strong evidence for emotional modulation of amygdala RS, but functional connectivity analyses revealed valence-specificity: Negative and positive valence were associated with repetition suppression and repetition enhancement of amygdala-occipital connectivity, respectively. Negative implicit memory patterns in most frontal regions-but not occipital areas-overlapped with explicit memory effects. Thus, implicit memory effects for a single visual stimulus presentation are modulated by emotional valence, can be observed 24hours after initial exposure, and show some overlap with explicit memory.

Probing the Mechanisms of Repetition Suppression in Inferior Temporal Cortex with Optogenetics

Neurons in macaque inferior temporal (IT) cortex show a decrease in the response with stimulus repetition, known as repetition suppression (RS). Several mechanisms may contribute to RS in IT, such as firing rate-dependent fatigue and transsynaptic mechanisms, like synaptic depression or reduced input from neurons within the same area or from up- or downstream areas. We examined the role of firing rate fatigue and transsynaptic mechanisms by stimulating directly IT neurons using optogenetics and measured the effect of photo-stimulation on their responses using timing parameters that resulted in RS for visual stimuli. Photo-stimulation of IT neurons resulted in a marginally decreased probability of spiking activity to a subsequent photo-stimulation or to a subsequent low-contrast visual stimulus. This response reduction was small relative to that for repeated visual stimuli and was related to post-stimulation inhibition of the activity during the interval between adapter and test stimuli. Presentation of a visual adapter did not change the response to subsequent photo-stimulation. In neurons whose response to the visual adapter was inhibited by simultaneous photo-stimulation, RS to visual stimuli was unaffected. Overall, these data imply that RS in IT has a transsynaptic origin, with little or no contribution of intrinsic firing rate fatigue. In addition, they suggest a limited contribution of both local synaptic depression and reduced input from nearby IT neurons, whose responses were postulated to be decreased by firing rate fatigue, to RS in IT.

GABAergic and cholinergic modulation of repetition suppression in inferior temporal cortex

Neurons in many brain areas of different species reduce their response when a stimulus is repeated. Such adaptation or repetition suppression is prevalent in inferior temporal (IT) cortex. The mechanisms underlying repetition suppression in IT are still poorly understood. Studies in rodents and in-vitro experiments suggest that acetylcholine and GABA can contribute to repetition suppression by interacting with fatigue-related or local adaptation mechanisms. Here, we examined the contribution of cholinergic and GABAergic mechanisms to repetition suppression in macaque IT, using an adaptation paradigm in which familiar images were presented successively with a short interstimulus interval. We found that intracortical local injections of acetylcholine and of the GABA_A receptor antagonist Gabazine both increased repetition suppression in awake macaque IT. The increased repetition suppression was observed for both spiking activity and local field potential power. The latter was present mainly for frequencies below 50 Hz, spectral bands that typically do not show consistent repetition suppression in IT. Although increased with drug application, repetition suppression remained stimulus selective. These findings agree with the hypothesis that repetition suppression of IT neurons mainly results from suppressed input from upstream and other IT neurons but depend less on intrinsic neuronal fatigue.

Face Repetition Probability Does Not Affect Repetition Suppression in Macaque Inferotemporal Cortex

Repetition suppression, which refers to reduced neural activity for repeated stimuli, is typically explained by bottom-up or local adaptation mechanisms. However, recent theories have emphasized the role of top-down processes, suggesting that this response reduction reflects the fulfillment of perceptual expectations. To support this, an influential human fMRI study showed that the magnitude of suppression is modulated by the probability of a repetition. No such repetition probability effect was found in macaque inferior temporal (IT) cortex for spiking activity despite the presence of repetition suppression. Contrary to the human fMRI studies that showed an effect of repetition probability, the macaque single-unit study used a large variety of unfamiliar stimuli and the monkeys were not required to attend the stimuli. Here, as in the human fMRI studies, we used faces as stimuli and made the monkeys attend to the stimulus content. We simultaneously recorded spiking activity and local field potentials (LFPs) in the middle lateral face patch (ML) of one monkey (male) and a face-responsive region of another (female). Although we observed significant repetition suppression of spiking activity and high gamma-band LFPs in both animals, there were no effects of repetition probability even when repetitions were task relevant and repetition probability affected behavioral decisions. In conclusion, despite the use of face stimuli and a stimulus-related task, no neural signature of repetition probability was present for faces in a face responsive patch of macaque IT. This further challenges a general perceptual expectation account of repetition suppression.SIGNIFICANCE STATEMENT Repetition suppression is a reduced brain activity for repeated stimuli commonly observed across species. In the predictive coding framework, such suppression is thought to reflect fulfilled perceptual expectations. Although this hypothesis is supported by several human fMRI studies reporting an effect of repetition probability on repetition suppression, this could not be replicated in single-cell recordings in monkey inferior temporal (IT) cortex. Subsequent studies narrowed down the conditions for the effect to requiring attention and being limited to particular stimulus categories such as faces. Here, we show that, even under these conditions, repetition suppression in monkey IT neurons is still unaffected by repetition probability, even in a task with a behavioral effect, challenging the perceptual expectation account of repetition suppression.

The impact of stimulus size and orientation on individual face coding in monkey face-selective cortex

Face-selective neurons in the monkey temporal cortex discharge at different rates in response to pictures of different individual faces. Here we tested whether this pattern of response across single neurons in the face-selective area ML (located in the middle Superior Temporal Sulcus) tolerates two affine transformations; picture-plane inversion, known to decrease the average response of face-selective neurons and the other, stimulus size. We recorded the response of 57 ML neurons in two awake and fixating monkeys. Face stimuli were presented at two sizes (10 and 5 degrees of visual angle) and two orientations (upright and inverted). Different faces elicited distinct patterns of activity across ML neurons that were reliable (i.e., predictable with a classifier) within a specific size and orientation condition. Despite observing a reduction in the average response magnitude of face-selective neurons to inverted faces, compared to upright faces, classifier performance was above chance for both upright and inverted faces. While decoding was largely preserved across changes in stimulus size, a classifier trained with one orientation condition and tested on the other did not lead to performance above chance level. We conclude that different individual faces can be decoded from patterns of responses in the monkey area ML regardless of orientation or size, but with qualitatively different patterns of responses for upright and inverted faces.

Adaptation aftereffects reveal representations for encoding of contingent social actions

A hallmark of human social behavior is the effortless ability to relate one's own actions to that of the interaction partner, e.g., when stretching out one's arms to catch a tripping child. What are the behavioral properties of the neural substrates that support this indispensable human skill? Here we examined the processes underlying the ability to relate actions to each other, namely the recognition of spatiotemporal contingencies between actions (e.g., a "giving" that is followed by a "taking"). We used a behavioral adaptation paradigm to examine the response properties of perceptual mechanisms at a behavioral level. In contrast to the common view that action-sensitive units are primarily selective for one action (i.e., primary action, e.g., 'throwing"), we demonstrate that these processes also exhibit sensitivity to a matching contingent action (e.g., "catching"). Control experiments demonstrate that the sensitivity of action recognition processes to contingent actions cannot be explained by lower-level visual features or amodal semantic adaptation. Moreover, we show that action recognition processes are sensitive only to contingent actions, but not to noncontingent actions, demonstrating their selective sensitivity to contingent actions. Our findings show the selective coding mechanism for action contingencies by action-sensitive processes and demonstrate how the representations of individual actions in social interactions can be linked in a unified representation.