The effect of attention on repetition suppression and multivoxel pattern similarity

Attention enhances category representations across the brain with strengthened residual correlations to ventral temporal cortex

How does attention enhance neural representations of goal-relevant stimuli while suppressing representations of ignored stimuli across regions of the brain? While prior studies have shown that attention enhances visual responses, we lack a cohesive understanding of how selective attention modulates visual representations across the brain. Here, we used functional magnetic resonance imaging (fMRI) while participants performed a selective attention task on superimposed stimuli from multiple categories and used a data-driven approach to test how attention affects both decodability of category information and residual correlations (after regressing out stimulus-driven variance) with category-selective regions of ventral temporal cortex (VTC). Our data reveal three main findings. First, when two objects are simultaneously viewed, the category of the attended object can be decoded more readily than the category of the ignored object, with the greatest attentional enhancements observed in occipital and temporal lobes. Second, after accounting for the response to the stimulus, the correlation in the residual brain activity between a cortical region and a category-selective region of VTC was elevated when that region’s preferred category was attended vs. ignored, and more so in the right occipital, parietal, and frontal cortices. Third, we found that the stronger the residual correlations between a given region of cortex and VTC, the better visual category information could be decoded from that region. These findings suggest that heightened residual correlations by selective attention may reflect the sharing of information between sensory regions and higher-order cortical regions to provide attentional enhancement of goal-relevant information.

Attention modulates repetition effects in a context of low periodicity

Stimulus repetition can result in a reduction in neural responses (i.e., repetition suppression) in neuroimaging studies. Predictive coding models of perception postulate that this phenomenon largely reflects the top-down attenuation of prediction errors. Electroencephalography research further demonstrated that repetition effects consist of sequentially ordered attention-independent and attention-dependent components in a context of high periodicity. However, the statistical structure of our auditory environment is richer than that of a fixed pattern. It remains unclear if the attentional modulation of repetition effects can be generalised to a setting which better represents the nature of our auditory environment. Here we used electroencephalography to investigate whether the attention-independent and attention-dependent components of repetition effects previously described in the auditory modality remain in a context of low periodicity where temporary disruption might be absent/present. Participants were presented with repetition trains of various lengths, with/without temporary disruptions. We found attention-independent and attention-dependent repetition effects on, respectively, the P2 and P3a event-related potential components. This pattern of results is in line with previous research, confirming that the attenuation of prediction errors upon stimulus repetition is first registered regardless of attentional state before further attenuation of attended but not unattended prediction errors takes place. However, unlike previous reports, these effects manifested on later components. This divergence from previous studies is discussed in terms of the possible contribution of contextual factors.

Transient Neural Activation of Abstract Relations on an Incidental Analogy Task

Although a large proportion of the lexicon consists of abstract concepts, little is known about how they are represented by the brain. Here, we investigated how the mind represents relations shared between sets of mental representations that are superficially unrelated, such as car-engine and dog-tongue, but that nonetheless share a more general, abstract relation, such as whole-part. Participants saw a pair of words on each trial and were asked to indicate whether they could think of a relation between them. Importantly, they were not explicitly asked whether different word pairs shared the same relation, as in analogical reasoning tasks. We observed representational similarity for abstract relations in regions in the "conceptual hub" network, even when controlling for semantic relatedness between word pairs. By contrast, we did not observe representational similarity in regions previously implicated in explicit analogical reasoning. A given relation was sometimes repeated across sequential word pairs, allowing us to test for behavioral and neural priming of abstract relations. Indeed, we observed faster RTs and greater representational similarity for primed than unprimed trials, suggesting that mental representations of abstract relations are transiently activated on this incidental analogy task. Finally, we found a significant correlation between behavioral and neural priming across participants. To our knowledge, this is the first study to investigate relational priming using functional neuroimaging and to show that neural representations are strengthened by relational priming. This research shows how abstract concepts can be brought to mind momentarily, even when not required for task performance.

Functions of ventral visual cortex after bilateral medial temporal lobe damage

Repeated stimuli elicit attenuated responses in visual cortex relative to novel stimuli. This adaptation can be considered as a form of rapid learning and a signature of perceptual memory. Adaptation occurs not only when a stimulus is repeated immediately, but also when there is a lag in terms of time and other intervening stimuli before the repetition. But how does the visual system keep track of which stimuli are repeated, especially after long delays and many intervening stimuli? We hypothesized that the hippocampus and medial temporal lobe (MTL) support long-lag adaptation, given that this memory system can learn from single experiences, maintain information over delays, and send feedback to visual cortex. We tested this hypothesis with fMRI in an amnesic patient, LSJ, who has encephalitic damage to the MTL resulting in extensive bilateral lesions including complete hippocampal loss. We measured adaptation at varying time lags between repetitions in functionally localized visual areas that were intact in LSJ. We observed that these areas track information over a few minutes even when the hippocampus and extended parts of the MTL are unavailable. LSJ and controls were identical when attention was directed away from the repeating stimuli: adaptation occurred for lags up to three minutes, but not six minutes. However, when attention was directed toward stimuli, controls now showed an adaptation effect at six minutes but LSJ did not. These findings suggest that visual cortex can support one-shot perceptual memories lasting for several minutes but that the hippocampus and surrounding MTL structures are necessary for adaptation in visual cortex after longer delays when stimuli are task-relevant.

Reduced Fidelity of Neural Representation Underlies Episodic Memory Decline in Normal Aging

Emerging studies have emphasized the importance of the fidelity of cortical representation in forming enduring episodic memory. No study, however, has examined whether there are age-related reductions in representation fidelity that can explain memory declines in normal aging. Using functional MRI and multivariate pattern analysis, we found that older adults showed reduced representation fidelity in the visual cortex, which accounted for their decreased memory performance even after controlling for the contribution of reduced activation level. This reduced fidelity was specifically due to older adults' poorer item-specific representation, not due to their lower activation level and variance, greater variability in neuro-vascular coupling, or decreased selectivity of categorical representation (i.e., dedifferentiation). Older adults also showed an enhanced subsequent memory effect in the prefrontal cortex based on activation level, and their prefrontal activation was associated with greater fidelity of representation in the visual cortex and better memory performance. The fidelity of cortical representation thus may serve as a promising neural index for better mechanistic understanding of the memory declines and its compensation in normal aging.

Representational differences between line drawings and photographs of natural scenes: A dissociation between multi-voxel pattern analysis and repetition suppression

Distributed representations of scene categories are consistent between color photographs (CPs) and line drawings (LDs) in the parahippocampal place area (PPA) and the retrosplenial cortex (RSC), as shown using multi-voxel pattern analysis (MVPA). Here, we used repetition suppression (RS) to further investigate the degree of representational convergence between CPs and LDs of natural scenes. MVPA and RS can capture different aspects of visual representations, and RS may prove useful in elucidating important differences in the representations of CPs and LDs of natural scenes. We performed an event-related fMRI experiment, including image-repetitions either within-type (i.e., CP to CP or LD to LD) or between-types (CP to LD, LD to CP). We found significant RS for within-type repetitions in PPA, RSC and the occipital place area (OPA), but did not observe RS for between-types repetitions. By contrast, scene categories were decodable from activity patterns evoked by both CPs and LDs using SVM classification for both within-type decoding and between-types cross-decoding. We conclude that there are representational differences between CPs and LDs in scene-selective cortex despite a category-level correspondence.

Lower Parietal Encoding Activation Is Associated with Sharper Information and Better Memory

Mean fMRI activation in ventral posterior parietal cortex (vPPC) during memory encoding often negatively predicts successful remembering. A popular interpretation of this phenomenon is that vPPC reflects "off-task" processing. However, recent fMRI studies considering distributed patterns of activity suggest that vPPC actively represents encoded material. Here, we assessed the relationships between pattern-based content representations in vPPC, mean activation in vPPC, and subsequent remembering. We analyzed data from two fMRI experiments where subjects studied then recalled word-face or word-scene associations. For each encoding trial, we measured 1) mean univariate activation within vPPC and 2) the strength of face/scene information as indexed by pattern analysis. Mean activation in vPPC negatively predicted subsequent remembering, but the strength of pattern-based information in the same vPPC voxels positively predicted later memory. Indeed, univariate amplitude averaged across vPPC voxels negatively correlated with pattern-based information strength. This dissociation reflected a tendency for univariate reductions to maximally occur in voxels that were not strongly tuned for the category of encoded stimuli. These results indicate that vPPC activity patterns reflect the content and quality of memory encoding and constitute a striking example of lower univariate activity corresponding to stronger pattern-based information.

An improved P300 pattern in BCI to catch user's attention

OBJECTIVE

Brain-computer interfaces (BCIs) can help patients who have lost control over most muscles but are still conscious and able to communicate or interact with the environment. One of the most popular types of BCI is the P300-based BCI. With this BCI, users are asked to count the number of appearances of target stimuli in an experiment. To date, the majority of visual P300-based BCI systems developed have used the same character or picture as the target for every stimulus presentation, which can bore users. Consequently, users attention may decrease or be negatively affected by adjacent stimuli.

APPROACH

In this study, a new stimulus is presented to increase user concentration. Honeycomb-shaped figures with 1-3 red dots were used as stimuli. The number and the positions of the red dots in the honeycomb-shaped figure were randomly changed during BCI control. The user was asked to count the number of the dots presented in each flash instead of the number of times they flashed. To assess the performance of this new stimulus, another honeycomb-shaped stimulus, without red dots, was used as a control condition.

MAIN RESULTS

The results showed that the honeycomb-shaped stimuli with red dots obtained significantly higher classification accuracies and information transfer rates (p  <  0.05) compared to the honeycomb-shaped stimulus without red dots.

SIGNIFICANCE

The results indicate that this proposed method can be a promising approach to improve the performance of the BCI system and can be an efficient method in daily application.

Brain regions that show repetition suppression and enhancement: A meta-analysis of 137 neuroimaging experiments

Repetition suppression and enhancement refer to the reduction and increase in the neural responses for repeated rather than novel stimuli, respectively. This study provides a meta-analysis of the effects of repetition suppression and enhancement, restricting the data used to that involving fMRI/PET, visual stimulus presentation, and healthy participants. The major findings were as follows. First, the global topography of the repetition suppression effects was strikingly similar to that of the "subsequent memory" effects, indicating that the mechanism for repetition suppression is the reduced engagement of an encoding system. The lateral frontal cortex effects involved the frontoparietal control network regions anteriorly and the dorsal attention network regions posteriorly. The left fusiform cortex effects predominantly involved the dorsal attention network regions, whereas the right fusiform cortex effects mainly involved the visual network regions. Second, the category-specific meta-analyses and their comparisons indicated that most parts of the alleged category-specific regions showed repetition suppression for more than one stimulus category. In this regard, these regions may not be "dedicated cortical modules," but are more likely parts of multiple overlapping large-scale maps of simple features. Finally, the global topography of the repetition enhancement effects was similar to that of the "retrieval success" effects, suggesting that the mechanism for repetition enhancement is voluntary or involuntary explicit retrieval during an implicit memory task. Taken together, these results clarify the network affiliations of the regions showing reliable repetition suppression and enhancement effects and contribute to the theoretical interpretations of the local and global topography of these two effects. Hum Brain Mapp 38:1894-1913, 2017. © 2017 Wiley Periodicals, Inc.

Distinct Neural Suppression and Encoding Effects for Conceptual Novelty and Familiarity

Like yin and yang, novelty and familiarity are typically described as separate-yet-complementary aspects of an experience, two ends of a single continuum. However, novelty and familiarity are also multifaceted. For instance, novelty can sometimes result in enhanced mnemonic performance, whereas at other times familiarity is better remembered. As previous investigations focused primarily on the experimental aspect of novelty, the mechanisms supporting conceptual novelty (the novel combination of two previously unrelated existing concepts) remain unclear. Importantly, conceptual novelty can be recognized as such only when compared with preexperimental familiar knowledge, regardless of experimental status. Here we applied a combined repetition suppression/subsequent memory fMRI paradigm, focusing on the conceptual aspect of novelty and familiarity as the subject matter. Conceptual novelty was characterized by sustained neural activity; familiarity, on the other hand, exhibited repetition effects in multiple cortical regions, a subset of which was modulated by successful encoding. Subsequent memory of novelty was associated only with activation differences in a distinct set of regions, including the hippocampus and medial cortical regions. These results suggest that conceptual novelty (a) does not (easily) trigger the repetition suppression phenomenon but requires sustained neural recruitment and (b) activates dedicated encoding mechanisms. Conceptual familiarity, in contrast, allows rapid neural processing that depends upon existing neural representations. Overall, these findings challenge the definition of novelty as a unitary concept. Furthermore, they bear important implications for research into the neural bases of knowledge representation and recognition memory.

Distinct Visual Processing of Real Objects and Pictures of Those Objects in 7- to 9-month-old Infants

The present study examined 7- and 9-month-old infants' visual habituation to real objects and pictures of the same objects and their preferences between real and pictorial versions of the same objects following habituation. Different hypotheses would predict that infants may habituate faster to pictures than real objects (based on proposed theoretical links between behavioral habituation in infants and neuroimaging adaptation in adults) or to real objects vs. pictures (based on past infant electrophysiology data). Sixty-one 7-month-old infants and fifty-nine 9-month-old infants were habituated to either a real object or a picture of the same object and afterward preference tested with the habituation object paired with either the novel real object or its picture counterpart. Infants of both age groups showed basic information-processing advantages for real objects. Specifically, during the initial presentations, 9-month-old infants looked longer at stimuli in both formats than the 7-month olds but more importantly both age groups looked longer at real objects than pictures, though with repeated presentations, they habituated faster for real objects such that at the end of habituation, they looked equally at both types of stimuli. Surprisingly, even after habituation, infants preferred to look at the real objects, regardless of whether they had habituated to photos or real objects. Our findings suggest that from as early as 7-months of age, infants show strong preferences for real objects, perhaps because real objects are visually richer and/or enable the potential for genuine interactions.

Neural representation of object orientation: A dissociation between MVPA and Repetition Suppression

How is object orientation represented in the brain? Behavioral error patterns reveal systematic tendencies to confuse certain orientations with one another. Using fMRI, we asked whether more confusable orientations are represented more similarly in object selective cortex (LOC). We compared two widely-used measures of neural similarity: multi-voxel pattern similarity (MVP-similarity) and Repetition Suppression. In LO, we found that multi-voxel pattern similarity was predicted by the confusability of two orientations. By contrast, Repetition Suppression effects in LO were unrelated to the confusability of orientations. To account for these differences between MVP-similarity and Repetition Suppression, we propose that MVP-similarity reflects the topographical distribution of neural populations, whereas Repetition Suppression depends on repeated activation of particular groups of neurons. This hypothesis leads to a unified interpretation of our results and may explain other dissociations between MVPA and Repetition Suppression observed in the literature.

Task influences pattern discriminability for faces and bodies in ventral occipitotemporal cortex

Our prior research showed that faces and bodies activate overlapping regions of the ventral occipitotemporal cortex (VOTC). However, faces and bodies were nonetheless discriminable in these same overlapping regions when their spatial patterns of activity were classified using multivoxel pattern analysis (MVPA). Here we investigated whether these spatial patterns and their time courses were influenced by different categorization tasks. Participants viewed pictures of faces or headless bodies depicting a happy or fearful emotion. In one task, they categorized the picture as a face or a body regardless of emotion. In the other task, they categorized the emotion regardless of whether it was depicted by a face or body. Using a classifier trained on independent data, we found higher face-body classification accuracy for the emotion categorization task. The classifier was applied to each post-stimulus time-point to characterize the temporal course of classification. Accuracy initially rose equivalently above chance for both tasks, but then increased over a longer duration when participants categorized emotions. Thus, the temporal course of pattern differences between faces and bodies in VOTC was modulated by the behavioral goal of the observer, suggesting the top-down modulatory effect of task context on the category-selectivity activity in the VOTC.

Repetition suppression and its contextual determinants in predictive coding

This paper presents a review of theoretical and empirical work on repetition suppression in the context of predictive coding. Predictive coding is a neurobiologically plausible scheme explaining how biological systems might perform perceptual inference and learning. From this perspective, repetition suppression is a manifestation of minimising prediction error through adaptive changes in predictions about the content and precision of sensory inputs. Simulations of artificial neural hierarchies provide a principled way of understanding how repetition suppression – at different time scales – can be explained in terms of inference and learning implemented under predictive coding. This formulation of repetition suppression is supported by results of numerous empirical studies of repetition suppression and its contextual determinants.

Neural Discriminability of Object Features Predicts Perceptual Organization

How does the neural representation of simple visual features affect perceptual operations, such as perceptual grouping? If the strength of feature representations in the brain is indicative of how the perceptual system partitions information into visual elements, then identifying the underlying neural representation may determine why things look the way they do. During functional MRI, participants viewed objects that varied along three feature dimensions: shape, color, and orientation. Afterward, participants performed an independent perceptual-grouping task outside the scanner to measure the strength of feature grouping. In lateral occipital cortex, neural feature discriminability, characterized using functional MRI multivariate pattern classification, positively predicted feature grouping strength: The more distinct the neural representations of a particular feature, the stronger the grouping was for that feature outside the scanner. Thus, variation in neural feature representation can be quantified to predict perceptual organization.

Spatiotemporal neural pattern similarity supports episodic memory

Formal computational models of human memory posit a central role of feature representations in episodic memory encoding and retrieval [1-4]. Correspondingly, fMRI studies have found that, in addition to activity level [5, 6], the neural activation pattern similarity across repetitions (i.e., self-similarity) was greater for subsequently remembered than forgotten items [7-9]. This self-similarity has been suggested to reflect pattern reinstatement due to study-phase retrieval [7, 10, 11]. However, the low temporal resolution of fMRI measures could determine neither the temporal precision of study-phase reinstatement nor the processing stage at which the reinstatement supported subsequent memory [12]. Meanwhile, although self-similarity has been shown to correlate with the activity level in the left lateral prefrontal cortex (LPFC) [10, 13], a causal link between left LPFC function and pattern similarity remains to be established. Combining transcranial direct current stimulation (tDCS) and EEG, we found that greater spatiotemporal pattern similarity (STPS) across repetitions of the same item (i.e., self-STPS) during encoding predicted better subsequent memory. The self-STPS located in the right frontal electrodes occurred approximately 500 ms after stimulus onset, reflected item-specific encoding, and contributed to memory above and beyond the effects of ERP amplitude and global pattern similarity (i.e., similarity to all other items in memory space). Anodal stimulation over the left LPFC specifically enhanced memory performance and item-specific STPS in the right frontal electrodes. These results support a causal role of LPFC in enhancing STPS and memory and contribute to a mechanistic understanding of memory formation.

Closed-loop training of attention with real-time brain imaging

Lapses of attention can have negative consequences, including accidents and lost productivity. Here we used closed-loop neurofeedback to improve sustained attention abilities and reduce the frequency of lapses. During a sustained attention task, the focus of attention was monitored in real time with multivariate pattern analysis of whole-brain neuroimaging data. When indicators of an attentional lapse were detected in the brain, we gave human participants feedback by making the task more difficult. Behavioral performance improved after one training session, relative to control participants who received feedback from other participants’ brains. This improvement was largest when feedback carried information from a frontoparietal attention network. A neural consequence of training was that the basal ganglia and ventral temporal cortex came to represent attentional states more distinctively. These findings suggest that attentional failures do not reflect an upper limit on cognitive potential and that attention can be trained with appropriate feedback about neural signals.

Attentional Enhancement of Auditory Mismatch Responses: a DCM/MEG Study

Despite similar behavioral effects, attention and expectation influence evoked responses differently: Attention typically enhances event-related responses, whereas expectation reduces them. This dissociation has been reconciled under predictive coding, where prediction errors are weighted by precision associated with attentional modulation. Here, we tested the predictive coding account of attention and expectation using magnetoencephalography and modeling. Temporal attention and sensory expectation were orthogonally manipulated in an auditory mismatch paradigm, revealing opposing effects on evoked response amplitude. Mismatch negativity (MMN) was enhanced by attention, speaking against its supposedly pre-attentive nature. This interaction effect was modeled in a canonical microcircuit using dynamic causal modeling, comparing models with modulation of extrinsic and intrinsic connectivity at different levels of the auditory hierarchy. While MMN was explained by recursive interplay of sensory predictions and prediction errors, attention was linked to the gain of inhibitory interneurons, consistent with its modulation of sensory precision.

Briefly cuing memories leads to suppression of their neural representations

Previous studies have linked partial memory activation with impaired subsequent memory retrieval (e.g., Detre et al., 2013) but have not provided an account of this phenomenon at the level of memory representations: How does partial activation change the neural pattern subsequently elicited when the memory is cued? To address this question, we conducted a functional magnetic resonance imaging (fMRI) experiment in which participants studied word-scene paired associates. Later, we weakly reactivated some memories by briefly presenting the cue word during a rapid serial visual presentation (RSVP) task; other memories were more strongly reactivated or not reactivated at all. We tested participants' memory for the paired associates before and after RSVP. Cues that were briefly presented during RSVP triggered reduced levels of scene activity on the post-RSVP memory test, relative to the other conditions. We used pattern similarity analysis to assess how representations changed as a function of the RSVP manipulation. For briefly cued pairs, we found that neural patterns elicited by the same cue on the pre- and post-RSVP tests (preA-postA; preB-postB) were less similar than neural patterns elicited by different cues (preA-postB; preB-postA). These similarity reductions were predicted by neural measures of memory activation during RSVP. Through simulation, we show that our pattern similarity results are consistent with a model in which partial memory activation triggers selective weakening of the strongest parts of the memory.

Similarity breeds proximity: pattern similarity within and across contexts is related to later mnemonic judgments of temporal proximity

Experiences unfold over time, but little is known about the mechanisms that support the formation of coherent episodic memories for temporally extended events. Recent work in animals has provided evidence for signals in hippocampus that could link events across temporal gaps; however, it is unknown whether and how such signals might be related to later memory for temporal information in humans. We measured patterns of fMRI BOLD activity as people encoded items that were separated in time and manipulated the presence of shared or distinct context across items. We found that hippocampal pattern similarity in the BOLD response across trials predicted later temporal memory decisions when context changed. By contrast, pattern similarity in lateral occipital cortex was related to memory only when context remained stable. These data provide evidence in humans that representational stability in hippocampus across time may be a mechanism for temporal memory organization.