Orienting attention to objects in visual short-term memory

Steady-state Visual Evoked Potentials Reveal Dynamic (Re)allocation of Spatial Attention during Maintenance and Utilization of Visual Working Memory

Visual working memory (VWM) allows storing goal-relevant information to guide future behavior. Prior work suggests that VWM is spatially organized and relies on spatial attention directed toward locations at which memory items were encoded, even if location is task-irrelevant. Importantly, attention often needs to be dynamically redistributed between locations, for example, in preparation for an upcoming probe. Very little is known about how attentional resources are distributed between multiple locations during a VWM task and even less about the dynamic changes governing such attentional shifts over time. This is largely due to the inability to use behavioral outcomes to reveal fast dynamic changes within trials. We here demonstrated that EEG steady-state visual evoked potentials (SSVEPs) successfully track the dynamic allocation of spatial attention during a VWM task. Participants were presented with to-be-memorized gratings and distractors at two distinct locations, tagged with flickering discs. This allowed us to dynamically track attention allocated to memory and distractor items via their coupling with space by quantifying the amplitude and coherence of SSVEP responses in the EEG signal to flickering stimuli at the former memory and distractor locations. SSVEP responses did not differ between memory and distractor locations during early maintenance. However, shortly before probe comparison, we observed a decrease in SSVEP coherence over distractor locations indicative of a reallocation of spatial attentional resources. RTs were shorter when preceded by stronger decreases in SSVEP coherence at distractor locations, likely reflecting attentional shifts from the distractor to the probe or memory location. We demonstrate that SSVEPs can inform about dynamic processes in VWM, even if location does not have to be reported by participants. This finding not only supports the notion of a spatially organized VWM but also reveals that SSVEPs betray a dynamic prioritization process of working memory items and locations over time that is directly predictive of memory performance.

Object-based visual working memory: an object benefit for equidistant memory items presented within simple contours

Previous research has shown that more information can be stored in visual working memory (VWM) when multiple items belong to the same object. Here, in four experiments, we investigated the object effect on memory for spatially equidistant features by manipulating simple, task-irrelevant contours that combined these features. In Experiments 1, 3, and, 4, three grating orientations, and in Experiment 2, one color and two orientations, were presented simultaneously to be memorized. Mixture modeling was applied to estimate both the precision and the guess rates of recall errors. Overall results showed that two target features were remembered more accurately when both were part of the same object. Further analysis showed that the probability of recall increased in particular when both features were extracted from the same object. In Experiment 2, we found that the object effect was greater for features from orthogonal dimensions, but this came at the cost of lower memory precision. In Experiment 3, when we kept the locations of the features perfectly consistent over trials so that the participants could attend to these locations rather than the contour, we still found object benefits. Finally, in Experiment 4 when we manipulated the temporal order of the object and the memory features presentations, it was confirmed that the object benefit is unlikely to stem from the strategical usage of object information. These results suggested that the object benefit arises automatically, likely at an early perceptual level.

Automatically binding relevant and irrelevant features in visual working memory

It is generally assumed that, to save storage space, features are stored as integrated objects in visual working memory (VWM). Although such an object-based account does not always hold because features can be processed in parallel, a previous study has shown that different features can be automatically bound with their locations (task-irrelevant feature) into an integrated unit, resulting in improved memory performance. The present study was designed to further explore this phenomenon by investigating whether other features, which are not spatial in origin, can act as the binding cue to form such automatic binding. To test this, we used three different features as binding cues (i.e., colour, spatial frequency, and shape) over multiple separate experiments. The results consistently showed that when two features shared the same binding cue, memory performance was better relative to when each of those features had their own binding cue. We conclude that any task-irrelevant feature can act as a binding cue to automatically bind with task-relevant features even across different objects, resulting in memory enhancement.

Individual Differences in Working Memory and the N2pc

The lateralized ERP N2pc component has been shown to be an effective marker of attentional object selection when elicited in a visual search task, specifically reflecting the selection of a target item among distractors. Moreover, when targets are known in advance, the visual search process is guided by representations of target features held in working memory at the time of search, thus guiding attention to objects with target-matching features. Previous studies have shown that manipulating working memory availability via concurrent tasks or within task manipulations influences visual search performance and the N2pc. Other studies have indicated that visual (non-spatial) vs. spatial working memory manipulations have differential contributions to visual search. To investigate this the current study assesses participants' visual and spatial working memory ability independent of the visual search task to determine whether such individual differences in working memory affect task performance and the N2pc. Participants (n = 205) completed a visual search task to elicit the N2pc and separate visual working memory (VWM) and spatial working memory (SPWM) assessments. Greater SPWM, but not VWM, ability is correlated with and predicts higher visual search accuracy and greater N2pc amplitudes. Neither VWM nor SPWM was related to N2pc latency. These results provide additional support to prior behavioral and neural visual search findings that spatial WM availability, whether as an ability of the participant's processing system or based on task demands, plays an important role in efficient visual search.

Memory-driven capture occurs for individual features of an object

Items held in working memory (WM) capture attention (memory-driven capture). People can selectively prioritize specific object features in WM. Here, we examined whether feature-specific prioritization within WM modulates memory-driven capture. In Experiment 1, after remembering the color and orientation of a triangle, participants were instructed, via retro-cue, whether the color, the orientation, or both features were relevant. To measure capture, we asked participants to execute a subsequent search task, and we compared performance in displays that did and did not contain the memory-matching feature. Color attracted attention only when it was relevant. No capture by orientation was found. In Experiment 2, we presented the retro-cue at one of the four locations of the search display to direct attention to specific objects. We found capture by color and this capture was larger when it was indicated as relevant. Crucially, orientation also attracted attention, but only when it was relevant. These findings provide evidence for reciprocal interaction between internal prioritization and external attention on the features level. Specifically, internal feature-specific prioritization modulates memory-driven capture but this capture also depends on the salience of the features.

Age-related changes in selection, recognition, updating and maintenance information in WM. An ERP study in children and adolescents

Possible age-related changes in different working memory (WM) subcomponents were assessed by analyzing the event-related-potentials associated with the n-back task. Two versions of the task (0- and 1-back) were administered to 168 subjects between 6 and 20 years of age. In both n-back tasks, lists of symbol-letter pairs were presented. Participants had to select the letter and decide whether it matched the target in memory. Selection-matching of the relevant item, as indexed by an N2pc component, was evident in all age groups, indicating early maturation of this ability. The decreasing amplitude of the P300 with age, coupled with the longer duration of the load effect in young children, suggests that WM updating requires greater processing resources at younger ages. The slow wave, present during the maintenance period, showed an inversion of polarity with age in anterior sites that could reflect age-related changes in the active maintenance of information in WM.

Adaptive Changes in the Dynamics of Visual Attention With Extended Practice

Previous research indicates that visual attention can adapt to temporal stimulus patterns utilizing the rapid serial visual presentation (RSVP) task. However, how the temporal dynamics of an attentional pulse adapt to temporal patterns has not been explored. We addressed this question by conducting an attentional component analysis on RSVP performance and explored whether changes in attentional dynamics were accompanied by explicit learning about predictable target timing. We utilized an RSVP task in which a target letter appeared either in two possible RSVP positions in fixed-timing conditions or in random positions over 1, 2, or 3 days of training. In a transfer phase, the target appeared in previously presented or new positions. Over 3 days of practice the target identification rate, efficacy, and precision of a putative attentional pulse increased. These changes reflected general learning in the RSVP task resulting in attentional dynamics more efficiently focused on the target. Although group performance effects did not support learning of fixed target positions, target identification rates and the measure of the efficacy of an attentional pulse at these positions were positively associated with explicit learning. The current study is the first to provide a detailed description of practice related adaptation of attentional dynamics and suggests that timing specific changes might be mediated by explicit temporal learning.

Towards Understanding the Task Dependency of Embodied Language Processing: The Influence of Colour During Language-Vision Interactions

A main challenge for theories of embodied cognition is to understand the task dependency of embodied language processing. One possibility is that perceptual representations (e.g., typical colour of objects mentioned in spoken sentences) are not activated routinely but the influence of perceptual representation emerges only when context strongly supports their involvement in language. To explore this question, we tested the effects of colour representations during language processing in three visual-world eye-tracking experiments. On critical trials, participants listened to sentence-embedded words associated with a prototypical colour (e.g., ‘…spinach…’) while they inspected a visual display with four printed words (Experiment 1), coloured or greyscale line drawings (Experiment 2) and a ‘blank screen’ after a preview of coloured or greyscale line drawings (Experiment 3). Visual context always presented a word/object (e.g., frog) associated with the same prototypical colour (e.g. green) as the spoken target word and three distractors. When hearing spinach participants did not prefer the written word frog compared to other distractor words (Experiment 1). In Experiment 2, colour competitors attracted more overt attention compared to average distractors, but only for the coloured condition and not for greyscale trials. Finally, when the display was removed at the onset of the sentence, and in contrast to the previous blank-screen experiments with semantic competitors, there was no evidence of colour competition in the eye-tracking record (Experiment 3). These results fit best with the notion that the main role of perceptual representations in language processing is to contextualize language in the immediate environment.

Get Set or Get Distracted? Disentangling Content-Priming and Attention-Catching Effects of Background Lure Stimuli on Identifying Targets in Two Simultaneously Presented Series

In order to study the changing relevance of stimulus features in time and space, we used a task with rapid serial presentation of two stimulus streams where two targets (“T1” and “T2”) had to be distinguished from background stimuli and where the difficult T2 distinction was impeded by background stimuli presented before T1 that resemble T2 (“lures”). Such lures might actually have dual characteristics: Their capturing attention might interfere with target identification, whereas their similarity to T2 might result in positive priming. To test this idea here, T2 was a blue digit among black letters, and lures resembled T2 either by alphanumeric category (black digits) or by salience (blue letters). Same-category lures were expected to prime T2 identification whereas salient lures would impede T2 identification. Results confirmed these predictions, yet the precise pattern of results did not fit our conceptual framework. To account for this pattern, we speculate that lures serve to confuse participants about the order of events, and the major factor distinguishing color lures and digit lures is their confusability with T2. Mechanisms of effects were additionally explored by measuring event-related EEG potentials. Consistent with the assumption that they attract more attention, color lures evoked larger N2pc than digit lures and affected the ensuing T1-evoked N2pc. T2-evoked N2pc was indistinguishably reduced by all kinds of preceding lures, though. Lure-evoked mesio-frontal negativity increased from first to third lures both with digit and color lures and, thereby, might have reflected expectancy for T1.

Human gaze tracks attentional focusing in memorized visual space

Brain areas that control gaze are also recruited for covert shifts of spatial attention1–9. In the external space of perception, there is a natural, ecological link between the control of gaze and of spatial attention, since information sampled at covertly attended locations can inform where to look next2,10,11. Attention can also be directed internally to representations held within the spatial layout of visual working memory12–16. In such cases, the incentive for using attention to direct gaze disappears since there are no external targets to scan. Here we investigate whether the brain’s oculomotor system also participates in attention within the internal space of memory. Paradoxically, we reveal this participation through gaze behaviour itself. We demonstrate that selecting an item from visual working memory biases gaze in the direction of the memorised location of that item – despite there being nothing to look at and even though location memory was never explicitly probed. This retrospective ‘gaze bias’ occurs only when an item is not already in the internal focus of attention, and predicts the performance benefit associated with the focusing of internal attention. We conclude that the oculomotor system also participates in the focusing of attention within memorised space, leaving traces all the way to the eyes.

A two-level hierarchical framework of visual short-term memory

Over the last couple of decades, a vast amount of research has been dedicated to understanding the nature and the architecture of visual short-term memory (VSTM), the mechanism by which currently relevant visual information is maintained. According to discrete-capacity models, VSTM is constrained by a limited number of discrete representations held simultaneously. In contrast, shared-resource models regard VSTM as limited in resources, which can be distributed flexibly between varying numbers of representations; and a new interference model posits that capacity is limited by interference among items. In this article, we begin by reviewing benchmark findings regarding the debate over VSTM limitations, focusing on whether VSTM storage is all-or-none and on whether object complexity affects capacity. After that, we put forward a hybrid framework of VSTM architecture, arguing that this system is composed of a two-level hierarchy of memory stores, each containing a different set of representations: (1) perceptual memory, a resourcelike level containing analog automatically formed representations of visual stimuli in varying degrees of activation, and (2) visual working memory, in which a subset of three to four items from perceptual memory are bound to conceptual representations and to their locations, thus conveying discrete (digital/symbolic) information which appears quantized. While perceptual memory has a large capacity and is relatively nonselective, visual working memory is restricted in the number of items that can be maintained simultaneously, and its content is regulated by a gating mechanism.

Separate and combined effects of action relevance and motivational value on visual working memory

Visual working memory contents can be selectively weighted according to differences in their task-relevance. In the present study, we examined the influence of two more indirect selection biases established by a concurrent task or learned reward associations: action relevance and motivational value. In three experiments, memory performance was assessed with the same color change detection task. Potential action relevance and motivational value were each determined by a specific feature of the memory items (location or shape, respectively) and manipulated orthogonally. Investigated separately (Experiments 1A and 1B), both selection biases modulated visual working memory. In combination (Experiment 2), action relevance and motivational value still each exerted an influence, but not in a fully independent and additive manner. While action relevance impacted performance irrespective of the reward associated with the items, an effect of motivational value was only observed for action-relevant items. These results support the notion that visual working memory is automatically biased as an inherent part of action planning. More generally, these findings highlight the versatile nature of visual working memory: Contents can be flexibly weighted to reflect differences in their importance, taking into account several sources of information.

On the Role of the Inferior Intraparietal Sulcus in Visual Working Memory for Lateralized Single-feature Objects

A consolidated practice in cognitive neuroscience is to explore the properties of human visual working memory through the analysis of electromagnetic signals using cued change detection tasks. Under these conditions, EEG/MEG activity increments in the posterior parietal cortex scaling with the number of memoranda are often reported in the hemisphere contralateral to the objects' position in the memory array. This highly replicable finding clashes with several reported failures to observe compatible hemodynamic activity modulations using fMRI or fNIRS in comparable tasks. Here, we reconcile this apparent discrepancy by acquiring fMRI data on healthy participants and employing a cluster analysis to group voxels in the posterior parietal cortex based on their functional response. The analysis identified two distinct subpopulations of voxels in the intraparietal sulcus (IPS) showing a consistent functional response among participants. One subpopulation, located in the superior IPS, showed a bilateral response to the number of objects coded in visual working memory. A different subpopulation, located in the inferior IPS, showed an increased unilateral response when the objects were displayed contralaterally. The results suggest that a cluster of neurons in the inferior IPS is a candidate source of electromagnetic contralateral responses to working memory load in cued change detection tasks.

The role of central attention in retrieval from visual short-term memory

The role of central attention in visual short-term memory (VSTM) encoding and maintenance is well established, yet its role in retrieval has been largely unexplored. This study examined the involvement of central attention in retrieval from VSTM using a dual-task paradigm. Participants performed a color change-detection task. Set size varied between 1 and 3 items, and the memory sample was maintained for either a short or a long delay period. A secondary tone discrimination task was introduced at the end of the delay period, shortly before the appearance of a central probe, and occupied central attention while participants were searching within VSTM representations. Similarly to numerous previous studies, reaction time increased as a function of set size reflecting the occurrence of a capacity-limited memory search. When the color targets were maintained over a short delay, memory was searched for the most part without the involvement of central attention. However, with a longer delay period, the search relied entirely on the operation of central attention. Taken together, this study demonstrates that central attention is involved in retrieval from VSTM, but the extent of its involvement depends on the duration of the delay period. Future studies will determine whether the type of memory search (parallel or serial) carried out during retrieval depends on the nature of the attentional mechanism involved the task.

The Focus of Attention in Visual Working Memory: Protection of Focused Representations and Its Individual Variation

Visual working memory can be modulated according to changes in the cued task relevance of maintained items. Here, we investigated the mechanisms underlying this modulation. In particular, we studied the consequences of attentional selection for selected and unselected items, and the role of individual differences in the efficiency with which attention is deployed. To this end, performance in a visual working memory task as well as the CDA/SPCN and the N2pc, ERP components associated with visual working memory and attentional processes, were analysed. Selection during the maintenance stage was manipulated by means of two successively presented retrocues providing spatial information as to which items were most likely to be tested. Results show that attentional selection serves to robustly protect relevant representations in the focus of attention while unselected representations which may become relevant again still remain available. Individuals with larger retrocueing benefits showed higher efficiency of attentional selection, as indicated by the N2pc, and showed stronger maintenance-associated activity (CDA/SPCN). The findings add to converging evidence that focused representations are protected, and highlight the flexibility of visual working memory, in which information can be weighted according its relevance.

Top-Down Activation of Spatiotopic Sensory Codes in Perceptual and Working Memory Search

A critical requirement of an efficient cognitive system is the selection and prioritization of relevant information. This occurs when selecting specific items from our sensory inputs, which then receive preferential status at subsequent levels of processing. Many everyday tasks also require us to select internal representations, such as a relevant item from memory. We show that both of these types of search are underpinned by the spatiotopic activation of sensory codes, using both fMRI and MEG data. When individuals searched for perceived and remembered targets, the MEG data highlighted a sensor level electrophysiological effect that reflects the contralateral organization of the visual system-namely, the N2pc. The fMRI data were used to identify a network of frontoparietal areas common to both types of search, as well as the early visual areas activated by the search display. We then combined fMRI and MEG data to explore the temporal dynamics of functional connections between the frontoparietal network and the early visual areas. Searching for a target item resulted in significantly enhanced phase-phase coupling between the frontoparietal network and the visual areas contralateral to the perceived or remembered location of that target. This enhancement of spatially specific phase-phase coupling occurred before the N2pc effect and was significantly associated with it on a trial-by-trial basis. The combination of these two imaging modalities suggests that perceptual and working memory search are underpinned by the synchronization of a frontoparietal network and the relevant sensory cortices.

EEG can Track the Time Course of Successful Reference Resolution in Small Visual Worlds

Previous research has shown that language comprehenders resolve reference quickly and incrementally, but not much is known about the neural processes and representations that are involved. Studies of visual short-term memory suggest that access to the representation of an item from a previously seen display is associated with a negative evoked potential at posterior electrodes contralateral to the spatial location of that item in the display. In this paper we demonstrate that resolving the reference of a noun phrase in a recently seen visual display is associated with an event-related potential that is analogous to this effect. Our design was adapted from the visual world paradigm: in each trial, participants saw a display containing three simple objects, followed by a question about the objects, such as Was the pink fish next to a boat?, presented word by word. Questions differed in whether the color adjective allowed the reader to identify the referent of the noun phrase or not (i.e., whether one or more objects of the named color were present). Consistent with our hypothesis, we observed that reference resolution by the adjective was associated with a negative evoked potential at posterior electrodes contralateral to spatial location of the referent, starting approximately 333 ms after the onset of the adjective. The fact that the laterality of the effect depended upon the location of the referent within the display suggests that reference resolution in visual domains involves, at some level, a modality-specific representation. In addition, the effect gives us an estimate of the time course of processing from perception of the written word to the point at which its meaning is brought into correspondence with the referential domain.

Electrophysiological Evidence of a Delay in the Visual Recognition Process in Young Children

The present study analyzes the development of the visual recognition processing of the relevant stimulus in a Delayed Match-To-Sample (DMS) task during the matching phase. To do so, Electroencephalograms of 170 subjects between 6 and 26 years old were recorded. Behavioral responses and Event Related Potentials (ERPs) induced by the stimuli were obtained. Reaction times and errors, mainly omissions, were inversely related to age. The ERPs analysis showed a parietal negativity in the P7 and P8 electrodes when the relevant stimulus was presented in the contralateral site. This negativity resulting from the recognition and selection of the relevant stimulus was present in all age groups. However, the youngest children showed an extended latency in the recognition process. The results suggest that children and adults use similar processes to recognize the item maintained in visual short-term memory (VSTM), but children need more time to successfully recognize the memorized item.

ERP markers of target selection discriminate children with high vs. low working memory capacity

Selective attention enables enhancing a subset out of multiple competing items to maximize the capacity of our limited visual working memory (VWM) system. Multiple behavioral and electrophysiological studies have revealed the cognitive and neural mechanisms supporting adults’ selective attention of visual percepts for encoding in VWM. However, research on children is more limited. What are the neural mechanisms involved in children’s selection of incoming percepts in service of VWM? Do these differ from the ones subserving adults’ selection? Ten-year-olds and adults used a spatial arrow cue to select a colored item for later recognition from an array of four colored items. The temporal dynamics of selection were investigated through EEG signals locked to the onset of the memory array. Both children and adults elicited significantly more negative activity over posterior scalp locations contralateral to the item to-be-selected for encoding (N2pc). However, this activity was elicited later and for longer in children compared to adults. Furthermore, although children as a group did not elicit a significant N2pc during the time-window in which N2pc was elicited in adults, the magnitude of N2pc during the “adult time-window” related to their behavioral performance during the later recognition phase of the task. This in turn highlights how children’s neural activity subserving attention during encoding relates to better subsequent VWM performance. Significant differences were observed when children were divided into groups of high vs. low VWM capacity as a function of cueing benefit. Children with large cue benefits in VWM capacity elicited an adult-like contralateral negativity following attentional selection of the to-be-encoded item, whereas children with low VWM capacity did not. These results corroborate the close coupling between selective attention and VWM from childhood and elucidate further the attentional mechanisms constraining VWM performance in children.

Searching for targets in visual working memory: investigating a dimensional feature bundle (DFB) model

The human visual working memory (WM) system enables us to store a limited amount of task-relevant visual information temporally in mind. One actively debated issue in cognitive neuroscience centers around the question of how this WM information is maintained. The currently dominant views advocated by prominent WM models hold that the units of memory are configured either as independent feature representations, integrated bound objects, or a combination of both. Here, we approached this issue by measuring lateralized brain electrical activity during a retro-cue paradigm, in order to track people's ability to access WM representations as a function of the dimensional relation between WM items and task settings. Both factors were revealed to selectively influence WM access: whereas cross relative to intradimensional WM targets gave rise to enhanced contralateral delay activity (CDA) amplitudes, localization relative to identification task demands yielded speeded CDA and manual response times. As these dimension-based findings are not reconcilable with contemporary feature- and/or object-based accounts, an alternative view that is based on the hierarchical feature-bundle model is proposed. We argue that WM units may consist of three hierarchically structured levels of representations, with an intermediate dimensionally organized level that mediates between top-level object and lower-level feature representations.

Consolidation time affects performance and neural activity during visual working memory

We tested the effects of variation of stimulus onset asynchrony (SOA) on visual working memory (WM) performance across different load levels and the underlying brain activation patterns using functional magnetic resonance imaging (fMRI) in 48 healthy participants. Participants were instructed to memorise arrays of coloured squares and had to perform a match/non-match judgement on a probe stimulus after a jittered delay. We presented visual pattern masks at four SOAs after the offset of the memory array (100 ms, 200 ms, 400 ms, and 800 ms). Memory performance decreased with increased load and shortened SOA. Brain activation data showed significant effects of load (during encoding and retrieval), SOA (retrieval) and an interaction of load by SOA (encoding), mainly in frontal and parietal areas. There was also a direct relationship between successfully stored items and activation in the right inferior parietal lobule and the left middle frontal gyrus. The neurobehavioral results suggest that the frontal regions, together with the inferior parietal lobe, are associated with successful WM performance, especially under the most challenging conditions of high load and short SOAs.

Temporal dynamics of attention during encoding versus maintenance of working memory: complementary views from event-related potentials and alpha-band oscillations

Working memory (WM) is strongly influenced by attention. In visual WM tasks, recall performance can be improved by an attention-guiding cue presented before encoding (precue) or during maintenance (retrocue). Although precues and retrocues recruit a similar frontoparietal control network, the two are likely to exhibit some processing differences, because precues invite anticipation of upcoming information whereas retrocues may guide prioritization, protection, and selection of information already in mind. Here we explored the behavioral and electrophysiological differences between precueing and retrocueing in a new visual WM task designed to permit a direct comparison between cueing conditions. We found marked differences in ERP profiles between the precue and retrocue conditions. In line with precues primarily generating an anticipatory shift of attention toward the location of an upcoming item, we found a robust lateralization in late cue-evoked potentials associated with target anticipation. Retrocues elicited a different pattern of ERPs that was compatible with an early selection mechanism, but not with stimulus anticipation. In contrast to the distinct ERP patterns, alpha-band (8-14 Hz) lateralization was indistinguishable between cue types (reflecting, in both conditions, the location of the cued item). We speculate that, whereas alpha-band lateralization after a precue is likely to enable anticipatory attention, lateralization after a retrocue may instead enable the controlled spatiotopic access to recently encoded visual information.

Contralateral delay activity reveals dimension-based attentional orienting to locations in visual working memory

In research on visual working memory (WM), a contentiously debated issue concerns whether or not items are stored independently of one another in WM. Here we addressed this issue by exploring the role of the physical context that surrounds a given item in the memory display in the formation of WM representations. In particular, we employed bilateral memory displays that contained two or three lateralized singleton items (together with six or five distractor items), defined either within the same or in different visual feature dimensions. After a variable interval, a retro-cue was presented centrally, requiring participants to discern the presence (vs. the absence) of this item in the previously shown memory array. Our results show that search for targets in visual WM is determined interactively by dimensional context and set size: For larger, but not smaller, set sizes, memory search slowed down when targets were defined across rather than within dimensions. This dimension-specific cost manifested in a stronger contralateral delay activity component, an established neural marker of the access to WM representations. Overall, our findings provide electrophysiological evidence for the hierarchically structured nature of WM representations, and they appear inconsistent with the view that WM items are encoded in isolation.

Modulation of alpha and gamma oscillations related to retrospectively orienting attention within working memory

Selective attention mechanisms allow us to focus on information that is relevant to the current behavior and, equally important, ignore irrelevant information. An influential model proposes that oscillatory neural activity in the alpha band serves as an active functional inhibitory mechanism. Recent studies have shown that, in the same way that attention can be selectively oriented to bias sensory processing in favor of relevant stimuli in perceptual tasks, it is also possible to retrospectively orient attention to internal representations held in working memory. However, these studies have not explored the associated oscillatory phenomena. In the current study, we analysed the patterns of neural oscillatory activity recorded with magnetoencephalography while participants performed a change detection task, in which a spatial retro-cue was presented during the maintenance period, indicating which item or items were relevant for subsequent retrieval. Participants benefited from retro-cues in terms of accuracy and reaction time. Retro-cues also modulated oscillatory activity in the alpha and gamma frequency bands. We observed greater alpha activity in a ventral visual region ipsilateral to the attended hemifield, thus supporting its suppressive role, i.e. a functional disengagement of task-irrelevant regions. Accompanying this modulation, we found an increase in gamma activity contralateral to the attended hemifield, which could reflect attentional orienting and selective processing. These findings suggest that the oscillatory mechanisms underlying attentional orienting to representations held in working memory are similar to those engaged when attention is oriented in the perceptual space.

The Temporal Cascade of Neural Processes Underlying Target Detection and Attentional Processing During Auditory Search

The posterior visual event-related potential (ERP) component, the N2pc, has been widely used to study lateralized shifts of attention within visual arrays. Recently, Gamble and Luck (2011) reported an auditory analog of this activity (the fronto-central "N2ac"), reflecting the lateralized focusing of attention toward a Target sound among 2 simultaneous auditory stimuli. Here, we directed an electrophysiological approach toward understanding auditory Target search within a more complex auditory environment in which rapidly occurring sounds were distributed across both time and space. Trials consisted of ten 40-ms monaural sounds rapidly presented to the 2 ears: 8 medium-pitch tones and 2 deviant sounds (one high and one low). For each block, one deviant type was designated as the Target, which participants needed to identify within each trial to discriminate its tonal quality. The extracted electrophysiological results included a very early enhancement, starting at approximately 50 ms, of a bilateral negative-polarity auditory brain response to the designated Target Deviant (compared with the Nontarget Deviant), followed at approximately 130 ms by the N2ac activity reflecting the lateralized focusing of attention toward that Target. The results delineate the tightly orchestrated sequence of neural processes underlying the detection of, and focusing of attention toward, Target sounds in complex auditory scenes.

Age Group and Individual Differences in Attentional Orienting Dissociate Neural Mechanisms of Encoding and Maintenance in Visual STM

Selective attention biases the encoding and maintenance of representations in visual STM (VSTM). However, precise attentional mechanisms gating encoding and maintenance in VSTM and across development remain less well understood. We recorded EEG while adults and 10-year-olds used cues to guide attention before encoding or while maintaining items in VSTM. Known neural markers of spatial orienting to incoming percepts, that is, Early Directing Attention Negativity, Anterior Directing Attention Negativity, and Late Directing Attention Positivity, were examined in the context of orienting within VSTM. Adults elicited a set of neural markers that were broadly similar in preparation for encoding and during maintenance. In contrast, in children these processes dissociated. Furthermore, in children, individual differences in the amplitude of neural markers of prospective orienting related to individual differences in VSTM capacity, suggesting that children with high capacity are more efficient at selecting information for encoding into VSTM. Finally, retrospective, but not prospective, orienting in both age groups elicited the well-known marker of visual search (N2pc), indicating the recruitment of additional neural circuits when orienting during maintenance. Developmental and individual differences differentiate seemingly similar processes of orienting to perceptually available representations and to representations held in VSTM.

Deployment of spatial attention towards locations in memory representations. An EEG study

Recalling information from visual short-term memory (VSTM) involves the same neural mechanisms as attending to an actually perceived scene. In particular, retrieval from VSTM has been associated with orienting of visual attention towards a location within a spatially-organized memory representation. However, an open question concerns whether spatial attention is also recruited during VSTM retrieval even when performing the task does not require access to spatial coordinates of items in the memorized scene. The present study combined a visual search task with a modified, delayed central probe protocol, together with EEG analysis, to answer this question. We found a temporal contralateral negativity (TCN) elicited by a centrally presented go-signal which was spatially uninformative and featurally unrelated to the search target and informed participants only about a response key that they had to press to indicate a prepared target-present vs. -absent decision. This lateralization during VSTM retrieval (TCN) provides strong evidence of a shift of attention towards the target location in the memory representation, which occurred despite the fact that the present task required no spatial (or featural) information from the search to be encoded, maintained, and retrieved to produce the correct response and that the go-signal did not itself specify any information relating to the location and defining feature of the target.

The retention of simultaneous tones in auditory short-term memory: a magnetoencephalography study

We used magnetoencephalography (MEG) to localize brain activity related to the retention of tones differing in pitch. Participants retained one or two simultaneously presented tones. After a two second interval a test tone was presented and the task was to determine if that tone was in memory. We focused on brain activity during the retention interval that increased as the number of sounds retained in auditory short-term memory (ASTM) increased. Source analyses revealed that the superior temporal gyrus in both hemispheres is involved in ASTM. In the right hemisphere, the inferior temporal gyrus, the inferior frontal gyrus, and parietal structures also play a role. Our method provides good spatial and temporal resolution for investigating neuronal correlates of ASTM and, as it is the first MEG study using a memory load manipulation without using sequences of tones, it allowed us to isolate brain regions that most likely reflect the simple retention of tones.

N1pc reversal following repeated eccentric visual stimulation

Early event-related potential (ERP) hemispheric asymmetries recorded at occipitoparietal sites are usually observed following the sudden onset of a lateral peripheral stimulus. This is usually reflected in an onset-locked larger N1 over the posterior contralateral hemisphere relative to the ipsilateral hemisphere, an early ERP asymmetry labeled N1pc. When the peripheral sudden onset is followed by a central stimulus, or by a bilaterally balanced visual array of stimuli, these events evoke a reversed N1pc, that is, a larger N1 over the hemisphere ipsilateral to the peripheral sudden onset. This N1pc reversal has been taken as evidence for a remapping of the visual space from an absolute, retinally based frame of reference to a relative, attentionally based frame of reference that codes the spatial positions of objects relative to the peripheral sudden onset, rather than relative to the fovea. Here, we pit the reference frame-remapping account against an alternative account based on reduced neural reactivity following the peripheral sudden onset. In three experiments, we varied the spatial location of an object relative to a preceding sudden onset, and tested the opposite predictions generated by the frame-remapping and the reduced neural reactivity accounts. Taken together, the results from the present experiments were consistent with the reduced neural reactivity account and inconsistent with the frame-remapping account.

The mental wormhole: internal attention shifts without regard for distance

Attention operates perceptually on items in the environment, and internally on objects in visuospatial working memory. In the present study, we investigated whether spatial and temporal constraints affecting endogenous perceptual attention extend to internal attention. A retro-cue paradigm in which a cue is presented beyond the range of iconic memory and after stimulus encoding was used to manipulate shifts of internal attention. Participants' memories were tested for colored circles (Experiments 1, 2, 3a, 4) or for novel shapes (Experiment 3b) and their locations within an array. In these experiments, the time to shift internal attention (Experiments 1 and 3) and the eccentricity of encoded objects (Experiments 2-4) were manipulated. Our data showed that, unlike endogenous perceptual attention, internal shifts of attention are not modulated by stimulus eccentricity. Across several timing parameters and stimuli, we found that shifts of internal attention require a minimum quantal amount of time regardless of the object eccentricity at encoding. Our findings are consistent with the view that internal attention operates on objects whose spatial information is represented in relative terms. Although endogenous perceptual attention abides by the laws of space and time, internal attention can shift across spatial representations without regard for physical distance.

Differential frontal involvement in shifts of internal and perceptual attention

BACKGROUND

Perceptual attention enhances the processing of items in the environment, whereas internal attention enhances processing of items encoded in visual working memory. In perceptual and internal attention cueing paradigms, cues indicate the to-be-probed item before (pre-cueing) or after (retro-cueing) the memory display, respectively. Pre- and retro-cues confer similar behavioral accuracy benefits (pre-: 14-19%, retro-: 11-17%) and neuroimaging data show that they activate overlapping frontoparietal networks. Yet reports of behavioral and neuroimaging differences suggest that pre- and retro-cueing differentially recruit frontal and parietal cortices (Lepsien and Nobre, 2006).

OBJECTIVE/HYPOTHESIS

This study examined whether perceptual and internal attention are equally disrupted by neurostimulation to frontal and parietal cortices. We hypothesized that neurostimulation applied to frontal cortex would disrupt internal attention to a greater extent than perceptual attention.

METHODS

Cathodal transcranial direct current stimulation (tDCS) was applied to frontal or parietal cortices. After stimulation, participants completed a change detection task coupled with either pre- or retro-cues.

RESULTS

Cathodal tDCS across site (frontal, parietal) hindered performance. However, frontal tDCS had a greater negative impact on the retro-cued trials demonstrating greater frontal involvement during shifts of internal attention.

CONCLUSIONS

These results complement the neuroimaging data and provide further evidence suggesting that perceptual and internal attention are not identical processes. We conclude that although internal and perceptual attention are mediated by similar frontoparietal networks, the weight of contribution of these structures differs, with internal attention relying more heavily on the frontal cortex.

Nonspatial Cueing of Tactile STM Causes Shift of Spatial Attention

The focus of attention can be flexibly altered in mnemonic representations of past sensory events. We investigated the neural mechanisms of selection in tactile STM by applying vibrotactile sample stimuli of different intensities to both hands, followed by a symmetrically shaped visual retro-cue. The retro-cue indicated whether the weak or strong sample was relevant for subsequent comparison with a single tactile test stimulus. Locations of tactile stimuli were randomized, and the required response did not depend upon the spatial relation between cued sample and test stimulus. Selection between spatially segregated items in tactile STM was mirrored in lateralized activity following visual retro-cues (N2pc) and influenced encoding of task-irrelevant tactile probe stimuli (N140). Our findings support four major conclusions. First, retrospective selection results in transient shifts of spatial attention. Second, retrospective selection is functionally dissociable from attention-based rehearsal of locations. Third, selection mechanisms are linked across processing stages, as attention shifts in STM influence encoding of sensory signals. Fourth, selection in tactile STM recruits attentional control mechanisms that are, at least partially, supramodal.

Frontoparietal theta activity supports behavioral decisions in movement-target selection

There is recent EEG evidence describing task-related changes of theta power in spatial attention and reaching/pointing tasks. Here, we aim to better characterize this theta activity and determine whether it is associated with visuospatial memory or with visuospatial selection functions of the frontoparietal cortex. We recorded EEG from 20 participants during a movement precuing task with center-out joystick movements. Precues displayed 1, 2, or 4 potential targets and were followed (stimulus onset asynchrony 1.2 s) by a central response cue indicating the movement-target. Remembering the precued target location(s) was mandatory in one and optional in a second version of the task. Analyses evaluated two slow brain potentials (CNV, contingent negative variation and CDA, contralateral delay activity) and task-related power changes. Results showed a differential modulation of frontal CNV and parietal CDA, consistent with earlier described set-size effects on motor preparation and visual short-term memory. Short-lived phases of theta event-related synchronization (ERS) were found 150-500 ms after precue and response cue presentation, exhibiting parietal and frontal maxima. The increase of frontoparietal theta power following response cue presentation was strongly modulated by target load, i.e., absent for 1-target (when the movement-target could be selected in advance), contrasting with a robust 20-50% ERS response in 2- and 4-target conditions. The scalp distribution, the timing, and the modulation by set-size suggest a role of theta activity in movement-target selection. The results support a recently proposed view of theta as emerging around behavioral decision points, linked to the evaluation of choice-relevant information.

Attention modulates maintenance of representations in visual short-term memory

Recent studies have shown that selective attention is of considerable importance for encoding task-relevant items into visual short-term memory (VSTM) according to our behavioral goals. However, it is not known whether top-down attentional biases can continue to operate during the maintenance period of VSTM. We used ERPs to investigate this question across two experiments. Specifically, we tested whether orienting attention to a given spatial location within a VSTM representation resulted in modulation of the contralateral delay activity (CDA), a lateralized ERP marker of VSTM maintenance generated when participants selectively encode memory items from one hemifield. In both experiments, retrospective cues during the maintenance period could predict a specific item (spatial retrocue) or multiple items (neutral retrocue) that would be probed at the end of the memory delay. Our results revealed that VSTM performance is significantly improved by orienting attention to the location of a task-relevant item. The behavioral benefit was accompanied by modulation of neural activity involved in VSTM maintenance. Spatial retrocues reduced the magnitude of the CDA, consistent with a reduction in memory load. Our results provide direct evidence that top-down control modulates neural activity associated with maintenance in VSTM, biasing competition in favor of the task-relevant information.

Top-down modulation: bridging selective attention and working memory

Selective attention, the ability to focus our cognitive resources on information relevant to our goals, influences working memory (WM) performance. Indeed, attention and working memory are increasingly viewed as overlapping constructs. Here, we review recent evidence from human neurophysiological studies demonstrating that top-down modulation serves as a common neural mechanism underlying these two cognitive operations. The core features include activity modulation in stimulus-selective sensory cortices with concurrent engagement of prefrontal and parietal control regions that function as sources of top-down signals. Notably, top-down modulation is engaged during both stimulus-present and stimulus-absent stages of WM tasks; that is, expectation of an ensuing stimulus to be remembered, selection and encoding of stimuli, maintenance of relevant information in mind and memory retrieval.

Biasing perception by spatial long-term memory

Human perception is highly flexible and adaptive. Selective processing is tuned dynamically according to current task goals and expectations to optimize behavior. Arguably, the major source of our expectations about events yet to unfold is our past experience; however, the ability of long-term memories to bias early perceptual analysis has remained untested. We used a noninvasive method with high temporal resolution to record neural activity while human participants detected visual targets that appeared at remembered versus novel locations within naturalistic visual scenes. Upon viewing a familiar scene, spatial memories changed oscillatory brain activity in anticipation of the target location. Memory also enhanced neural activity during early stages of visual analysis of the target and improved behavioral performance. Both measures correlated with subsequent target-detection performance. We therefore demonstrated that memory can directly enhance perceptual functions in the human brain.

Shifting attention among working memory representations: testing cue type, awareness, and strategic control

It is well known that visual working memory (VWM) performance is modulated by attentional cues presented during encoding. Interestingly, retrospective cues presented after encoding, but prior to the test phase also improve performance. This improvement in performance is termed the retro-cue benefit. We investigated whether the retro-cue benefit is sensitive to cue type, whether participants were aware of their improvement in performance due to the retro-cue, and whether the effect was under strategic control. Experiment 1 compared the potential cueing benefits of abrupt onset retro-cues relying on bottom-up attention, number retro-cues relying on top-down attention, and arrow retro-cues, relying on a mixture of both. We found a significant retro-cue effect only for arrow retro-cues. In Experiment 2, we tested participants' awareness of their use of the informative retro-cue and found that they were aware of their improved performance. In Experiment 3, we asked whether participants have strategic control over the retro-cue. The retro-cue was difficult to ignore, suggesting that strategic control is low. The retro-cue effect appears to be within conscious awareness but not under full strategic control.

Recoding between two types of STM representation revealed by the dynamics of memory search

Visual STM (VSTM) is thought to be related to visual attention in several ways. Attention controls access to VSTM during memory encoding and plays a role in the maintenance of stored information by strengthening memorized content. We investigated the involvement of visual attention in recall from VSTM. In two experiments, we measured electrophysiological markers of attention in a memory search task with varying intervals between VSTM encoding and recall, and so we were able to track recoding of representations in memory. Results confirmed the involvement of attention in VSTM recall. However, the amplitude of the N2pc and N3rs components, which mark orienting of attention and search within VSTM, decreased as a function of delay. Conversely, the amplitude of the P3 and sustained posterior contralateral negativity components increased as a function of delay, effectively the opposite of the N2pc and N3rs modulations. These effects were only observed when verbal memory was not taxed. Thus, the results suggested that gradual recoding from visuospatial orienting of attention into verbal recall mechanisms takes place from short to long retention intervals. Interestingly, recall at longer delays was faster than at short delays, indicating that verbal representation is coupled with faster responses. These results extend the orienting-of-attention hypothesis by including an account of representational recoding during short-term consolidation and its consequences for recall from VSTM.

Look out for strangers! Sustained neural activity during visual working memory maintenance of other-race faces is modulated by implicit racial prejudice

We tested the ability of white participants to encode and retain over a brief period of time information about the identity of white and black people, using faces as stimuli in a standard change detection task and tracking neural activity using electroencephalography. Neural responses recorded over the posterior parietal cortex reflecting visual working memory activity increased in amplitude as a function of the number of faces that had to be maintained in memory. Critically, these memory-related neural responses varied as a function of participants' implicit racial prejudice toward black people. High-prejudiced participants encoded black people faces with a lower degree of precision compared to low-prejudiced participants, suggesting that the class of mental operations affected by implicit racial prejudice includes basic cognitive mechanisms underpinning the encoding and maintenance of faces' visual representations in visual working memory.

Spatial layout of letters in nonwords affects visual short-term memory load: evidence from human electrophysiology

The sustained posterior contralateral negativity (SPCN) was used to investigate the effect of spatial layout on the maintenance of letters in VSTM. SPCN amplitude was measured for words, nonwords, and scrambled nonwords. We reexamined the effects of spatial layout of letters on SPCN amplitude in a design that equated the mean frequency of use of each position. Scrambled letters that did not form words elicited a larger SPCN than either words or nonwords, indicating lower VSTM load for nonwords presented in a typical horizontal array than the load observed for the same letters presented in spatially scrambled locations. In contrast, prior research has shown that the spatial extent of arrays of simple stimuli did not influence the amplitude of the SPCN. Thus, the present results indicate the existence of encoding and VSTM maintenance mechanisms specific to letter and word processing.