The time course of protecting a visual memory representation from perceptual interference

The differential impact of face distractors on visual working memory across encoding and delay stages

External distractions often occur when information must be retained in visual working memory (VWM)-a crucial element in cognitive processing and everyday activities. However, the distraction effects can differ if they occur during the encoding rather than the delay stages. Previous research on these effects used simple stimuli (e.g., color and orientation) rather than considering distractions caused by real-world stimuli on VWM. In the present study, participants performed a facial VWM task under different distraction conditions across the encoding and delay stages to elucidate the mechanisms of distraction resistance in the context of complex real-world stimuli. VWM performance was significantly impaired by delay-stage but not encoding-stage distractors (Experiment 1). In addition, the delay distraction effect arose primarily due to the absence of distractor process at the encoding stage rather than the presence of a distractor during the delay stage (Experiment 2). Finally, the impairment in the delay-distraction condition was not due to the abrupt appearance of distractors (Experiment 3). Taken together, these findings indicate that the processing mechanisms previously established for resisting distractions in VWM using simple stimuli can be extended to more complex real-world stimuli, such as faces.

Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.

Bend but don't break: Prioritization protects working memory from displacement but leaves it vulnerable to distortion from distraction

Perceptual distraction distorts visual working memory representations. Previous research has shown that memory responses are systematically biased towards passively viewed visual distractors that are similar to the memoranda. However, it remains unclear whether the prioritization of one working memory representation over another reduces the impact of perceptual distractors. We designed a study with five different types of visual distraction that varied in engagement and found evidence for both subtle distortions and catastrophic failures of memory. Importantly, prioritization protected working memories from catastrophic loss (fewer "swap errors") but rendered them more vulnerable to distortion (greater attractive "biases" towards the distractor). Our findings demonstrate that prioritization does not simply protect working memory from any and all interference, but rather it reduces the likelihood of catastrophic disruption from perceptual distraction at the cost of an increased likelihood of distortion.

Selection within working memory impairs perceptual detection

There is broad consensus supporting the reciprocal influence of working memory (WM) and attention. Top-down mechanisms operate to cope with either environmental or internal demands. In that sense, it is possible to select an item within the contents of WM to endow it with prioritized access. Although evidence supports that maintaining an item in this privileged state does not rely on sustained visual attention, it is unknown whether selection within WM depends on perceptual attention. To answer this question, we recorded electrophysiological neural activity while participants performed a retro-cue task in which we inserted a detection task in the delay period after retro-cue presentation. Critically, the onset of to-be-detected near threshold stimuli was unpredictable, and thus, sustained perceptual spatial attention was needed to accomplish the detection task from the offset of the retro-cue. At a behavioral level, we found decreased visual detection when a WM representation was retro-cued. At a neural level, alpha oscillatory activity confirmed a spatial shift of attention to the retro-cued representation. We interpret the convergence of neural oscillations and behavioral data to point towards the theory that selection within WM could be accomplished through a perceptual attentional mechanism.

Prioritization sharpens working memories but does not protect them from distraction

Perceptual distraction distorts visual working memory representations. Previous research has shown that memory responses are systematically biased toward visual distractors that are similar to the memoranda. However, it remains unclear whether the prioritization of one working memory representation over another reduces the impact of perceptual distractors. In five behavioral experiments, we used different forms of retrospective cues (indicating the likelihood of testing each item and/or the reward for responding correctly to each item) to manipulate the prioritization of items in working memory before visual distraction. We examined the effects of distraction with nonparametric analyses and a novel distractor intrusion model. We found that memory responses were more precise (lower absolute response errors and stronger memory signals) for items that were prioritized. However, these prioritized items were not immune to distraction, and their memory responses were biased toward the visual distractors to the same degree as were unprioritized items. Our findings demonstrate that the benefits associated with prioritization in working memory do not include protection from distraction biases. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Turning Attention Inside Out: How Working Memory Serves Behavior

Flexible behavior requires guidance not only by sensations that are available immediately but also by relevant mental contents carried forward through working memory. Therefore, selective-attention functions that modulate the contents of working memory to guide behavior (inside-out) are just as important as those operating on sensory signals to generate internal contents (outside-in). We review the burgeoning literature on selective attention in the inside-out direction and underscore its functional, flexible, and future-focused nature. We discuss in turn the purpose (why), targets (what), sources (when), and mechanisms (how) of selective attention inside working memory, using visual working memory as a model. We show how the study of internal selective attention brings new insights concerning the core cognitive processes of attention and working memory and how considering selective attention and working memory together paves the way for a rich and integrated understanding of how mind serves behavior.

Internal attention is the only retroactive mechanism for controlling precision in working memory

Recent research has suggested that humans can assert control over the precision of working memory (WM) items. However, the mechanisms that enable this control are unclear. While some studies suggest that internal attention improves precision, it may not be the only factor, as previous work also demonstrated that WM storage is disentangled from attention. To test whether there is a precision control mechanism beyond internal attention, we contrasted internal attention and precision requirements within the same trial in three experiments. In every trial, participants memorized two items briefly. Before the test, a retro-cue indicated which item would be tested first, thus should be attended. Importantly, we encouraged participants to store the unattended item with higher precision by testing it using more similar lure colors at the probe display. Accuracy was analyzed on a small proportion of trials where the target-lure similarity, hence the task difficulty, was equal for attended and unattended items. Experiments 2 and 3 controlled for output interference by the first test and involuntary precision boost by the retro-cue, respectively. In all experiments, the unattended item had lower accuracy than the attended item, suggesting that individuals were not able to remember it more precisely than the attended item. Thus, we conclude that there is no precision control mechanism beyond internal attention, highlighting the close relationship between attentional and qualitative prioritization within WM. We discuss the important implications of these findings for our understanding of the fundamentals of WM and WM-driven behaviors.

Modulation of similarity on the distraction resistance of visual working memory representation

Controversy exists regarding the distraction resistance of priority items in visual working memory (VWM). The protection account proposes that high-priority items resist distraction more than low-priority items, while the vulnerability account proposes that distractors impair high-priority items more. We proposed another available resource threshold account: distraction will not impair items if available resources exceed their threshold needed for sufficient processing. Using a change-detection paradigm, we manipulated item priority by color similarity and inserted distractors during VWM retention. We investigated the effect of similarity on distraction resistance of relevant (color) and irrelevant (shape) feature representations (Experiments 1-2), and the neural mechanism of this effect using event-related potentials (ERPs; Experiment 3). Behavioral results showed distractors impaired the accuracy of dissimilar items when relevant features were memorized and of similar items when irrelevant features were memorized under simultaneous presentation of similar and dissimilar items. Moreover, distractors impaired the accuracy of dissimilar items when relevant features were memorized and of both similar and dissimilar items when irrelevant features were memorized under separate presentation of items. ERP results showed a smaller negative slow wave and P2 but larger N2 under the similar condition. Similarity protected relevant features of similar items against distraction by reducing memory load, decreasing attentional resources allocated to distractors, and strengthening inhibition of distractors. However, similarity did not protect irrelevant features of similar items. Our results support the available resource threshold account, suggesting that VWM is a flexible and intelligent system despite its limited capacity.

Tracking attentional states: Assessing the relationship between sustained and selective focused attention in visual working memory

Attention has multiple influences on visual working memory (VWM). Fluctuations in sustained attention predict VWM performance. Furthermore, focusing selective attention in VWM by retro-cuing the to-be-tested item during maintenance boosts retrieval. So far, we lack knowledge how the ability to focus selective attention relates to the state of sustained attention during the VWM trial. Here, we combined a retro-cue task and a self-rated attention protocol to test whether focusing selective attention via retro-cues: (1) mitigates spontaneous attention fluctuations, in which case retro-cues should be more helpful under low levels of self-rated attention; (2) depends on an optimal state of sustained attention, in which case retro-cue benefits should be largest under high levels of self-rated attention; or (3) is independent of sustained attention, in which case retro-cue benefits and self-rated attention effects should be additive. Our data supported the additive hypothesis. Across four experiments, self-rated attention levels predicted continuous reproduction of colors. Retro-cue trials produced better recall and higher rated attention. Critically, retro-cues improved recall to a similar extent across all levels of self-rated attention. This indicates that attention has multi-faceted and independent contributions to VWM.

Disruption of working memory and contralateral delay activity by nociceptive stimuli is modulated by task demands

ABSTRACT

Top-down processes allow the selection and prioritization of information by limiting attentional capture by distractors, and these mechanisms depend on task demands such as working memory (WM) load. However, bottom-up processes give salient stimuli a stronger neuronal representation and provoke attentional capture. The aim of this study was to examine the effect of salient nociceptive stimuli on WM while manipulating task demands. Twenty-one healthy participants performed a change detection task during which they had to determine whether 2 successive visual arrays were different or the same. Task demands were modulated by manipulating the WM load (set size included 2 or 4 objects to recall) and by the correspondence between the 2 successive visual arrays (change vs no change). Innocuous stimuli (control) or nociceptive stimuli (distractors) were delivered during the delay period between the 2 visual arrays. Contralateral delay activity and laser-evoked potentials were recorded to examine neural markers of visual WM and nociceptive processes. Nociceptive stimuli decreased WM performance depending on task demands (all P < 0.05). Moreover, compared with control stimuli, nociceptive stimuli abolished the increase in contralateral delay activity amplitude for set size 4 vs set size 2 (P = 0.04). Consistent with these results, laser-evoked potential amplitude was not decreased when task demands were high (P = 0.5). These findings indicate that WM may shield cognition from nociceptive stimuli, but nociceptive stimuli disrupt WM and alter task performance when cognitive resources become insufficient to process all task-relevant information.

Keeping it real: Looking beyond capacity limits in visual cognition

Research within visual cognition has made tremendous strides in uncovering the basic operating characteristics of the visual system by reducing the complexity of natural vision to artificial but well-controlled experimental tasks and stimuli. This reductionist approach has for example been used to assess the basic limitations of visual attention, visual working memory (VWM) capacity, and the fidelity of visual long-term memory (VLTM). The assessment of these limits is usually made in a pure sense, irrespective of goals, actions, and priors. While it is important to map out the bottlenecks our visual system faces, we focus here on selected examples of how such limitations can be overcome. Recent findings suggest that during more natural tasks, capacity may be higher than reductionist research suggests and that separable systems subserve different actions, such as reaching and looking, which might provide important insights about how pure attentional or memory limitations could be circumvented. We also review evidence suggesting that the closer we get to naturalistic behavior, the more we encounter implicit learning mechanisms that operate "for free" and "on the fly." These mechanisms provide a surprisingly rich visual experience, which can support capacity-limited systems. We speculate whether natural tasks may yield different estimates of the limitations of VWM, VLTM, and attention, and propose that capacity measurements should also pass the real-world test within naturalistic frameworks. Our review highlights various approaches for this and suggests that our understanding of visual cognition will benefit from incorporating the complexities of real-world cognition in experimental approaches.

Controlling the Flow of Distracting Information in Working Memory

Visual working memory (WM) must maintain relevant information, despite the constant influx of both relevant and irrelevant information. Attentional control mechanisms help determine which of this new information gets access to our capacity-limited WM system. Previous work has treated attentional control as a monolithic process-either distractors capture attention or they are suppressed. Here, we provide evidence that attentional capture may instead be broken down into at least two distinct subcomponent processes: (1) Spatial capture, which refers to when spatial attention shifts towards the location of irrelevant stimuli and (2) item-based capture, which refers to when item-based WM representations of irrelevant stimuli are formed. To dissociate these two subcomponent processes of attentional capture, we utilized a series of electroencephalography components that track WM maintenance (contralateral delay activity), suppression (distractor positivity), item individuation (N2pc), and spatial attention (lateralized alpha power). We show that new, relevant information (i.e., a task-relevant distractor) triggers both spatial and item-based capture. Irrelevant distractors, however, only trigger spatial capture from which ongoing WM representations can recover more easily. This fractionation of attentional capture into distinct subcomponent processes provides a refined framework for understanding how distracting stimuli affect attention and WM.

Distraction in Visual Working Memory: Resistance is Not Futile

Over half a century of research focused on understanding how working memory is capacity constrained has overshadowed the fact that it is also remarkably resistant to interference. Protecting goal-relevant information from distraction is a cornerstone of cognitive function that involves a multifaceted collection of control processes and storage mechanisms. Here, we discuss recent advances in cognitive psychology and neuroscience that have produced new insights into the nature of visual working memory and its ability to resist distraction. We propose that distraction resistance should be an explicit component in any model of working memory and that understanding its behavioral and neural correlates is essential for building a comprehensive understanding of real-world memory function.

Selection in working memory is resource-demanding: Concurrent task effects on the retro-cue effect

In a retro-cue paradigm, after memorizing a set of objects, people are cued to remember only a subset. Improved memory from the retro-cue suggests that selection processes can benefit items stored in working memory. Does selection in working memory require attention? If so, an attention-demanding task should disrupt retro-cue effects. Studies using a dual-task paradigm have found mixed results, with only one study (Janczyk & Berryhill, Attention, Perception, and Psychophysics, 76 (3), 715–724, 2014) showing a decreased retro-cue effect by a secondary task. Here we explore a potential issue in that study – the temporal overlap of the secondary task response with the memory test presentation. This raises questions about whether the secondary task was impairing selection processes in memory or was impacting the memory response. We replicated their paradigm by inserting a tone discrimination task at the retro-cue offset, but we also included a condition in which the tone task and the memory test were temporally separated. In Experiment 1, performing the tone task did not impair the retro-cue effect. In Experiment 2, we added an articulatory suppression task as in Janczyk and Berryhill’s study, and we found that the requirement to execute the tone task impaired retro-cue effects. This impairment was independent of whether the tone and memory tasks overlapped. These findings suggest that internal prioritization can be impaired by dual-task interference, but may only occur when such interference is robust enough, for example, due to switching between multiple tasks.

Bold moves: Inevitable saccadic selection in visual short-term memory

Selection for visual short-term memory (vstm) provides a basis for many cognitive functions. Saccadic eye movements sway this selection in favor of stimuli previously seen at locations congruent with their target. In three experiments, we provide converging evidence that this saccadic selection is implemented as a fundamental, inevitable selection process, rather than a top-down strategy. In particular, benefits for congruent over incongruent items were largely constant across set sizes ranging from two to eight items (Experiment 1), showing that saccadic selection imposes priorities on vstm irrespective of memory load and is effective even when only few representations need to be maintained. Moreover, a decrement in performance for incongruent items occurred reliably, whether the congruent location contained a task-relevant item or an irrelevant noise patch (Experiment 2). Finally, saccadic selection was immune to a strong manipulation of the observer's attentional priorities (Experiment 3). Given the prevalence of saccades in natural vision, our results demonstrate a fundamental and ecologically relevant selection mechanism for vstm: Saccades systematically eliminate information seen at non-target locations, while information at the saccade target remains available to recall. This simple heuristic is effective in the absence of informative cues and may incapacitate voluntary selection mechanisms that are incongruent with ongoing movement plans.

Attention and binding in visual working memory: Two forms of attention and two kinds of buffer storage

We review our research on the episodic buffer in the multicomponent model of working memory (Baddeley, 2000), making explicit the influence of Anne Treisman's work on the way our research has developed. The crucial linking theme concerns binding, whereby the individual features of an episode are combined as integrated representations. We summarize a series of experiments on visual working memory that investigated the retention of feature bindings and individual features. The effects of cognitive load, perceptual distraction, prioritization, serial position, and their interactions form a coherent pattern. We interpret our findings as demonstrating contrasting roles of externally driven and internally driven attentional processes, as well as a distinction between visual buffer storage and the focus of attention. Our account has strong links with Treisman's concept of focused attention and aligns with a number of contemporary approaches to visual working memory.

Working memory prioritization impacts neural recovery from distraction

The ability to protect goal-relevant information from disruption over short intervals is a hallmark of working memory. Recent behavioral data suggest that high-priority items in working memory are more vulnerable to disruption. We used functional magnetic resonance imaging to evaluate the hypothesis that prioritization of working memories might impact the recovery of their neural representation(s) after distraction. A delay-period retrospective cue informed participants which of two memory items (a face or a scene) to prioritize during a first delay period. Consistent with prior work, and confirming successful prioritization, multivoxel pattern classifier evidence in perceptual brain regions was higher for cued versus uncued memory items. A distraction task was then imposed before a second retrospective cue informed participants to either "stay" remembering the previously cued item or "switch" to the previously uncued item. This allowed for the evaluation of recovery for high-priority items (on stay trials) and also low-priority items (on switch trials). Classifiers showed successful reinstatement of both high- and low-priority items after distraction, but only low-priority items recovered to their pre-distraction representational levels. Moreover, the degree of prioritization before distraction predicted the amount of disruption for high-priority items after distraction, suggesting that the more a participant prioritized the cued item, the greater the impact of distraction. Our data provide neural evidence that prioritizing working memory information in perceptual regions makes that information more vulnerable to disruption.

EEG dynamics reveal a dissociation between storage and selective attention within working memory

Selective attention plays a prominent role in prioritizing information in working memory (WM), improving performance for attended representations. However, it remains unclear whether unattended WM representations suffer from information loss. Here we tested the hypothesis that within WM, selectively attending to an item and stopping storing other items are independent mechanisms. We recorded EEG while participants performed a WM recall task in which the item most likely to be tested was cued retrospectively during retention. By manipulating retro-cue reliability (i.e., the ratio of valid to invalid cue trials), we varied the incentive to retain non-cued items. Storage and selective attention in WM were measured during the retention interval by contralateral delay activity (CDA) and contralateral alpha power suppression, respectively. Soon after highly reliable cues, the cued item was attended, and non-cued items suffered information loss. However, for less reliable cues, initially the cued item was attended, but unattended items were kept in WM. Later during the delay, previously unattended items suffered information loss despite now attention being reallocated to their locations, presumably to strengthen their weakening traces. These results show that storage and attention in WM are distinct processes that can behave differently depending on the relative importance of representations.

Working Memory and Attention - Response to Commentaries

This is a brief reply to the commentaries by Adam and deBettencourt (2019); Allen (2019); Kiyonaga (2019); Schneider (2019); and Van der Stigchel and Olivers (2019), focusing on four topics: (1) I defend the idea that attention need not be characterized as a limited resource. (2) I explain how I conceptualize the role of WM in cognitive control, and how recruitment of sensory processing networks contributes to control but not maintenance. (3) I discuss different ways in which information can be selectively prioritized during or after being encoded into WM, and the different consequences of these processes. (4) I argue that sustained attention to a task can be understood as the mind's ability to prioritize that task over task-unrelated representations.

Multiple high-reward items can be prioritized in working memory but with greater vulnerability to interference

Emerging literature indicates that working memory and attention interact in determining what is retained over time, though the nature of this relationship and the impacts on performance across different task contexts remain to be mapped. In the present study, four experiments examined whether participants can prioritize one or more high-reward items within a four-item target array for the purposes of an immediate cued recall task, and the extent to which this mediates the disruptive impact of a postdisplay to-be-ignored suffix. All four experiments indicated that endogenous direction of attention toward high-reward items results in their improved recall. Furthermore, increasing the number of high-reward items from one to three (Experiments 1-3) produces no decline in recall performance for those items, while associating each item in an array with a different reward value results in correspondingly graded levels of recall performance (Experiment 4). These results suggest the ability to exert precise voluntary control in the prioritization of multiple targets. However, in line with recent outcomes drawn from serial visual memory, this endogenously driven focus on high-reward items results in greater susceptibility to exogenous suffix interference, relative to low-reward items. This contrasts with outcomes from cueing paradigms, indicating that different methods of attentional direction may not always result in equivalent outcomes on working memory performance.

A two-level hierarchical framework of visual short-term memory

Over the last couple of decades, a vast amount of research has been dedicated to understanding the nature and the architecture of visual short-term memory (VSTM), the mechanism by which currently relevant visual information is maintained. According to discrete-capacity models, VSTM is constrained by a limited number of discrete representations held simultaneously. In contrast, shared-resource models regard VSTM as limited in resources, which can be distributed flexibly between varying numbers of representations; and a new interference model posits that capacity is limited by interference among items. In this article, we begin by reviewing benchmark findings regarding the debate over VSTM limitations, focusing on whether VSTM storage is all-or-none and on whether object complexity affects capacity. After that, we put forward a hybrid framework of VSTM architecture, arguing that this system is composed of a two-level hierarchy of memory stores, each containing a different set of representations: (1) perceptual memory, a resourcelike level containing analog automatically formed representations of visual stimuli in varying degrees of activation, and (2) visual working memory, in which a subset of three to four items from perceptual memory are bound to conceptual representations and to their locations, thus conveying discrete (digital/symbolic) information which appears quantized. While perceptual memory has a large capacity and is relatively nonselective, visual working memory is restricted in the number of items that can be maintained simultaneously, and its content is regulated by a gating mechanism.

The Retro-Cue Benefit for Verbal Material and Its Influence on the Probability of Intrusions Under Dual-Task Conditions

It is well established in the working memory literature, that performance can be improved by cueing attention toward the position of a to-be-tested item, even after that item's presentation. This retro-cue benefit is often characterized as the joint outcome of two different effects: facilitation of recall and memory strengthening at the cued position. While the latter has been mainly explained by increased context-content binding, competing hypotheses exist to explain the facilitation of recall. The present study focuses on two of these hypotheses: the removal of non-cued information and the protection of cued information against interference. I replicate the retro-cue effect for verbal material and provide strong evidence for its protective effect. However, I did not find support for the removal hypothesis. This lack of support follows from two empirical findings: Retro-cueing does not decrease, rather increases the conditional probability of intrusions, and the retro-cue benefit does not interact with memory load.

Social attention directs working memory maintenance

Visual working memory (vWM) performance is enhanced when a memorized object is cued after encoding. This so-called retro-cue effect is typically observed with a predictive (80% valid), retrospective cue. The current study examined whether a nonpredictive (50% valid) retro-cue can similarly enhance internal memory representations in cases where the cue conveys social signals. To this end, gaze cues were presented during the retention interval of a change-detection task, which are capable to engender a mutual attentional focus of two individuals towards one location. In line with our prediction, Experiment 1 demonstrated that a polygon presented at the gazed-at location was remembered better than that at both non-gazed and gazed-away locations. Experiments 2 and 3 showed that low-level motion cues did not elicit attentional orienting in a comparable manner as the gaze cue, and these differences in cuing were found to be reliable and independent of memory load. Furthermore, the gaze retro-cue effect disappeared when the face was inverted (Experiment 4). In sum, these results clearly show that sharing the focus of another individual establishes a point of reference from which visual information is restored with priority, suggesting that a gaze retro-cue leads to social attention, thus, modulating vWM maintenance in a reflexive, automatic manner.

In search of the focus of attention in working memory: 13 years of the retro-cue effect

The concept of attention has a prominent place in cognitive psychology. Attention can be directed not only to perceptual information, but also to information in working memory (WM). Evidence for an internal focus of attention has come from the retro-cue effect: Performance in tests of visual WM is improved when attention is guided to the test-relevant contents of WM ahead of testing them. The retro-cue paradigm has served as a test bed to empirically investigate the functions and limits of the focus of attention in WM. In this article, we review the growing body of (behavioral) studies on the retro-cue effect. We evaluate the degrees of experimental support for six hypotheses about what causes the retro-cue effect: (1) Attention protects representations from decay, (2) attention prioritizes the selected WM contents for comparison with a probe display, (3) attended representations are strengthened in WM, (4) not-attended representations are removed from WM, (5) a retro-cue to the retrieval target provides a head start for its retrieval before decision making, and (6) attention protects the selected representation from perceptual interference. The extant evidence provides support for the last four of these hypotheses.

Working memory accuracy for multiple targets is driven by reward expectation and stimulus contrast with different time-courses

The richness of sensory input dictates that the brain must prioritize and select information for further processing and storage in working memory. Stimulus salience and reward expectations influence this prioritization but their relative contributions and underlying mechanisms are poorly understood. Here we investigate how the quality of working memory for multiple stimuli is determined by priority during encoding and later memory phases. Selective attention could, for instance, act as the primary gating mechanism when stimuli are still visible. Alternatively, observers might still be able to shift priorities across memories during maintenance or retrieval. To distinguish between these possibilities, we investigated how and when reward cues determine working memory accuracy and found that they were only effective during memory encoding. Previously learned, but currently non-predictive, color-reward associations had a similar influence, which gradually weakened without reinforcement. Finally, we show that bottom-up salience, manipulated through varying stimulus contrast, influences memory accuracy during encoding with a fundamentally different time-course than top-down reward cues. While reward-based effects required long stimulus presentation, the influence of contrast was strongest with brief presentations. Our results demonstrate how memory resources are distributed over memory targets and implicates selective attention as a main gating mechanism between sensory and memory systems.

Benefits of flexible prioritization in working memory can arise without costs

Most recent models conceptualize working memory (WM) as a continuous resource, divided up according to task demands. When an increasing number of items need to be remembered, each item receives a smaller chunk of the memory resource. These models predict that the allocation of attention to high-priority WM items during the retention interval should be a zero-sum game: improvements in remembering cued items come at the expense of uncued items because resources are dynamically transferred from uncued to cued representations. The current study provides empirical data challenging this model. Four precision retrocueing WM experiments assessed cued and uncued items on every trial. This permitted a test for trade-off of the memory resource. We found no evidence for trade-offs in memory across trials. Moreover, robust improvements in WM performance for cued items came at little or no cost to uncued items that were probed afterward, thereby increasing the net capacity of WM relative to neutral cueing conditions. An alternative mechanism of prioritization proposes that cued items are transferred into a privileged state within a response-gating bottleneck, in which an item uniquely controls upcoming behavior. We found evidence consistent with this alternative. When an uncued item was probed first, report of its orientation was biased away from the cued orientation to be subsequently reported. We interpret this bias as competition for behavioral control in the output-driving bottleneck. Other items in WM did not bias each other, making this result difficult to explain with a shared resource model.

This study challenges the dominant model for how we remember and prioritize pieces of information over short intervals (working memory). The dominant view is that all items in working memory share a single resource, and that we can prioritize one item by redistributing resources in its favor. This view predicts that nonprioritized memories become lost or impoverished. By testing how well participants remember both prioritized and nonprioritized items, we show that this is not the case. Our findings suggest that memories can be prioritized flexibly without necessarily jeopardizing others that may still become relevant.

Prioritizing Information during Working Memory: Beyond Sustained Internal Attention

Working memory (WM) has limited capacity. This leaves attention with the important role of allowing into storage only the most relevant information. It is increasingly evident that attention is equally crucial for prioritizing representations within WM as the importance of individual items changes. Retrospective prioritization has been proposed to result from a focus of internal attention highlighting one of several representations. Here, we suggest an updated model, in which prioritization acts in multiple steps: first orienting towards and selecting a memory, and then reconfiguring its representational state in the service of upcoming task demands. Reconfiguration sets up an optimized perception-action mapping, obviating the need for sustained attention. This view is consistent with recent literature, makes testable predictions, and links WM with task switching and action preparation.

On the neural mechanisms underlying the protective function of retroactive cuing against perceptual interference: Evidence by event-related potentials of the EEG

This EEG study investigated the protective effect of retroactive attentional focusing on working memory. To this effect, we used a visuo-spatial working memory task and presented block-wise distractor displays after working memory contents had been updated by means of a retroactive cue (retro-cue). Retroactive attention attenuated the interfering effect of distractors on memory precision. The reduction of working memory load by means of a selective retro-cue was reflected by a decline of a negative slow wave over parietal sites. Posterior N1 to the distractor was reduced following a selective retro-cue compared to a neutral retro-cue condition, most notably at left hemispheric sites. P3b referred to the distractor was suppressed completely only following a selective retro-cue. This suggests that focusing on only a subset of visuo-spatial information represented in working memory releases cognitive resources for preventing the in-depth processing of subsequently irrelevant visual events, thereby inhibiting their transfer into working memory.

Adverse orienting effects on visual working memory encoding and maintenance

Visual working memory (VWM) representations can be strengthened by pre-cues presented before, and retro-cues presented after, the memory display, providing evidence that attentional orienting plays a role in memory encoding and maintenance. It is unknown whether attentional orienting to VWM stimuli can also have adverse effects (known as inhibition of return; IOR), as has been found for perceptual-cueing tasks. If so, this would provide further evidence for common attentional orienting mechanisms for mnemonic and perceptual representations. In Experiment 1, we used pre-cueing and demonstrated an increased encoding probability, but not precision, at short SOAs, but probability decreased at long SOAs, reminiscent of the classic IOR findings. In Experiment 2, we used retro-cueing and showed that it improved memory performance, unless attention was cued back to the center of the display by a second cue. In this case, the deleterious effects were on precision, indicating that the item was still retained, but its quality of representation suffered. Together, these results provide further evidence for universal spatial attentional mechanisms operating on perceptual as well as mnemonic representations.

Retro-dimension-cue benefit in visual working memory

In visual working memory (VWM) tasks, participants’ performance can be improved by a retro-object-cue. However, previous studies have not investigated whether participants’ performance can also be improved by a retro-dimension-cue. Three experiments investigated this issue. We used a recall task with a retro-dimension-cue in all experiments. In Experiment 1, we found benefits from retro-dimension-cues compared to neutral cues. This retro-dimension-cue benefit is reflected in an increased probability of reporting the target, but not in the probability of reporting the non-target, as well as increased precision with which this item is remembered. Experiment 2 replicated the retro-dimension-cue benefit and showed that the length of the blank interval after the cue disappeared did not influence recall performance. Experiment 3 replicated the results of Experiment 2 with a lower memory load. Our studies provide evidence that there is a robust retro-dimension-cue benefit in VWM. Participants can use internal attention to flexibly allocate cognitive resources to a particular dimension of memory representations. The results also support the feature-based storing hypothesis.

Retrospective Attention Interacts with Stimulus Strength to Shape Working Memory Performance

Orienting attention retrospectively to selective contents in working memory (WM) influences performance. A separate line of research has shown that stimulus strength shapes perceptual representations. There is little research on how stimulus strength during encoding shapes WM performance, and how effects of retrospective orienting might vary with changes in stimulus strength. We explore these questions in three experiments using a continuous-recall WM task. In Experiment 1 we show that benefits of cueing spatial attention retrospectively during WM maintenance (retrocueing) varies according to stimulus contrast during encoding. Retrocueing effects emerge for supraliminal but not sub-threshold stimuli. However, once stimuli are supraliminal, performance is no longer influenced by stimulus contrast. In Experiments 2 and 3 we used a mixture-model approach to examine how different sources of error in WM are affected by contrast and retrocueing. For high-contrast stimuli (Experiment 2), retrocues increased the precision of successfully remembered items. For low-contrast stimuli (Experiment 3), retrocues decreased the probability of mistaking a target with distracters. These results suggest that the processes by which retrospective attentional orienting shape WM performance are dependent on the quality of WM representations, which in turn depends on stimulus strength during encoding.

The reliability of retro-cues determines the fate of noncued visual working memory representations

Retrospectively cueing an item retained in visual working memory during maintenance is known to improve its retention. However, studies have provided conflicting results regarding the costs of such retro-cues for the noncued items, leading to different theories on the mechanisms behind visual working memory maintenance and retro-cueing. Here we tested an alternative explanation of the conflicting results regarding retro-cue costs-namely, that they are caused at least partly by differences in retro-cue reliability. We manipulated the ratio of valid-cue trials to invalid-cue trials within blocks. We used a continuous-report procedure that allowed fitting a model that provided recall probability and precision estimates for the memory representations. Reconciling previous contradictory findings, benefits for valid cues were observed in all conditions, but invalid cueing costs were found only when the retro-cue had a high reliability (i.e., was 80 % valid), but not when it had a lower reliability (i.e., 50 % valid). This was found for both the recall probability and the precision of visual working memory representations. Our results suggest that the cognitive mechanisms underlying retro-cue effects are strategically adjusted by participants, depending on the perceived retro-cue reliability.

The time course of visuo-spatial working memory updating revealed by a retro-cuing paradigm

We investigated the influence of non-cued working memory representations on further information processing. A retro-cue indicated if items on the left or right side of a previous memory array remained relevant. Subsequently, a central probe item was presented with a varying stimulus onset asynchrony (SOA: 300, 400, 600, 1000, 1800 ms). Participants had to state whether this stimulus was shown on the cued side of the memory array. The probe was either a cued, non-cued or new item. Non-cued probes were associated with delayed response times and an increased frontal negativity from 400–600 ms indicating a higher processing conflict compared to new probes. These effects were strongest for the 300 and 400 ms SOAs and decreased in longer SOA conditions, pointing toward a benefit when there was sufficient time for working memory updating. Furthermore, contralateral negativities at posterior (PCN) and anterior sites (ADAN) reflected the attentional orienting toward cued information while selective retention was associated with a sustained suppression of posterior induced alpha power contralateral to retro-cue direction. Results suggest that retro-cue induced updating of visuo-spatial working memory requires about 500 ms to transfer the non-cued contents into a passive and fragile short-term memory state with less impact on ongoing information processing.

Attention and memory protection: Interactions between retrospective attention cueing and interference

Visual working memory (VWM) and attention have a number of features in common, but despite extensive research it is still unclear how the two interact. Can focused attention improve VWM precision? Can it protect VWM from interference? Here we used a partial-report, continuous-response orientation memory task to examine how attention and interference affect different aspects of VWM and how they interact with one another. Both attention and interference were orthogonally manipulated during the retention interval. Attention was manipulated by presenting informative retro-cues, whereas interference was manipulated by introducing a secondary interfering task. Mixture-model analyses revealed that retro-cues, compared to uninformative cues, improved all aspects of performance: Attention increased recall precision and decreased guessing rate and swap-errors (reporting a wrong item in memory). Similarly, performing a secondary task impaired all aspects of the VWM task. In particular, an interaction between retro-cue and secondary task interference was found primarily for swap-errors. Together these results suggest that both the quantity and quality of VWM representations are sensitive to attention cueing and interference modulations, and they highlight the role of attention in protecting the feature-location associations needed to access the correct items in memory.