The strategic retention of task-relevant objects in visual working memory

Selectively maintaining an object's feature in visual working memory: A comparison between highly discriminable and fine-grained features

Selectively maintaining information is an essential function of visual working memory (VWM). Recent VWM studies have mainly focused on selective maintenance of objects, leaving the mechanisms of selectively maintaining an object's feature in VWM unknown. Based on the interactive model of perception and VWM, we hypothesized that there are distinct selective maintenance mechanisms for objects containing fine-grained features versus objects containing highly discriminable features. To test this hypothesis, we first required participants to memorize a dual-feature object (colored simple shapes vs. colored polygons), and informed them about the target feature via a retro-cue. Then a visual search task was added to examine the fate of the irrelevant feature. The selective maintenance of an object's feature predicted that the irrelevant feature should be removed from the active state of VWM and should not capture attention when presented as a distractor in the visual search task. We found that irrelevant simple shapes impaired performance in the visual search task (Experiment 1). However, irrelevant polygons did not affect visual search performance (Experiment 2), and this could not be explained by decay of polygons (Experiment 3) or by polygons not capturing attention (Experiment 4). These findings suggest that VWM adopts dissociable mechanisms to selectively maintain an object's feature, depending on the feature's perceptual characteristics.

Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.

Working memory is updated by reallocation of resources from obsolete to new items

Visual working memory (VWM) resources are limited, placing constraints on how much visual information can be simultaneously retained. During visually guided activity, stored information can quickly become outdated, so updating mechanisms are needed to ensure the contents of memory remain relevant to current task goals. In particular, successful deallocation of resources from items that become obsolete is likely to be critical for maintaining the precision of those representations still in memory. The experiments in this study involved presenting two memory arrays of coloured disks in sequence. The appearance of the second array was a cue to replace, rehearse, or add a new colour to the colours in memory. We predicted that successful resource reallocation should result in comparable recall precision when an item was replaced or rehearsed, owing to the removal of pre-replacement features. In contrast, a failure to update WM should lead to comparable precision with a condition in which a new colour was added to memory. We identified a very small proportion (∼5%) of trials in which participants incorrectly reported a feature from the first array in place of its replacement in the second, which we interpreted as a failure to incorporate the information from the second display into memory. Once these trials were discounted, precision estimates were consistent with complete redistribution of resources in the case of updating a single item. We conclude that working memory can be efficiently updated when previous information becomes obsolete, but that this is a demanding active process that occasionally fails.

Comparing the temporal dynamics and efficacy of task-relevant and task-irrelevant memory-driven attention

People's attention is well attracted to a stimulus matching their memory. For example, when people are required to remember the color of a visual object, stimuli matching the memory color powerfully capture attention. Remarkably, stimuli with the shape of the memory object, that is, irrelevant-matching stimuli were also found to capture attention. Here, we examined how task relevance affects the temporal dynamics and the strength of memory-driven attention. In the experiment, participants performed a visual search task while maintaining the color or shape of a colored shape. When participants were required to memorize the color of the memory sample, the shape of the sample stimulus is task-irrelevant feature and vice versa. Importantly, while a search item matching working memory in the task-relevant dimension was presented for one group of participants, an irrelevant-matching search item appeared for the other group of participants. Further, we varied stimulus onset asynchrony (SOA) between the memory sample and search items. We found that relevant-matching stimuli captured attention regardless of whether the SOA was short or long. However, attentional capture by irrelevant-matching stimuli depended on the SOA; no memory-driven capture was observed at the shortest SOA, but significant capture was found at longer SOAs. Further, the capture effects by relevant-matching stimuli were greater than that of irrelevant-matching stimuli. These findings suggest both task-relevant and -irrelevant features in working memory affect the attentional selection in visual search task, but its temporal dynamics and strength are modulated by the task-relevance.

Attentional templates are protected from retroactive interference during visual search: Converging evidence from event-related potentials

Attentional templates are stored representations of target features that guide visual search. Target features may remain fixed or change on every trial, requiring sustained or transient templates, respectively. In separate blocks of trials, two sustained templates guide visual search as efficiently as two transient templates. In mixed blocks, however, the transient template interferes with the sustained template, impairing its efficiency in guiding visual search. Here, we hypothesized that the priority of the sustained template would increase when threatened by interference, eventually restoring efficient guidance of visual search. Participants memorized two possible target colors before the onset of the search display. At encoding, we assessed attentional selection of the two possible target colors with the N2pc. During subsequent maintenance, we measured the CDA as an index of resource allocation in working memory. In Experiment 1, the CDA was smaller with sustained than transient templates in separate blocks, but similar in mixed blocks. Thus, the sustained template received more working memory resources when maintained concurrently with an interfering transient template, suggesting that it was prioritized. In Experiment 2, the priority of the sustained template was further increased as it guided visual search in 80% of cases. The N2pc to possible target colors matching the sustained template was enhanced both at encoding and during visual search, thus eliminating interference from the transient template. Therefore, sustained templates are not necessarily less efficient than transient templates. Rather, prioritization through attentional selection at encoding and resource allocation during maintenance may restore efficient guidance of visual search.

Keeping it real: Looking beyond capacity limits in visual cognition

Research within visual cognition has made tremendous strides in uncovering the basic operating characteristics of the visual system by reducing the complexity of natural vision to artificial but well-controlled experimental tasks and stimuli. This reductionist approach has for example been used to assess the basic limitations of visual attention, visual working memory (VWM) capacity, and the fidelity of visual long-term memory (VLTM). The assessment of these limits is usually made in a pure sense, irrespective of goals, actions, and priors. While it is important to map out the bottlenecks our visual system faces, we focus here on selected examples of how such limitations can be overcome. Recent findings suggest that during more natural tasks, capacity may be higher than reductionist research suggests and that separable systems subserve different actions, such as reaching and looking, which might provide important insights about how pure attentional or memory limitations could be circumvented. We also review evidence suggesting that the closer we get to naturalistic behavior, the more we encounter implicit learning mechanisms that operate "for free" and "on the fly." These mechanisms provide a surprisingly rich visual experience, which can support capacity-limited systems. We speculate whether natural tasks may yield different estimates of the limitations of VWM, VLTM, and attention, and propose that capacity measurements should also pass the real-world test within naturalistic frameworks. Our review highlights various approaches for this and suggests that our understanding of visual cognition will benefit from incorporating the complexities of real-world cognition in experimental approaches.

Allocation of resources in working memory: Theoretical and empirical implications for visual search

Recently, working memory (WM) has been conceptualized as a limited resource, distributed flexibly and strategically between an unlimited number of representations. In addition to improving the precision of representations in WM, the allocation of resources may also shape how these representations act as attentional templates to guide visual search. Here, we reviewed recent evidence in favor of this assumption and proposed three main principles that govern the relationship between WM resources and template-guided visual search. First, the allocation of resources to an attentional template has an effect on visual search, as it may improve the guidance of visual attention, facilitate target recognition, and/or protect the attentional template against interference. Second, the allocation of the largest amount of resources to a representation in WM is not sufficient to give this representation the status of attentional template and thus, the ability to guide visual search. Third, the representation obtaining the status of attentional template, whether at encoding or during maintenance, receives an amount of WM resources proportional to its relevance for visual search. Thus defined, the resource hypothesis of visual search constitutes a parsimonious and powerful framework, which provides new perspectives on previous debates and complements existing models of template-guided visual search.

The role of perception and action on the use of allocentric information in a large-scale virtual environment

In everyday life, our brain constantly builds spatial representations of the objects surrounding us. Many studies have investigated the nature of these spatial representations. It is well established that we use allocentric information in real-time and memory-guided movements. Most studies relied on small-scale and static experiments, leaving it unclear whether similar paradigms yield the same results on a larger scale using dynamic objects. We created a virtual reality task that required participants to encode the landing position of a virtual ball thrown by an avatar. Encoding differed in the nature of the task in that it was either purely perceptual (“view where the ball landed while standing still”—Experiment 1) or involved an action (“intercept the ball with the foot just before it lands”—Experiment 2). After encoding, participants were asked to place a real ball at the remembered landing position in the virtual scene. In some trials, we subtly shifted either the thrower or the midfield line on a soccer field to manipulate allocentric coding of the ball’s landing position. In both experiments, we were able to replicate classic findings from small-scale experiments and to generalize these results to different encoding tasks (perception vs. action) and response modes (reaching vs. walking-and-placing). Moreover, we found that participants preferably encoded the ball relative to the thrower when they had to intercept the ball, suggesting that the use of allocentric information is determined by the encoding task by enhancing task-relevant allocentric information. Our findings indicate that results previously obtained from memory-guided reaching are not restricted to small-scale movements, but generalize to whole-body movements in large-scale dynamic scenes.

Electrophysiological evidence supports the role of sustained visuospatial attention in maintaining visual WM contents

Recent empirical and theoretical work suggests that there is a close relationship between visual working memory (WM) and visuospatial attention. Here, we investigated whether visuospatial attention was involved in maintaining object representations in visual WM. To this end, the alpha lateralization and contralateral delay activity (CDA) were analyzed as neural markers for visuospatial attention and visual WM storage, respectively. In the single-task condition, participants performed a grating change-detection task. To probe the role of visuospatial attention in maintaining WM contents, two color squares were presented above and below the fixation point during the retention interval, which remained visible until the detection display was present. In the dual-task condition, participants were required to maintain lateralized gratings while staring at the center-presented color squares, to detect possible subsequent color change. With this task, sustained visuospatial attention that guided to individual memory representations was disrupted. The behavioral data showed that, the insertion of secondary task significantly deteriorated WM performance. For electrophysiological data, we divided the retention interval into two stages, the early stage and late stage, bounded by the onset of the secondary task. We found that CDA amplitude was lower under the dual-task condition than the single-task condition during the late stage, but not the early stage, and the extent to which CDA reduced tracked the impaired memory performance at the individual level. Also, alpha lateralization only could be observed in the single-task condition of the late stage, and completely disappeared in the dual-task condition, indicating the disruption of visuospatial attention directed to memory representations. Individuals who experienced greater visuospatial attention disruption, as indicated by the alpha lateralization, had lower maintenance-associated neural activity (CDA), and suffered greater impairment of memory performance. These findings confirm that sustained visuospatial attention continues improving visual WM processing after the initial encoding phase, and most likely participates in this process by supporting the maintenance of representations in an active state.

Feature-based guidance of attention by visual working memory is applied independently of remembered object location

Visual working memory (VWM) has been implicated both in the online representation of object tokens (in the object-file framework) and in the top-down guidance of attention during visual search, implementing a feature template. It is well established that object representations in VWM are structured by location, with access to the content of VWM modulated by position consistency. In the present study, we examined whether this property generalizes to the guidance of attention. Specifically, in two experiments, we probed whether the guidance of spatial attention from features in VWM is modulated by the position of the object from which these features were encoded. Participants remembered an object with an incidental color. Items in a subsequent search array could match either the color of the remembered object, the location, or both. Robust benefits of color match (when the matching item was the target) and costs (when the matching items was a distractor) were observed. Critically, the magnitude of neither effect was influenced by spatial correspondence. The results demonstrate that features in VWM influence attentional priority maps in a manner that does not necessarily inherit the spatial structure of the object representations in which those features are maintained.

Selective maintenance of value information helps resolve the exploration/exploitation dilemma

In natural environments with many options of uncertain value, one faces a difficult tradeoff between exploiting familiar, valuable options or searching for better alternatives. Reinforcement learning models of this exploration/exploitation dilemma typically modulate the rate of exploratory choices or preferentially sample uncertain options. The extent to which such models capture human behavior remains unclear, in part because they do not consider the constraints on remembering what is learned. Using reinforcement-based timing as a motivating example, we show that selectively maintaining high-value actions compresses the amount of information to be tracked in learning, as quantified by Shannon's entropy. In turn, the information content of the value representation controls the balance between exploration (high entropy) and exploitation (low entropy). Selectively maintaining preferred action values while allowing others to decay renders the choices increasingly exploitative across learning episodes. To adjudicate among alternative maintenance and sampling strategies, we developed a new reinforcement learning model, StrategiC ExPloration/ExPloitation of Temporal Instrumental Contingencies (SCEPTIC). In computational studies, a resource-rational selective maintenance approach was as successful as more resource-intensive strategies. Furthermore, human behavior was consistent with selective maintenance; information compression was most pronounced in subjects with superior performance and non-verbal intelligence, and in learnable vs. unlearnable contingencies. Cognitively demanding uncertainty-directed exploration recovered a more accurate representation in simulations with no foraging advantage and was strongly unsupported in our human study.

In search of the focus of attention in working memory: 13 years of the retro-cue effect

The concept of attention has a prominent place in cognitive psychology. Attention can be directed not only to perceptual information, but also to information in working memory (WM). Evidence for an internal focus of attention has come from the retro-cue effect: Performance in tests of visual WM is improved when attention is guided to the test-relevant contents of WM ahead of testing them. The retro-cue paradigm has served as a test bed to empirically investigate the functions and limits of the focus of attention in WM. In this article, we review the growing body of (behavioral) studies on the retro-cue effect. We evaluate the degrees of experimental support for six hypotheses about what causes the retro-cue effect: (1) Attention protects representations from decay, (2) attention prioritizes the selected WM contents for comparison with a probe display, (3) attended representations are strengthened in WM, (4) not-attended representations are removed from WM, (5) a retro-cue to the retrieval target provides a head start for its retrieval before decision making, and (6) attention protects the selected representation from perceptual interference. The extant evidence provides support for the last four of these hypotheses.

Benefits of flexible prioritization in working memory can arise without costs

Most recent models conceptualize working memory (WM) as a continuous resource, divided up according to task demands. When an increasing number of items need to be remembered, each item receives a smaller chunk of the memory resource. These models predict that the allocation of attention to high-priority WM items during the retention interval should be a zero-sum game: improvements in remembering cued items come at the expense of uncued items because resources are dynamically transferred from uncued to cued representations. The current study provides empirical data challenging this model. Four precision retrocueing WM experiments assessed cued and uncued items on every trial. This permitted a test for trade-off of the memory resource. We found no evidence for trade-offs in memory across trials. Moreover, robust improvements in WM performance for cued items came at little or no cost to uncued items that were probed afterward, thereby increasing the net capacity of WM relative to neutral cueing conditions. An alternative mechanism of prioritization proposes that cued items are transferred into a privileged state within a response-gating bottleneck, in which an item uniquely controls upcoming behavior. We found evidence consistent with this alternative. When an uncued item was probed first, report of its orientation was biased away from the cued orientation to be subsequently reported. We interpret this bias as competition for behavioral control in the output-driving bottleneck. Other items in WM did not bias each other, making this result difficult to explain with a shared resource model.

This study challenges the dominant model for how we remember and prioritize pieces of information over short intervals (working memory). The dominant view is that all items in working memory share a single resource, and that we can prioritize one item by redistributing resources in its favor. This view predicts that nonprioritized memories become lost or impoverished. By testing how well participants remember both prioritized and nonprioritized items, we show that this is not the case. Our findings suggest that memories can be prioritized flexibly without necessarily jeopardizing others that may still become relevant.

Scene Configuration and Object Reliability Affect the Use of Allocentric Information for Memory-Guided Reaching

Previous research has shown that egocentric and allocentric information is used for coding target locations for memory-guided reaching movements. Especially, task-relevance determines the use of objects as allocentric cues. Here, we investigated the influence of scene configuration and object reliability as a function of task-relevance on allocentric coding for memory-guided reaching. For that purpose, we presented participants images of a naturalistic breakfast scene with five objects on a table and six objects in the background. Six of these objects served as potential reach-targets (= task-relevant objects). Participants explored the scene and after a short delay, a test scene appeared with one of the task-relevant objects missing, indicating the location of the reach target. After the test scene vanished, participants performed a memory-guided reaching movement toward the target location. Besides removing one object from the test scene, we also shifted the remaining task-relevant and/or task-irrelevant objects left- or rightwards either coherently in the same direction or incoherently in opposite directions. By varying object coherence, we manipulated the reliability of task-relevant and task-irrelevant objects in the scene. In order to examine the influence of scene configuration (distributed vs. grouped arrangement of task-relevant objects) on allocentric coding, we compared the present data with our previously published data set (Klinghammer et al., 2015). We found that reaching errors systematically deviated in the direction of object shifts, but only when the objects were task-relevant and their reliability was high. However, this effect was substantially reduced when task-relevant objects were distributed across the scene leading to a larger target-cue distance compared to a grouped configuration. No deviations of reach endpoints were observed in conditions with shifts of only task-irrelevant objects or with low object reliability irrespective of task-relevancy. Moreover, when solely task-relevant objects were shifted incoherently, the variability of reaching endpoints increased compared to coherent shifts of task-relevant objects. Our results suggest that the use of allocentric information for coding targets for memory-guided reaching depends on the scene configuration, in particular the average distance of the reach target to task-relevant objects, and the reliability of task-relevant allocentric information.

Selection in spatial working memory is independent of perceptual selective attention, but they interact in a shared spatial priority map

We examined the relationship between the attentional selection of perceptual information and of information in working memory (WM) through four experiments, using a spatial WM-updating task. Participants remembered the locations of two objects in a matrix and worked through a sequence of updating operations, each mentally shifting one dot to a new location according to an arrow cue. Repeatedly updating the same object in two successive steps is typically faster than switching to the other object; this object switch cost reflects the shifting of attention in WM. In Experiment 1, the arrows were presented in random peripheral locations, drawing perceptual attention away from the selected object in WM. This manipulation did not eliminate the object switch cost, indicating that the mechanisms of perceptual selection do not underlie selection in WM. Experiments 2a and 2b corroborated the independence of selection observed in Experiment 1, but showed a benefit to reaction times when the placement of the arrow cue was aligned with the locations of relevant objects in WM. Experiment 2c showed that the same benefit also occurs when participants are not able to mark an updating location through eye fixations. Together, these data can be accounted for by a framework in which perceptual selection and selection in WM are separate mechanisms that interact through a shared spatial priority map.

Using electrophysiology to demonstrate that cueing affects long-term memory storage over the short term

As researchers who study working memory, we often assume that participants keep a representation of an object in working memory when we present a cue that indicates that the object will be tested in a couple of seconds. This intuitively accounts for how well people can remember a cued object, relative to their memory for that same object presented without a cue. However, it is possible that this superior memory does not purely reflect storage of the cued object in working memory. We tested the hypothesis that cues presented during a stream of objects, followed by a short retention interval and immediate memory test, can change how information is handled by long-term memory. We tested this hypothesis by using a family of frontal event-related potentials believed to reflect long-term memory storage. We found that these frontal indices of long-term memory were sensitive to the task relevance of objects signaled by auditory cues, even when the objects repeated frequently, such that proactive interference was high. Our findings indicate the problematic nature of assuming process purity in the study of working memory, and demonstrate that frequent stimulus repetitions fail to isolate the role of working memory mechanisms.

Retro-cue benefits in working memory without sustained focal attention

In working memory (WM) tasks, performance can be boosted by directing attention to one memory object: When a retro-cue in the retention interval indicates which object will be tested, responding is faster and more accurate (the retro-cue benefit). We tested whether the retro-cue benefit in WM depends on sustained attention to the cued object by inserting an attention-demanding interruption task between the retro-cue and the memory test. In the first experiment, the interruption task required participants to shift their visual attention away from the cued representation and to a visual classification task on colors. In the second and third experiments, the interruption task required participants to shift their focal attention within WM: Attention was directed away from the cued representation by probing another representation from the memory array prior to probing the cued object. The retro-cue benefit was not attenuated by shifts of perceptual attention or by shifts of attention within WM. We concluded that sustained attention is not needed to maintain the cued representation in a state of heightened accessibility.

Focused attention improves working memory: implications for flexible-resource and discrete-capacity models

Performance in working memory (WM) tasks depends on the capacity for storing objects and on the allocation of attention to these objects. Here, we explored how capacity models need to be augmented to account for the benefit of focusing attention on the target of recall. Participants encoded six colored disks (Experiment 1) or a set of one to eight colored disks (Experiment 2) and were cued to recall the color of a target on a color wheel. In the no-delay condition, the recall-cue was presented after a 1,000-ms retention interval, and participants could report the retrieved color immediately. In the delay condition, the recall-cue was presented at the same time as in the no-delay condition, but the opportunity to report the color was delayed. During this delay, participants could focus attention exclusively on the target. Responses deviated less from the target's color in the delay than in the no-delay condition. Mixture modeling assigned this benefit to a reduction in guessing (Experiments 1 and 2) and transposition errors (Experiment 2). We tested several computational models implementing flexible or discrete capacity allocation, aiming to explain both the effect of set size, reflecting the limited capacity of WM, and the effect of delay, reflecting the role of attention to WM representations. Both models fit the data better when a spatially graded source of transposition error is added to its assumptions. The benefits of focusing attention could be explained by allocating to this object a higher proportion of the capacity to represent color.

Attentional priority determines working memory precision

Visual working memory is a system used to hold information actively in mind for a limited time. The number of items and the precision with which we can store information has limits that define its capacity. How much control do we have over the precision with which we store information when faced with these severe capacity limitations? Here, we tested the hypothesis that rank-ordered attentional priority determines the precision of multiple working memory representations. We conducted two psychophysical experiments that manipulated the priority of multiple items in a two-alternative forced choice task (2AFC) with distance discrimination. In Experiment 1, we varied the probabilities with which memorized items were likely to be tested. To generalize the effects of priority beyond simple cueing, in Experiment 2, we manipulated priority by varying monetary incentives contingent upon successful memory for items tested. Moreover, we illustrate our hypothesis using a simple model that distributed attentional resources across items with rank-ordered priorities. Indeed, we found evidence in both experiments that priority affects the precision of working memory in a monotonic fashion. Our results demonstrate that representations of priority may provide a mechanism by which resources can be allocated to increase the precision with which we encode and briefly store information.

Integration of egocentric and allocentric information during memory-guided reaching to images of a natural environment

When interacting with our environment we generally make use of egocentric and allocentric object information by coding object positions relative to the observer or relative to the environment, respectively. Bayesian theories suggest that the brain integrates both sources of information optimally for perception and action. However, experimental evidence for egocentric and allocentric integration is sparse and has only been studied using abstract stimuli lacking ecological relevance. Here, we investigated the use of egocentric and allocentric information during memory-guided reaching to images of naturalistic scenes. Participants encoded a breakfast scene containing six objects on a table (local objects) and three objects in the environment (global objects). After a 2 s delay, a visual test scene reappeared for 1 s in which 1 local object was missing (= target) and of the remaining, 1, 3 or 5 local objects or one of the global objects were shifted to the left or to the right. The offset of the test scene prompted participants to reach to the target as precisely as possible. Only local objects served as potential reach targets and thus were task-relevant. When shifting objects we predicted accurate reaching if participants only used egocentric coding of object position and systematic shifts of reach endpoints if allocentric information were used for movement planning. We found that reaching movements were largely affected by allocentric shifts showing an increase in endpoint errors in the direction of object shifts with the number of local objects shifted. No effect occurred when one local or one global object was shifted. Our findings suggest that allocentric cues are indeed used by the brain for memory-guided reaching towards targets in naturalistic visual scenes. Moreover, the integration of egocentric and allocentric object information seems to depend on the extent of changes in the scene.

Seek and you shall remember: scene semantics interact with visual search to build better memories

Memorizing critical objects and their locations is an essential part of everyday life. In the present study, incidental encoding of objects in naturalistic scenes during search was compared to explicit memorization of those scenes. To investigate if prior knowledge of scene structure influences these two types of encoding differently, we used meaningless arrays of objects as well as objects in real-world, semantically meaningful images. Surprisingly, when participants were asked to recall scenes, their memory performance was markedly better for searched objects than for objects they had explicitly tried to memorize, even though participants in the search condition were not explicitly asked to memorize objects. This finding held true even when objects were observed for an equal amount of time in both conditions. Critically, the recall benefit for searched over memorized objects in scenes was eliminated when objects were presented on uniform, non-scene backgrounds rather than in a full scene context. Thus, scene semantics not only help us search for objects in naturalistic scenes, but appear to produce a representation that supports our memory for those objects beyond intentional memorization.

Can we throw information out of visual working memory and does this leave informational residue in long-term memory?

Can we entirely erase a temporary memory representation from mind? This question has been addressed in several recent studies that tested the specific hypothesis that a representation can be erased from visual working memory based on a cue that indicated that the representation was no longer necessary for the task. In addition to behavioral results that are consistent with the idea that we can throw information out of visual working memory, recent neurophysiological recordings support this proposal. However, given the infinite capacity of long-term memory, it is unclear whether throwing a representation out of visual working memory really removes its effects on memory entirely. In this paper, we advocate for an approach that examines our ability to erase memory representations from working memory, as well as possible traces that those erased representations leave in long-term memory.

Selective visual processing across competition episodes: a theory of task-driven visual attention and working memory

The goal of this review is to introduce a theory of task-driven visual attention and working memory (TRAM). Based on a specific biased competition model, the ‘theory of visual attention’ (TVA) and its neural interpretation (NTVA), TRAM introduces the following assumption. First, selective visual processing over time is structured in competition episodes. Within an episode, that is, during its first two phases, a limited number of proto-objects are competitively encoded—modulated by the current task—in activation-based visual working memory (VWM). In processing phase 3, relevant VWM objects are transferred via a short-term consolidation into passive VWM. Second, each time attentional priorities change (e.g. after an eye movement), a new competition episode is initiated. Third, if a phase 3 VWM process (e.g. short-term consolidation) is not finished, whereas a new episode is called, a protective maintenance process allows its completion. After a VWM object change, its protective maintenance process is followed by an encapsulation of the VWM object causing attentional resource costs in trailing competition episodes. Viewed from this perspective, a new explanation of key findings of the attentional blink will be offered. Finally, a new suggestion will be made as to how VWM items might interact with visual search processes.

Priorities for selection and representation in natural tasks

Selecting and remembering visual information is an active and competitive process. In natural environments, representations are tightly coupled to task. Objects that are task-relevant are remembered better due to a combination of increased selection for fixation and strategic control of encoding and/or retaining viewed information. However, it is not understood how physically manipulating objects when performing a natural task influences priorities for selection and memory. In this study, we compare priorities for selection and memory when actively engaged in a natural task with first-person observation of the same object manipulations. Results suggest that active manipulation of a task-relevant object results in a specific prioritization for object position information compared with other properties and compared with action observation of the same manipulations. Experiment 2 confirms that this spatial prioritization is likely to arise from manipulation rather than differences in spatial representation in real environments and the movies used for action observation. Thus, our findings imply that physical manipulation of task relevant objects results in a specific prioritization of spatial information about task-relevant objects, possibly coupled with strategic de-prioritization of colour memory for irrelevant objects.

The relationship between visual working memory and attention: retention of precise colour information in the absence of effects on perceptual selection

We examined the conditions under which a feature value in visual working memory (VWM) recruits visual attention to matching stimuli. Previous work has suggested that VWM supports two qualitatively different states of representation: an active state that interacts with perceptual selection and a passive (or accessory) state that does not. An alternative hypothesis is that VWM supports a single form of representation, with the precision of feature memory controlling whether or not the representation interacts with perceptual selection. The results of three experiments supported the dual-state hypothesis. We established conditions under which participants retained a relatively precise representation of a parcticular colour. If the colour was immediately task relevant, it reliably recruited attention to matching stimuli. However, if the colour was not immediately task relevant, it failed to interact with perceptual selection. Feature maintenance in VWM is not necessarily equivalent with feature-based attentional selection.

Selective maintenance in visual working memory does not require sustained visual attention

In four experiments, we tested whether sustained visual attention is required for the selective maintenance of objects in visual working memory (VWM). Participants performed a color change-detection task. During the retention interval, a valid cue indicated the item that would be tested. Change-detection performance was higher in the valid-cue condition than in a neutral-cue control condition. To probe the role of visual attention in the cuing effect, on half of the trials, a difficult search task was inserted after the cue, precluding sustained attention on the cued item. The addition of the search task produced no observable decrement in the magnitude of the cuing effect. In a complementary test, search efficiency was not impaired by simultaneously prioritizing an object for retention in VWM. The results demonstrate that selective maintenance in VWM can be dissociated from the locus of visual attention.