Is a search template an ordinary working memory? Comparing electrophysiological markers of working memory maintenance for visual search and recognition

Skills and cautiousness predict performance in difficult search

People differ in how well they search. What are the factors that might contribute to this variability? We tested the contribution of two cognitive abilities: visual working memory (VWM) capacity and object recognition ability. Participants completed three tasks: a difficult inefficient visual search task, where they searched for a target letter T among skewed L distractors; a VWM task, where they memorized a color array and then identified whether a probed color belonged to the previous array; and the Novel Object Memory Test (NOMT), where they learnt complex novel objects and then identified them amongst objects that closely resembled them. Exploratory and confirmatory factor analyses revealed that there are two latent factors that explain the shared variance among these three tasks: a factor indicative of the level of caution participants exercised during the challenging visual search task, and a factor representing their visual cognitive abilities. People who score high on the search cautiousness tend to perform a more accurate but slower search. People who score high on the visual cognitive ability factor tend to have a higher VWM capacity, a better object recognition ability, and a faster search speed. The results reflect two points: (1) Visual search tasks share components with visual working memory and object recognition tasks. (2) Search performance is influenced not only by the search display's properties but also by individual predispositions such as caution and general visual abilities. This study introduces new factors for consideration when interpreting variations in visual search behaviors.

The reactivation of task rules triggers the reactivation of task-relevant items

Working memory (WM) describes the temporary storage of task-relevant items and procedural rules to guide action. Despite its central importance for goal-directed behavior, the interplay between WM and long-term memory (LTM) remains poorly understood. Recent studies have shown that repeated use of the same task-relevant item in WM results in a hand-off of the storage of that item to LTM, and switching to a new item reactivates WM. To further elucidate the rules governing WM-LTM interactions, we here planned to probe whether a change in task rules, independent of a switch in task-relevant items, would also lead to WM reactivation of maintained items. To this end, we used scalp-recorded electroencephalogram (EEG) data, specifically the contralateral delay activity (CDA), to track WM item storage while manipulating repetitions and changes in task rules and task-relevant items across trials in a visual WM task. We tested two rival hypotheses: If changes in task rules result in a reactivation of the target item representation, then the CDA should increase when a task change is cued even when the same target has been repeated across trials. However, if the reactivation of a task-relevant item only depends on the mnemonic availability of the item itself instead of the task it is used for, then only the changes in task-relevant items should reactivate the representations. Accordingly, the CDA amplitude should decrease for repeated task-relevant items independently of a task change. We found a larger CDA on task-switch compared to task-repeat trials, suggesting that the reactivation of task rules triggers the reactivation of task-relevant items in WM. By demonstrating that WM reactivation of LTM is interdependent for task rules and task-relevant items, this study informs our understanding of visual WM and its interplay with LTM. PREREGISTERED STAGE 1 PROTOCOL: https://osf.io/zp9e8 (date of in-principle acceptance: 19/12/2021).

Behavioral and electrophysiological evidence for the flexible recruitment of feature- and object-based processing in visual working memory comparison

Previous research is inconclusive on when visual working memory (VWM) can be object-based or feature-based. Prior event-related potential (ERP) studies using change detection tasks have found that amplitudes of the N200-an ERP index of VWM comparison- are sensitive to changes in both relevant and irrelevant features, suggesting a bias toward object-based processing. To test whether VWM comparison processing can operate in a feature-based manner, we aimed to create circumstances that would support feature-based processing by: 1) using a strong task-relevance manipulation, and 2) repeating features within a display. Participants completed two blocks of a change detection task for four-item displays in which they were told to respond to color changes (task relevant) but not shape changes (task irrelevant). The first block contained only task-relevant changes to create a strong task-relevance manipulation. In the second block, both relevant and irrelevant changes were present. In both blocks, half of the arrays contained within-display feature repetitions (e.g. two items of the same color or shape). We found that during the second block, N200 amplitudes were sensitive to task-relevant but not irrelevant features regardless of repetition status, consistent with feature-based processing. However, analyses of behavioral data and N200 latencies suggested that object-based processing was occurring at some stages of VWM processing on task-irrelevant feature change trials. In particular, task-irrelevant changes may be processed after no task-relevant feature change is revealed. Overall, the results from the current study suggest that the VWM processing is flexible and can be either object- or feature-based.

The effects of search-irrelevant working memory content on visual search

Previous experiments investigating visual search have shown that distractors that are semantically related to a search target can capture attention and slow the search process. In two experiments, we examine if distractors exactly matching, or semantically related to, search-irrelevant information held in working memory (WM) can also influence visual search while ruling out potential effects of color similarity. Participants first viewed and memorized an image of an everyday object, then they determined if a target item was present or absent in a two-object search array. On exact-match trials, the memorized object appeared as a distractor; on semantic-match trials, an object semantically related to the memorized object appeared as a distractor. Both exact-match and semantic-match distractors slowed search when the target was present in the search array. Our findings extend previous findings by demonstrating WM-driven attentional guidance by complex objects rather than simple features. The results also suggest that visual search can be influenced by distractors sharing only semantic features with a search-irrelevant, but active, WM representation.

Visual working memory representations bias attention more when they are the target of an action plan

Attention has frequently been regarded as an emergent property of linking sensory representations to action plans. It has recently been proposed that similar mechanisms may operate within visual working memory (VWM), such that linking an object in VWM to an action plan strengthens its sensory memory representation, which then expresses as an attentional bias. Here we directly tested this hypothesis by comparing attentional biases induced by VWM representations which were the target of a future action, to those induced by VWM representations that were equally task-relevant, but not the direct target of action. We predicted that the first condition would result in a more prioritized memory state and hence stronger attentional biases. Specifically, participants memorized a geometric shape for a subsequent memory test. At test, in case of a match, participants either had to perform a grip movement on the matching object (action condition), or perform the same movement, but on an unrelated object (control condition). To assess any attentional biases, during the delay period between memorandum and test, participants performed a visual selection task in which either the target was surrounded by the memorized shape (congruent trials) or a distractor (incongruent trials). Eye movements were measured as a proxy for attentional priority. We found a significant interaction for saccade latencies between action condition and shape congruency, reflecting more pronounced VWM-based attentional biases in the action condition. Our results are consistent with the idea that action plans prioritize sensory representations in VWM.

Internal attention is the only retroactive mechanism for controlling precision in working memory

Recent research has suggested that humans can assert control over the precision of working memory (WM) items. However, the mechanisms that enable this control are unclear. While some studies suggest that internal attention improves precision, it may not be the only factor, as previous work also demonstrated that WM storage is disentangled from attention. To test whether there is a precision control mechanism beyond internal attention, we contrasted internal attention and precision requirements within the same trial in three experiments. In every trial, participants memorized two items briefly. Before the test, a retro-cue indicated which item would be tested first, thus should be attended. Importantly, we encouraged participants to store the unattended item with higher precision by testing it using more similar lure colors at the probe display. Accuracy was analyzed on a small proportion of trials where the target-lure similarity, hence the task difficulty, was equal for attended and unattended items. Experiments 2 and 3 controlled for output interference by the first test and involuntary precision boost by the retro-cue, respectively. In all experiments, the unattended item had lower accuracy than the attended item, suggesting that individuals were not able to remember it more precisely than the attended item. Thus, we conclude that there is no precision control mechanism beyond internal attention, highlighting the close relationship between attentional and qualitative prioritization within WM. We discuss the important implications of these findings for our understanding of the fundamentals of WM and WM-driven behaviors.

Storage in Visual Working Memory Recruits a Content-Independent Pointer System

Past work has shown that storage in working memory elicits stimulus-specific neural activity that tracks the stored content. Here, we present evidence for a distinct class of load-sensitive neural activity that indexes items without representing their contents per se. We recorded electroencephalogram (EEG) activity while adult human subjects stored varying numbers of items in visual working memory. Multivariate analysis of the scalp topography of EEG voltage enabled precise tracking of the number of individuated items stored and robustly predicted individual differences in working memory capacity. Critically, this signature of working memory load generalized across variations in both the type and number of visual features stored about each item, suggesting that it tracked the number of individuated memory representations and not the content of those memories. We hypothesize that these findings reflect the operation of a capacity-limited pointer system that supports on-line storage and attentive tracking.

The presence of a distractor matching the content of working memory induces delayed quitting in visual search

Previous research has shown that the presence of a distractor object matching the current content of working memory interacts with visual search. Because finding a target and quitting a search without finding a target may be implemented by qualitatively different processes, it is possible that the effects of a memory-matching distractor on target-present trials and on target-absent trials reveal different mechanisms by which the memory-matching distractor interacts with visual search. Although previous studies have well established the effect of attentional capture by a memory-matching distractor when the target object is found in the search display, there remains an open question about whether the presence of a memory-matching distractor can affect the process of search termination when no target is found. In the present study, we showed that search termination times on target-absent trials were delayed by the presence of a distractor matching the content of visual working memory. This delayed quitting effect cannot be conceived of as a more general influence of visual short-term memory, because the presence of a distractor matching the content of passive visual short-term memory (i.e., visual priming) did not influence quitting behavior in visual search. These findings offer a novel perspective that distractors matching the information maintained in visual working memory can cause observers to delay search termination when no target has been found.

The impact of retro-cue validity on working memory representation: Evidence from electroencephalograms

Visual working memory (VWM) performance can be improved by retrospectively cueing an item. The validity of retro-cues has an impact on the mechanisms underlying the retro-cue effect, but how non-cued representations are handled under different retro-cue validity conditions is not yet clear. Here, we used electroencephalograms to investigate whether retro-cue validity can affect the fate of non-cued representations in VWM. The participants were required to perform a change-detection task using a retro-cue with 80% or 20% validity. Contralateral delay activity and the lateralized alpha power were used to assess memory storage and selective attention, respectively. The retro-cue could redirect selective attention to the cued item under both validity conditions; however, the participants maintained the non-cued representations under the low-validity condition but dropped them from VWM under the high-validity condition. These results suggest that the maintenance of non-cued representations in VWM is affected by the expectation of cue validity and may be partially strategically driven. DATA AVAILABILITY: The datasets generated/analyzed during this study and experimental script have been added to https://osf.io/qtwc9/.

A relational account of visual short-term memory (VSTM)

Visual short-term memory (VSTM) is an important resource that allows temporarily storing visual information. Current theories posit that elementary features (e.g., red, green) are encoded and stored independently of each other in VSTM. However, they have difficulty explaining the similarity effect, that similar items can be remembered better than dissimilar items. In Experiment 1, we tested (N = 20) whether the similarity effect may be due to storing items in a context-dependent manner in VSTM (e.g., as the reddest/yellowest item). In line with a relational account of VSTM, we found that the similarity effect is not due to feature similarity, but to an enhanced sensitivity for detecting changes when the relative colour of a to-be-memorised item changes (e.g., from reddest to not-reddest item; than when an item underwent the same change but retained its relative colour; e.g., still reddest). Experiment 2 (N = 20) showed that VSTM load, as indexed by the CDA amplitude in the EEG, was smaller when the colours were ordered so that they all had the same relationship than when the same colours were out-of-order, requiring encoding different relative colours. With this, we report two new effects in VSTM - a relational detection advantage that describes an enhanced sensitivity to relative changes in change detection, and a relational CDA effect, which reflects that VSTM load, as indexed by the CDA, scales with the number of (different) relative features between the memory items. These findings support a relational account of VSTM and question the view that VSTM stores features such as colours independently of each other.

Guided Search 6.0: An updated model of visual search

This paper describes Guided Search 6.0 (GS6), a revised model of visual search. When we encounter a scene, we can see something everywhere. However, we cannot recognize more than a few items at a time. Attention is used to select items so that their features can be "bound" into recognizable objects. Attention is "guided" so that items can be processed in an intelligent order. In GS6, this guidance comes from five sources of preattentive information: (1) top-down and (2) bottom-up feature guidance, (3) prior history (e.g., priming), (4) reward, and (5) scene syntax and semantics. These sources are combined into a spatial "priority map," a dynamic attentional landscape that evolves over the course of search. Selective attention is guided to the most active location in the priority map approximately 20 times per second. Guidance will not be uniform across the visual field. It will favor items near the point of fixation. Three types of functional visual field (FVFs) describe the nature of these foveal biases. There is a resolution FVF, an FVF governing exploratory eye movements, and an FVF governing covert deployments of attention. To be identified as targets or rejected as distractors, items must be compared to target templates held in memory. The binding and recognition of an attended object is modeled as a diffusion process taking > 150 ms/item. Since selection occurs more frequently than that, it follows that multiple items are undergoing recognition at the same time, though asynchronously, making GS6 a hybrid of serial and parallel processes. In GS6, if a target is not found, search terminates when an accumulating quitting signal reaches a threshold. Setting of that threshold is adaptive, allowing feedback about performance to shape subsequent searches. Simulation shows that the combination of asynchronous diffusion and a quitting signal can produce the basic patterns of response time and error data from a range of search experiments.

Not Just a Pot: Visual Episodic Memory in Cannabis Users and Polydrug Cannabis Users: ROC and ERP Preliminary Investigation

Background While research has consistently identified an association between long-term cannabis use and memory impairments, few studies have examined this relationship in a polydrug context (i.e., when combining cannabis with other substances). Aims: In this preliminary study, we used event-related potentials to examine the recognition process in a visual episodic memory task in cannabis users (CU) and cannabis polydrug users (PU). We hypothesized that CU and PU will have both-behavioral and psychophysiological-indicators of memory processes affected, compared to matched non-using controls with the PU expressing more severe changes. Methods 29 non-using controls (CG), 24 CU and 27 PU were enrolled into the study. All participants completed a visual learning recognition task while brain electrical activity was recorded. Event-related potentials were calculated for familiar (old) and new images from a signal recorded during a subsequent recognition test. We used receiver operating characteristic curves for behavioral data analysis. Results The groups did not differ in memory performance based on receiver operating characteristic method in accuracy and discriminability indicators nor mean reaction times for old/new images. The frontal old/new effect expected from prior research was observed for all participants, while a parietal old/new effect was not observed. While, the significant differences in the late parietal component (LPC) amplitude was observed between CG and PU but not between CG and CU nor CU and PU. Linear regression analysis was used to examine the mean amplitude of the LPC component as a predictor of memory performance accuracy indicator. LPC amplitude predicts recognition accuracy only in the CG. Conclusion The results showed alterations in recognition memory processing in CU and PU groups compared to CG, which were not manifested on the behavioral level, and were the most prominent in cannabis polydrug users. We interpret it as a manifestation of the cumulative effect of multiple drug usage in the PU group.

Estimating the statistical power to detect set-size effects in contralateral delay activity

The contralateral delay activity (CDA) is an event-related potential component commonly used to examine the online processes of visual working memory. Here, we provide a robust analysis of the statistical power that is needed to achieve reliable and reproducible results with the CDA. Using two very large EEG datasets that examined the contrast between CDA amplitude with set sizes 2 and 6 items and set sizes 2 and 4 items, we present a subsampling analysis that estimates the statistical power achieved with varying numbers of subjects and trials based on the proportion of significant tests in 10,000 iterations. We also generated simulated data using Bayesian multilevel modeling to estimate power beyond the bounds of the original datasets. The number of trials and subjects required depends critically on the effect size. Detecting the presence of the CDA-a reliable difference between contralateral and ipsilateral electrodes during the memory period-required only 30-50 clean trials with a sample of 25 subjects to achieve approximately 80% statistical power. However, for detecting a difference in CDA amplitude between two set sizes, a substantially larger number of trials and subjects were required; approximately 400 clean trials with 25 subjects to achieve 80% power. Thus, to achieve robust tests of how CDA activity differs across conditions, it is essential to be mindful of the estimated effect size. We recommend researchers designing experiments to detect set-size differences in the CDA collect substantially more trials per subject.

How retaining objects containing multiple features in visual working memory regulates the priority for access to visual awareness

The content of visual working memory influences the access to visual awareness. Thus far, research has focused on retention of a single feature, whereas memoranda in real life typically contain multiple features. Here, we intermixed a delayed match-to-sample task to manipulate VWM content, and a breaking Continuous Flash Suppression (b-CFS) task to measure prioritization for visual awareness. Observers memorized either the color (Exp. 1), the shape (Exp. 2) or both the features (Exp. 3) of an item and indicated the location of a suppressed target. We observed that color-matching targets broke suppression faster than color-mismatching targets both when color was memory relevant or irrelevant. Shape only impacted priority for visual awareness through an interaction with color. We conclude that: (1) VWM can regulate the priority of visual information to access visual awareness along a single feature dimension; (2) different features of a memorandum vary in their potency to impact access to visual awareness, and the more dominant feature may even suppress the effect of the less dominant feature; (3) even stimuli that match an irrelevant feature dimension of the memorandum can be prioritized for visual awareness.

Bottom-up perceptual salience and top-down retro-cues concurrently determine state in visual working memory

Recent studies have demonstrated that in visual working memory (VWM), only items in an active state can guide attention. Further evidence has revealed that items with higher perceptual salience or items prioritised by a valid retro-cue in VWM tend to be in an active state. However, it is unclear which factor (perceptual salience or retro-cues) is more important for influencing the item state in VWM or whether the factors can act concurrently. Experiment 1 examined the role of perceptual salience by asking participants to hold two features with relatively different perceptual salience (colour vs. shape) in VWM while completing a visual search task. Guidance effects were found when either colour or both colour and shape in VWM matched one of the search distractors but not when shape matched. This demonstrated that the more salient feature in VWM can actively guide attention, while the less salient feature cannot. However, when shape in VWM was cued to be more relevant (prioritised) in Experiment 2, we found guidance effects in both colour-match and shape-match conditions. That is, both more salient but non-cued colour and less salient but cued shape could be active in VWM, such that attentional selection was affected by the matching colour or shape in the visual search task. This suggests that bottom-up perceptual salience and top-down retro-cues can jointly determine the active state in VWM.

Visual Search: How Do We Find What We Are Looking For?

In visual search tasks, observers look for targets among distractors. In the lab, this often takes the form of multiple searches for a simple shape that may or may not be present among other items scattered at random on a computer screen (e.g., Find a red T among other letters that are either black or red.). In the real world, observers may search for multiple classes of target in complex scenes that occur only once (e.g., As I emerge from the subway, can I find lunch, my friend, and a street sign in the scene before me?). This article reviews work on how search is guided intelligently. I ask how serial and parallel processes collaborate in visual search, describe the distinction between search templates in working memory and target templates in long-term memory, and consider how searches are terminated.

Working memory is corrupted by strategic changes in search templates

When searching for a specific object, we often form an image of the target, which we use as a search template. This template is thought to be maintained in working memory, primarily because of evidence that the contents of working memory influences search behavior. However, it is unknown whether this interaction applies in both directions. Here, we show that changes in search templates influence working memory. Participants were asked to remember the orientation of a line that changed every trial, and on some trials (75%) search for that orientation, but on remaining trials recall the orientation. Critically, we manipulated the target template by introducing a predictable context—distractors in the visual search task were always counterclockwise (or clockwise) from the search target. The predictable context produced a large bias in search. Importantly, we also found a similar bias in orientation memory reports, demonstrating that working memory and target templates were not held as completely separate, isolated representations. However, the memory bias was considerably smaller than the search bias, suggesting that, although there is a common source, the two may not be driven by a single, shared process.

Tension Experience Induced By Nested Structures In Music

Tension experience is the basis for music emotion. In music, discrete elements are always organized into complex nested structures to convey emotion. However, the processing of music tension in the nested structure remains unknown. The present study investigated the tension experience induced by the nested structure and the underlying neural mechanisms, using a continuous tension rating task and electroencephalography (EEG) at the same time. Thirty musicians listened to music chorale sequences with non-nested, singly nested and doubly nested structures and were required to rate their real-time tension experience. Behavioral data indicated that the tension experience induced by the nested structure had more fluctuations than the non-nested structure, and the difference was mainly exhibited in the process of tension induction rather than tension resolution. However, the EEG data showed that larger late positive components (LPCs) were elicited by the ending chords in the nested structure compared with the non-nested structure, reflecting the difference in cognitive integration for long-distance structural dependence. The discrepancy between resolution experience and neural responses revealed the non-parallel relations between emotion and cognition. Furthermore, the LPC elicited by the doubly nested structure showed a smaller scalp distribution than the singly nested structure, indicating the more difficult processing of the doubly nested structure. These findings revealed the dynamic tension experience induced by the nested structure and the influence of nested type, shedding new light on the relationship between structure and tension in music.

Probing the relationship between late endogenous ERP components with fluid intelligence in healthy older adults

The world population is rapidly aging, bringing together the necessity to better understand the advancing age. This characterization may be used to aid early diagnosis and to guide individually-tailored interventions. While some event-related potential (ERP) components, such as the P300 and late positive complex (LPC), have been associated with fluid intelligence (Gf) in young population; little is known whether these associations hold for older people. Therefore, the main goal of this study was to assess whether these ERP components are associated with Gf in the elderly. Fifty-seven older adults performed a continuous performance task (CPT) and a visual oddball paradigm while EEG was recorded. Participants were divided into two groups, according to their performance in the Raven’s Advanced Progressive Matrices test: high-performance (HP) and low-performance (LP). Results showed that the HP group, compared to the LP group, had higher LPC amplitudes in the CPT and shorter P300 latencies in the oddball task, highlighting the role of ERP components as a potential electrophysiological proxy of Gf abilities in the elderly.

Behavioral and electrophysiological evidence for a dissociation between working memory capacity and feature-based attention

When attending to visual objects with particular features, neural processing is typically biased toward those features. Previous work has suggested that maintaining such feature-based attentional sets may involve the same neural resources as visual working memory. If so, the extent to which feature-based attention influences stimulus processing should be related to individuals' working memory capacity. Here we used electroencephalography (EEG) to record brain activity in 60 human observers while they monitored stimulus streams for targets of a specific color. Distractors presented at irrelevant locations evoked strong electrophysiological markers of attentional signal enhancement (the N2pc and P_D components) despite producing little or no behavioral interference. Critically, there was no relationship between individual differences in the magnitude of these feature-based biases on distractor processing and individual differences in working memory capacity as measured using three separate working memory tasks. Bayes factor analyses indicated substantial evidence in support of the null hypothesis of no relationship between working memory capacity and the effects of feature-based attention on distractor processing. We consider three potential explanations for these findings. One is that working memory and feature-based attention draw upon distinct neural resources, contrary to previous claims. A second is that working memory is only related to feature-based attention when the attentional template has recently changed. A third is that feature-based attention tasks of the kind employed in the current study recruit just one of several subcomponents of working memory, and so are not invariably correlated with an individual's overall working memory capacity.

Visual working memory and action: Functional links and bi-directional influences

Working memory bridges perception to action over extended delays, enabling flexible goal-directed behaviour. To date, studies of visual working memory – concerned with detailed visual representations such as shape and colour – have considered visual memory predominantly in the context of visual task demands, such as visual identification and search. Another key purpose of visual working memory is to directly inform and guide upcoming actions. Taking this as a starting point, I review emerging evidence for the pervasive bi-directional links between visual working memory and (planned) action, and discuss these links from the perspective of their common goal of enabling flexible and precise behaviour.

No evidence for mnemonic modulation of interocularly suppressed visual input

Visual working memory (VWM) allows for keeping visual information available for upcoming goal-directed behavior, while new visual input is processed concurrently. Interactions between the mnemonic and perceptual systems cause VWM to affect the processing of visual input in a content-specific manner: visual input that is initially suppressed from consciousness is detected faster when it matches rather than mismatches the content of VWM. It is currently under debate whether such mnemonic influences on perception occur prior to or after conscious access. To address this issue, we investigated whether VWM content modulates the neural response to visual input that remains suppressed from consciousness. We measured fMRI responses to interocularly suppressed stimuli in 20 human participants performing a delayed match-to-sample task: Participants were retro-cued to memorize one of two geometrical shapes for subsequent recognition. During retention, an interocularly suppressed peripheral stimulus (the probe) was briefly presented, which was either of the cued (memorized) or uncued (not memorized) shape category. We found no evidence that VWM content modulated the neural response to the probe. Substantial evidence for the absence of this modulation was found despite leveraging a highly liberal analysis approach: (1) selecting regions of interest that were particularly prone to detecting said modulation, and (2) using directional Bayesian tests favoring the presence of the hypothesized modulation. We did observe faster detection of memory-matching compared to memory-mismatching probes in a behavioral control experiment, thus validating the stimulus set. We conclude that VWM impacts the processing of visual input only once suppression is mostly alleviated.

Preparation for upcoming attentional states in the hippocampus and medial prefrontal cortex

Goal-directed attention is usually studied by providing individuals with explicit instructions on what they should attend to. But in daily life, we often use past experiences to guide our attentional states. Given the importance of memory for predicting upcoming events, we hypothesized that memory-guided attention is supported by neural preparation for anticipated attentional states. We examined preparatory coding in the human hippocampus and mPFC, two regions that are important for memory-guided behaviors, in two tasks: one where attention was guided by memory and another in which attention was explicitly instructed. Hippocampus and mPFC exhibited higher activity for memory-guided vs. explicitly instructed attention. Furthermore, representations in both regions contained information about upcoming attentional states. In the hippocampus, this preparation was stronger for memory-guided attention, and occurred alongside stronger coupling with visual cortex during attentional guidance. These results highlight the mechanisms by which memories are used to prepare for upcoming attentional goals.

Encoding, storage, and response preparation-Distinct EEG correlates of stimulus and action representations in working memory

Working memory (WM) allows for the active storage of stimulus- and higher level representations, such as action plans. This electroencephalography (EEG) study investigated the specific electrophysiological correlates dissociating action-related from stimulus-related representations in WM using three different experimental conditions based on the same stimulus material. In the experiment, a random sequence of single numbers (from 1 to 6) was presented and participants had to indicate whether the current number (N0 condition), the preceding number (N-1 condition), or the sum of the current and the preceding number (S-1 condition) was odd or even. Accordingly, participants had to store a stimulus representation in S-1 and an action representation in N-1 until the onset of the next stimulus. In the EEG, the storage of stimulus representations (S-1) was reflected by a fronto-central slow wave indicating the rehearsal of information that was required for the response in the following trial. In contrast, the storage of action representations (N-1) went along with a posterior positive slow wave, suggesting that the action plan was actively stored in WM until the presentation of the next stimulus. Crucially, preparing for the next response in N-1 was associated with increased contralateral mu/beta suppression, predicting the response time in the given trial. Our findings, thus, show that the WM processes for stimulus- and action representations can be clearly dissociated from each other with a distinct sequence of EEG correlates for encoding, storage, and response preparation.

What not to look for: Electrophysiological evidence that searchers prefer positive templates

To-be-attended information can be specified either with positive cues (I'll be wearing a blue shirt) or with negative cues (I won't be wearing a red shirt). Numerous experiments have found that positive cues help search more than negative cues. Given that negative cues produce smaller benefits compared to positive cues, it stands to reason that searchers may choose to use positive templates instead of negative templates if given the opportunity. Here, we evaluate this possibility with behavioral measures as well as by directly measuring the formation of positive and negative templates with event-related potentials. Analysis of the contralateral delay activity (CDA) elicited by cues revealed that positive and negative templates relied on working memory to the same extent, even when negative working memory templates could have been circumvented by relying on long-term memories of target colors. Whereas the CDA did not discriminate positive and negative templates, a CNV-like potential did, suggesting cognitive differences between positive and negative templates beyond visual working memory. However, when both positive and negative information were presented in each cue, participants preferred to make use of the positive cues, as indicated by a CDA contralateral to the positive color in negative cue blocks, and a lack of search benefits for positive- and negative-color cues relative to positive-color cues alone. Our results show that searchers elect to selectively encode only positive information into visual working memory when both positive and negative information are available.

Scenes Modulate Object Processing Before Interacting With Memory Templates

When searching for relevant objects in our environment (say, an apple), we create a memory template (a red sphere), which causes our visual system to favor template-matching visual input (applelike objects) at the expense of template-mismatching visual input (e.g., leaves). Although this principle seems straightforward in a lab setting, it poses a problem in naturalistic viewing: Two objects that have the same size on the retina will differ in real-world size if one is nearby and the other is far away. Using the Ponzo illusion to manipulate perceived size while keeping retinal size constant, we demonstrated across 71 participants that visual objects attract attention when their perceived size matches a memory template, compared with mismatching objects that have the same size on the retina. This shows that memory templates affect visual selection after object representations are modulated by scene context, thus providing a working mechanism for template-based search in naturalistic vision.

Humans can efficiently look for but not select multiple visual objects

The human brain recurrently prioritizes task-relevant over task-irrelevant visual information. A central question is whether multiple objects can be prioritized simultaneously. To answer this, we let observers search for two colored targets among distractors. Crucially, we independently varied the number of target colors that observers anticipated, and the number of target colors actually used to distinguish the targets in the display. This enabled us to dissociate the preparation of selection mechanisms from the actual engagement of such mechanisms. Multivariate classification of electroencephalographic activity allowed us to track selection of each target separately across time. The results revealed only small neural and behavioral costs associated with preparing for selecting two objects, but substantial costs when engaging in selection. Further analyses suggest this cost is the consequence of neural competition resulting in limited parallel processing, rather than a serial bottleneck. The findings bridge diverging theoretical perspectives on capacity limitations of feature-based attention.

Prospectively reinstated memory drives conscious access of matching visual input

Maintaining information in visual working memory (VWM) biases attentional selection of concurrent visual input, by favoring VWM-matching over VWM-mismatching visual input. Recently, it was shown that this bias disappears when the same item is memorized on consecutive occasions (as memoranda presumably transit from VWM to long-term memory), but reemerges when observers anticipate to memorize a novel item on a subsequent trial. Here, we aimed to conceptually replicate and extend this intriguing finding, by investigating whether prospectively reinstated memory drives conscious access of memory-matching visual input. We measured the time it took for participants to detect interocularly suppressed target stimuli, which were either from the same color category as a concurrently memorized color or not. Our results showed that the advantage of memory-matching targets in overcoming suppression progresses non-monotonically across consecutive memorizations of the same color ('repetitions'): the advantage for memory-matching visual input initially declined to asymptote, before being fully revived on the last repetition. This revival was not observed in a control experiment in which targets were not interocularly suppressed. The results suggest that, as observers anticipate to memorize a novel item imminently, VWM usage is prospectively reinstated, causing memory-matching visual input to gain accelerated access to consciousness again.

Distinct prioritization of visual working memory representations for search and for recall

Despite the inherent limitations of visual working memory (VWM), it effectively supports numerous everyday behaviors - capabilities that are due, in part, to its flexibility. An observer can flexibly prioritize VWM representations to support at least two behavioral outcomes: An item can be prioritized to enhance its representational quality, thereby enhancing recall precision, and an item can be granted "template status," allowing it to bias attention during visual search, speeding search for matching targets. Here we examined the relationship between these two forms of prioritization. Research has shown that a byproduct of granting an item template status is that its precision is enhanced; however, it is unclear if the inverse is also true: Does prioritizing an item for enhanced representational quality cause that item to bias attention? In the present study, participants remembered the colors of two squares for a subsequent recall task, and one was cued, indicating it was 80% likely to be the target of the memory test. To assess template status, a subset of trials ended in a visual search task in which a colored singleton distractor matched the color of the cued item in memory, the non-cued item, or an unrelated color. We found that, although the cue was effective at enhancing recall precision of the cued item, it had no systematic effect on which of the two memory items was granted template status. Thus, we conclude that the two forms of prioritization in VWM - prioritization for recall and for search - are distinct.

Multiple representations in visual working memory simultaneously guide attention: The type of memory-matching representation matters

Whether multiple visual working memory (VWM) representations can simultaneously become active templates to guide attention is controversial. The single-item-template hypothesis argues that only one VWM representation can be active at a time, whereas the multiple-item-template hypothesis argues that multiple VWM templates can simultaneously guide attention. The present study examined the two hypotheses in three (out of four) experiments, using three different types of memory objects: Experiment 1: shapes; Experiment 2: colors; and Experiment 3: colored shapes. Participants were required to hold one (memory-1) or two objects (memory-2) in VWM while performing a tilted line search task. Zero (match-0), one (match-1), or two (match-2) memory stimuli reappeared as distractors in the search array. Guidance effects were found for each type of memory stimuli. More importantly, the guidance effect for memory-2/match-2 trials was significantly larger than that for memory-2/match-1 and memory-1/match-1 trials when holding two colors or two colored shapes in VWM, which is in line with the multiple-item-template hypothesis. However, the pattern of simultaneous guidance effect is not perfectly found for two memory shapes, which may indicate that a reliable simultaneous guidance effect from two representations in VWM can be observed only when the memory-matching stimuli is more effective in guiding attention. Experiment 4 directly compared the guidance effect induced by feature-based matches (partial matching) with the guidance effect induced object-based matches (complete matching) in memory-set-size 2. Reliable guidance effects in match-1 and match-2 trials for object-based matches but not for feature-based matches confirmed the crucial role of the type of memory-matching stimuli in guiding attention.

Priority Switches in Visual Working Memory are Supported by Frontal Delta and Posterior Alpha Interactions

Visual working memory (VWM) distinguishes between representations relevant for imminent versus future perceptual goals. We investigated how the brain sequentially prioritizes visual working memory representations that serve consecutive tasks. Observers remembered two targets for a sequence of two visual search tasks, thus making one target currently relevant, and the other prospectively relevant. We show that during the retention interval prior to the first search, lateralized parieto-occipital EEG alpha (8-14 Hz) suppression is stronger for current compared with prospective search targets. Crucially, between the first and second search task, this difference in posterior alpha lateralization reverses, reflecting the change in priority states of the two target representations. Connectivity analyses indicate that this switch in posterior alpha lateralization is driven by frontal delta/low-theta (2-6 Hz) activity. Moreover, this frontal low-frequency signal also predicts task performance after the switch. We thus obtained evidence for large-scale network interactions underlying the flexible shifting between the priority states of multiple memory representations in VWM.

Hide and seek: Directing top-down attention is not sufficient for accelerating conscious access

At any moment in time, we have a single conscious visual experience representing a minute part of our visual world. As such, the visual input stimulating our retinae is in continuous competition for reaching conscious access. Many complex cognitive operations can only be applied to consciously accessible visual information, thereby raising the question whether humans have the ability to select which parts of their visual input reaches consciousness. Top-down attention allows humans to flexibly assign more processing resources to certain parts of our visual input, making it a likely mechanism to volitionally bias conscious access. Here, we investigated whether directing top-down attention to a particular location or feature accelerates conscious access of an initially suppressed visual stimulus at the attended location, or of the attended feature. We instructed participants to attend a spatial location (Experiment 1) or color (Experiment 2) for a speeded discrimination task, using a highly predictive cue. The predictive cues were highly effective in prompting sustained attention towards the cued location or color, as evidenced by faster discrimination of cued relative to uncued targets. We simultaneously measured detection times to interocularly suppressed probes that were either of the cued (i.e., attended) color/location or not, and were visually distinct from the targets used for the discrimination task. Despite our successful manipulation of top-down attention, suppressed probes were not released from suppression faster when they were presented at the attended location, or in the attended color. In contrast, when observers were cued to attend a color for locating targets of an ill-defined shape (inciting a broader attentional template), we did observe faster conscious access of probes in the attended color (Experiment 3). We discuss our findings in light of the specificity of attentional templates, and the inherent limitations that this poses for top-down attentional biases on conscious access.

Behavioral decoding of working memory items inside and outside the focus of attention

How we attend to our thoughts affects how we attend to our environment. Holding information in working memory can automatically bias visual attention toward matching information. By observing attentional biases on reaction times to visual search during a memory delay, it is possible to reconstruct the source of that bias using machine learning techniques and thereby behaviorally decode the content of working memory. Can this be done when more than one item is held in working memory? There is some evidence that multiple items can simultaneously bias attention, but the effects have been inconsistent. One explanation may be that items are stored in different states depending on the current task demands. Recent models propose functionally distinct states of representation for items inside versus outside the focus of attention. Here, we use behavioral decoding to evaluate whether multiple memory items-including temporarily irrelevant items outside the focus of attention-exert biases on visual attention. Only the single item in the focus of attention was decodable. The other item showed a brief attentional bias that dissipated until it returned to the focus of attention. These results support the idea of dynamic, flexible states of working memory across time and priority.

Whatever you do, don't look at the...: Evaluating guidance by an exclusionary attentional template

People can use a target template consisting of one or more features to guide attention and gaze to matching objects in a search array. But can we also use feature information to guide attention away from known irrelevant items? Some studies found a benefit from foreknowledge of a distractor feature, whereas others found a cost. Importantly, previous work has largely relied on end-of-trial manual responses; it is unclear how feature-guided avoidance might unfold as candidate objects are inspected. In the current experiments, participants were cued with a distractor feature to avoid, then performed a visual search task while eye movements were recorded. Participants initially fixated a to-be-avoided object more frequently than predicted by chance, but they also demonstrated avoidance of cue-matching objects later in the trial. When provided more time between cue stimulus and search array, participants continued to be initially captured by a cued-color item. Furthermore, avoidance of cue-matching objects later in the trial was not contingent on initial capture by a cue-matching object. These results suggest that the conflicting findings in previous negative-cue experiments may be explained by a mixture of two independent processes: initial attentional capture by memory-matching items and later avoidance of known irrelevant items. (PsycINFO Database Record

Posterior α EEG Dynamics Dissociate Current from Future Goals in Working Memory-Guided Visual Search

Current models of visual search assume that search is guided by an active visual working memory representation of what we are currently looking for. This attentional template for currently relevant stimuli can be dissociated from accessory memory representations that are only needed prospectively, for a future task, and that should be prevented from guiding current attention. However, it remains unclear what electrophysiological mechanisms dissociate currently relevant (serving upcoming selection) from prospectively relevant memories (serving future selection). We measured EEG of 20 human subjects while they performed two consecutive visual search tasks. Before the search tasks, a cue instructed observers which item to look for first (current template) and which second (prospective template). During the delay leading up to the first search display, we found clear suppression of α band (8-14 Hz) activity in regions contralateral to remembered items, comprising both local power and interregional phase synchronization within a posterior parietal network. Importantly, these lateralization effects were stronger when the memory item was currently relevant (i.e., for the first search) compared with when it was prospectively relevant (i.e., for the second search), consistent with current templates being prioritized over future templates. In contrast, event-related potential analysis revealed that the contralateral delay activity was similar for all conditions, suggesting no difference in storage. Together, these findings support the idea that posterior α oscillations represent a state of increased processing or excitability in task-relevant cortical regions, and reflect enhanced cortical prioritization of memory representations that serve as a current selection filter.SIGNIFICANCE STATEMENT Our days are filled with looking for relevant objects while ignoring irrelevant visual information. Such visual search activity is thought to be driven by current goals activated in working memory. However, working memory not only serves current goals, but also future goals, with differential impact upon visual selection. Little is known about how the brain differentiates between current and future goals. Here we show, for the first time, that modulations of brain oscillations in the EEG α frequency band in posterior cortex can dissociate current from future search goals in working memory. Moreover, the dynamics of these oscillations uncover how we flexibly switch focus between memory representations. Together, we reveal how the brain assigns priority for selection.

Local and interregional alpha EEG dynamics dissociate between memory for search and memory for recognition

Attention during visual search is thought to be guided by an active visual working memory (VWM) representation of the search target. We tested the hypothesis that a VWM representation used for searching a target among competing information (a "search template") is distinct from VWM representations used for simple recognition tasks, without competition. We analyzed EEG from 20 human participants while they performed three different VWM-based visual detection tasks. All tasks started with identical lateralized VWM cues, but differed with respect to the presence and nature of competing distractors during the target display at test, where participants performed a simple recognition task without distractors, or visual search in pop-out (distinct) and serial (non-distinct) search displays. Performance was worst for non-distinct search, and best for simple recognition. During the one second delay period between cue and test, we observed robust suppression of EEG dynamics in the alpha (8-14Hz) band over parieto-occipital sites contralateral to the relevant VWM item, both in terms of local power as well as interregional phase synchrony within a posterior-parietal network. Importantly, these lateralization dynamics were more strongly expressed prior to search compared to simple recognition. Furthermore, before the VWM cue, alpha phase synchrony between prefrontal and mid-posterior-parietal sites was strongest for non-distinct search, reflecting enhanced anticipatory control prior to VWM encoding. Directional connectivity analyses confirmed this effect to be in an anterior-to-posterior direction. Together, these results provide evidence for frontally mediated top-down control of VWM in preparation of visual search.

Electrophysiological evidence that top-down knowledge controls working memory processing for subsequent visual search

Items in working memory guide visual attention toward a memory-matching object. Recent studies have shown that when searching for an object this attentional guidance can be modulated by knowing the probability that the target will match an item in working memory. Here, we recorded the P3 and contralateral delay activity to investigate how top-down knowledge controls the processing of working memory items. Participants performed memory task (recognition only) and memory-or-search task (recognition or visual search) in which they were asked to maintain two colored oriented bars in working memory. For visual search, we manipulated the probability that target had the same color as memorized items (0, 50, or 100%). Participants knew the probabilities before the task. Target detection in 100% match condition was faster than that in 50% match condition, indicating that participants used their knowledge of the probabilities. We found that the P3 amplitude in 100% condition was larger than in other conditions and that contralateral delay activity amplitude did not vary across conditions. These results suggest that more attention was allocated to the memory items when observers knew in advance that their color would likely match a target. This led to better search performance despite using qualitatively equal working memory representations.

Learning changes the attentional status of prospective memories

Objects in visual working memory (VWM) that are only prospectively relevant can nevertheless affect the guidance of attention in an ongoing visual search task. Here we investigated whether learning changes the attentional status of such prospective memories. Observers performed a visual search while holding an item in memory for a later memory test. This prospective memory was then repeated for several trials. When the memory was new, it interfered with the ongoing search task. However, with repetition, memory performance increased but memory-based interference rapidly diminished, suggesting that observers learned to shield the prospective memory from the ongoing task. This contrasts with earlier findings showing stronger attentional biases from learned memories when these are immediately task-relevant. Interestingly, interference resurfaced again in anticipation of a new memory, suggesting a reactivation of VWM. These effects were sensitive to task context, indicating that the attentional status of prospective memories is flexible.

The contralateral delay activity as a neural measure of visual working memory

The contralateral delay activity (CDA) is a negative slow wave sensitive to the number of objects maintained in visual working memory (VWM). In recent years, a growing number of labs started to use the CDA in order to investigate VWM, leading to many fascinating discoveries. Here, we discuss the recent developments and contribution of the CDA in various research fields. Importantly, we report two meta-analyses that unequivocally validate the relationship between the set-size increase in the CDA amplitude and the individual VWM capacity, and between the CDA and filtering efficiency. We further discuss how the CDA was used to study the role of VWM in visual search, multiple object tracking, grouping, binding, and whether VWM capacity allocation is determined by the items' resolution or instead by the number of objects regardless of their complexity. In addition, we report how the CDA has been used to characterize specific VWM deficits in special populations.