Allocation of resources in working memory: Theoretical and empirical implications for visual search

Top-down suppression of negative features applies flexibly contingent on visual search goals

Visually searching for a frequently changing target is assumed to be guided by flexible working memory representations of specific features necessary to discriminate targets from distractors. Here, we tested if these representations allow selective suppression or always facilitate perception based on search goals. Participants searched for a target (i.e., a horizontal bar) defined by one of two different negative features (e.g., not red vs. not blue; Experiment 1) or a positive (e.g., blue) versus a negative feature (Experiments 2 and 3). A prompt informed participants about the target identity, and search tasks alternated or repeated randomly. We used different peripheral singleton cues presented at the same (valid condition) or a different (invalid condition) position as the target to examine if negative features were suppressed depending on current instructions. In all experiments, cues with negative features elicited slower search times in valid than invalid trials, indicating suppression. Additionally, suppression of negative color cues tended to be selective when participants searched for the target by different negative features but generalized to negative and non-matching cue colors when switching between positive and negative search criteria was required. Nevertheless, when the same color - red - was used in positive and negative search tasks, red cues captured attention or were suppressed depending on whether red was positive or negative (Experiment 3). Our results suggest that working memory representations flexibly trigger suppression or attentional capture contingent on a task-relevant feature's functional meaning during visual search, but top-down suppression operates at different levels of specificity depending on current task demands.

Long-term memory for distractors: Effects of involuntary attention from working memory

In a visual search task, attention to task-irrelevant distractors impedes search performance. However, is it maladaptive to future performance? Here, I showed that attended distractors in a visual search task were better remembered in long-term memory (LTM) in the subsequent surprise recognition task than non-attended distractors. In four experiments, participants performed a visual search task using real-world objects of a single color. They encoded color in working memory (WM) during the task; because each object had a different color, participants directed their attention to the WM-matching colored distractor. Then, in the surprise recognition task, participants were required to indicate whether an object had been shown in the earlier visual search task, regardless of its color. The results showed that attended distractors were remembered better in LTM than non-attended distractors (Experiments 1 and 2). Moreover, the more participants directed their attention to distractors, the better they explicitly remembered them. Participants did not explicitly remember the color of the attended distractors (Experiment 3) but remembered integrated information with object and color (Experiment 4). When the color of the distractors in the recognition task was mismatched with the color in the visual search task, LTM decreased compared to color-matching distractors. These results suggest that attention to distractors impairs search for a target but is helpful in remembering distractors in LTM. When task-irrelevant distractors become task-relevant information in the future, their attention becomes beneficial.

Investigating the role of spatial filtering on distractor suppression

In recent years, evidence has accumulated towards a distractor suppression mechanism that enables efficient selection of targets in a visual search task. According to these findings, the search for a target is faster in the presence of a salient distractor in a display among homogenous distractors as opposed to its absence. Studies have also shown that distractor suppression not only operates on the feature level but can also be spatially guided. The motivation of the current study was to examine if spatially guided distractor suppression can be goal-driven. We tested this across four experiments. In Experiment 1A, the task was to search for a shape target (e.g., a circle) and discriminate the orientation of the line within it. In some trials, a salient color distractor was presented in the display while participants were told that it appeared in one of the two locations on the horizontal axis (or the vertical axis, counterbalanced across participants). We expected enhanced distractor suppression when the salient distractor appeared within this "spatial filter" but did not find it since the target was also presented at the filtered locations. Experiment 1B replicated Experiment 1A, except that the target was always presented outside the filter; filtering enhanced search performance. In Experiment 2 even when the filter contained the salient distractor in only 65% of the filtered trials, filtering benefited search performance. In Experiment 3, the filter changed on every trial and did not benefit suppression.

The time course of category-based attentional template pre-activation depends on the category framework

When searching for a target defined by a set of objects, attention can be directed toward task-relevant objects by creating a category-based attentional template (CAT). Previous studies have found that CAT can be activated before the onset of the target. However, the time course of CAT pre-activation and whether the category framework (prototypical or semantic) can modulate it remain unclear. To explore the time course of CAT pre-activation, we employed a rapid serial probe presentation paradigm (RSPP) with event-related potentials (ERPs). To investigate the effect of the category framework on the time course of CAT pre-activation, the target category was defined as the prototypical category (Experiment 1) or the semantic category (Experiment 2). The results showed that the prototype-based CAT was pre-activated 300 ms prior to the target, whereas the semantics-based CAT was pre-activated 1500 ms before the onset of the target. The difference in the time course of pre-activation between the two CAT types indicates that the category framework can modulate the time course of CAT pre-activation. Additionally, during the attentional selection phase, an overall comparison of the target revealed that a larger N2pc was elicited by the prototype-based CAT than by the semantics-based CAT, suggesting that the prototype-based CAT could capture more attention than the semantics-based CAT. The findings on the difference between the two CAT frameworks in the preparatory and attentional selection phases provide more evidence for categorical information in visual search and extend our understanding of the mechanism of categorical attentional selection.

Graded prioritisation of targets in search: reward diminishes the low prevalence effect

In many real-life contexts, observers are required to search for targets that are rarely present (e.g. tumours in X-rays; dangerous items in airport security screenings). Despite the rarity of these items, they are of enormous importance for the health and safety of the public, yet they are easily missed during visual search. This is referred to as the prevalence effect. In the current series of experiments, we investigate whether unequal reward can modulate the prevalence effect, in a multiple target search task. Having first established the impact of prevalence (Experiment 1) and reward (Experiment 2) on how efficiently participants can find one of several targets in the current paradigm, we then combined the two forms of priority to investigate their interaction. An unequal reward distribution (where lower prevalence items are more rewarded; Experiment 3) was found to diminish the effect of prevalence, compared to an equal reward distribution (Experiment 4) as indicated by faster response times and fewer misses. These findings suggest that when combined with an unequal reward distribution, the low prevalence effect can be diminished.

The role of middle frontal gyrus in working memory retrieval by the effect of target detection tasks: a simultaneous EEG-fMRI study

Maintained working memory (WM) representations have been shown to influence visual target detection selection, while the effect of the visual target detection process on WM retrieval remains largely unknown. In the current research, we used the dual-paradigm of the visual target detection task and the delayed matching task (DMT), which contained the following four conditions: the match condition: the DMT target contained the detection target; the mismatch condition: the DMT target contained the detection distractor; the neutral condition: only the detection target was presented; the catch condition: only the DMT target was presented. Twenty-six subjects were recruited in the experiment with simultaneous EEG-fMRI data. Behaviorally, faster responses were found in the mismatch condition than in the match and neutral conditions. The EEG data found a greater parieto-occipital N1 component in the mismatch condition compared to the neutral condition, and a greater frontal N2 component in the match condition than in the mismatch condition. Moreover, compared to the match and neutral conditions, weaker activations of the bilateral middle frontal gyrus (MFG) were observed in the mismatch condition. And the representational similarity analysis (RSA) revealed significant differences in the representational patterns of the bilateral MFG between mismatch and match conditions, as well as in the representational patterns of the left MFG between mismatch and neutral conditions. Additionally, the left MFG may be the brain source of the N1 component in the mismatch condition. These findings suggest that the mismatch between the DMT target and detection target affects early attention allocation and attentional control in WM retrieval, and the MFG may play an important role in WM retrieval by the effect of the target detection task. In conclusion, our work deepens the understanding of the neural mechanisms by which visual target detection affects WM retrieval.

Directing memory content to attentional templates: The finiteness effect of predictive information

Visual search can be accelerated according to the properties of information stored in memory and prior knowledge of the upcoming work. This helps the searcher direct their attention to (or avoid) items that match these properties. Meanwhile, different functional areas where these properties exist become attentional templates. Compared with neutral conditions, the use of attentional templates significantly benefits reaction time (RT). However, previous studies might have confounded the memory-driven and cue-driven effects. Thus, it is less clear which factor influences the template benefits. Modeled on previous research, this study employed a new design to explore the independent effects of textual cues, thus finding an inverse effect. More specifically, positively cueing an item retained in memory did not improve behavioral performance, whereas negatively cueing an item did achieve such an enhancement. Moreover, positive cueing even resulted in some damage to attentional searching under some conditions, thus indicating that the advantages of positive cueing reported in previous studies may be driven by working memory, while the effects of negative cueing are driven by prior knowledge.

Sensory Delay Activity: More than an Electrophysiological Index of Working Memory Load

Sustained contralateral delay activity emerges in the retention period of working memory (WM) tasks and has been commonly interpreted as an electrophysiological index of the number of items held in a discrete-capacity WM resource. More recent findings indicate that these visual and tactile components are sensitive to various cognitive operations beyond the storage of discrete items in WM. In this Perspective, we present recent evidence from unisensory and multisensory visual and tactile WM tasks suggesting that, in addition to memory load, sensory delay activity may also be indicative of attentional and executive processes, as well as reflecting the flexible, rather than discrete, allocation of a continuous WM resource. Together, these findings challenge the traditional model of the functional significance of the contralateral delay activity as a pure measure of item load, and suggest that it may also reflect executive, attentional, and perceptual mechanisms operating in hierarchically organized WM systems.

Biased Competition between Targets and Distractors Reduces Attentional Suppression: Evidence from the Positivity Posterior Contralateral and Distractor Positivity

The biased competition account claims that competition between two stimuli increases when they are close together compared with when they are far apart. The reason is that nearby stimuli are more likely to be represented in the same receptive fields, requiring top-down or bottom-up biases to resolve the ambiguity. Consistent with biased competition, previous research showed that an index of attentional enhancement, the N2pc component, was attenuated when two targets were close together. In contrast, it is unclear whether distractor processing would also be attenuated when the distractor is close to the target. To answer this question, we used the additional singleton paradigm where a target is sometimes accompanied by a more salient, but entirely irrelevant, distractor. In the conditions of interest, the distance between the target and the distractor was systematically manipulated whereas the eccentricity to central fixation was always the same. The results showed that two indices of attentional suppression, the positivity posterior contralateral and distractor positivity components, were attenuated when the distractor was close to the target. Consistent with biased competition, attentional suppression of distractors was inhibited when the distance between target and distractor was short. The reduced attentional suppression of distractors with nearby targets may contribute to the increased behavioral interference with close distractors.

Exploring how healthcare teams balance the neurodynamics of autonomous and collaborative behaviors: a proof of concept

Team members co-regulate their activities and move together at the collective level of behavior while coordinating their actions toward shared goals. In parallel with team processes, team members need to resolve uncertainties arising from the changing task and environment. In this exploratory study we have measured the differential neurodynamics of seven two-person healthcare teams across time and brain regions during autonomous (taskwork) and collaborative (teamwork) segments of simulation training. The questions posed were: (1) whether these abstract and mostly integrated constructs could be separated neurodynamically; and, (2) what could be learned about taskwork and teamwork by trying to do so? The taskwork and teamwork frameworks used were Neurodynamic Information (NI), an electroencephalography (EEG) derived measure shown to be a neurodynamic proxy for the pauses and hesitations associated with individual uncertainty, and inter-brain EEG coherence (IBC) which is a required component of social interactions. No interdependency was observed between NI and IBC, and second-by-second dynamic comparisons suggested mutual exclusivity. These studies show that proxies for fundamental properties of teamwork and taskwork can be separated neurodynamically during team performances of ecologically valid tasks. The persistent expression of NI and IBC were not simultaneous suggesting that it may be difficult for team members to maintain inter-brain coherence while simultaneously reducing their individual uncertainties. Lastly, these separate dynamics occur over time frames of 15–30 s providing time for real-time detection and mitigation of individual and collaborative complications during training or live patient encounters.

Statistical learning in visual search reflects distractor rarity, not only attentional suppression

In visual search tasks, salient distractors may capture attention involuntarily, but interference can be reduced when the salient distractor appears more frequently on one out of several possible positions. The reduction was attributed to attentional suppression of the high-probability position. However, all previous studies on this topic compared performance on the high-probability position to the remaining positions, which had a low probability of containing the distractor. Therefore, it is not clear whether the difference resulted from reduced interference on the high-probability position or from increased interference on the low-probability positions. To decide between these alternatives, we compared high-probability and low-probability with equal-probability positions. Consistent with attentional suppression, interference was reduced on the high-probability position compared with equal-probability positions. However, there was also an increase in interference on low-probability positions compared with equal-probability positions. The increase is in line with previous reports of boosted interference when distractors are rare. Our results show that the experimental design used in previous research is insufficient to separate effects of attentional suppression and those of distractor rarity.

Capacity limitations in template-guided multiple color search

Visual selection of target objects relies on representations of their known features in visual working memory. These representations are referred to as attentional templates. We asked how the capacity of visual working memory relates to the maximal number of attentional templates that can simultaneously guide visual selection. To measure the number of active attentional templates, we used the contingent capture paradigm where cues matching the attentional template have larger effects than cues in a non-matching color. We found larger cueing effects for matching than non-matching cues in one-, two-, and also three-color searches, suggesting that participants can establish up to three attentional templates. However, scrutiny of matching cue trials showed that with three attentional templates, larger cueing effects only occurred when the matching cue had the same color as the actual target. When the matching cue had a possible target color that was different from the actual target color, cueing effects were similar to non-matching cue colors. We assume that processing of a matching cue activates one of the three templates, which inhibits the remaining templates to the level of non-matching colors. With two colors, the inhibition from the activated template is less complete because the initial template activation is higher. Overall, only a maximum of two attentional templates can operate successfully in the contingent capture paradigm. The capacity of template-guided search is therefore far below the capacity of visual working memory.

Attentional templates are protected from retroactive interference during visual search: Converging evidence from event-related potentials

Attentional templates are stored representations of target features that guide visual search. Target features may remain fixed or change on every trial, requiring sustained or transient templates, respectively. In separate blocks of trials, two sustained templates guide visual search as efficiently as two transient templates. In mixed blocks, however, the transient template interferes with the sustained template, impairing its efficiency in guiding visual search. Here, we hypothesized that the priority of the sustained template would increase when threatened by interference, eventually restoring efficient guidance of visual search. Participants memorized two possible target colors before the onset of the search display. At encoding, we assessed attentional selection of the two possible target colors with the N2pc. During subsequent maintenance, we measured the CDA as an index of resource allocation in working memory. In Experiment 1, the CDA was smaller with sustained than transient templates in separate blocks, but similar in mixed blocks. Thus, the sustained template received more working memory resources when maintained concurrently with an interfering transient template, suggesting that it was prioritized. In Experiment 2, the priority of the sustained template was further increased as it guided visual search in 80% of cases. The N2pc to possible target colors matching the sustained template was enhanced both at encoding and during visual search, thus eliminating interference from the transient template. Therefore, sustained templates are not necessarily less efficient than transient templates. Rather, prioritization through attentional selection at encoding and resource allocation during maintenance may restore efficient guidance of visual search.

Preparatory Control Against Distraction Is Not Feature-Based

Salient-but-irrelevant stimuli (distractors) co-occurring with search targets can capture attention against the observer's will. Recently, evidence has accumulated that preparatory control can prevent this misguidance of spatial attention in predictable situations. However, the underlying mechanisms have remained elusive. Most pertinent theories assume that attention is guided by specific features. This widespread theoretical claim provides several strong predictions with regard to distractor handling that are disconfirmed here: Employing electrophysiological markers of covert attentional dynamics, in three experiments, we show that distractors standing out by a feature that is categorically different from the target consistently captures attention. However, equally salient distractors standing out in a different feature dimension are effectively down-weighted, even if unpredictably swapping their defining feature with the target. This shows that preparing for a distractor's feature is neither necessary nor sufficient for successful avoidance of attentional capture. Rather, capture is prevented by preparing for the distractor's feature dimension.