Building blocks of visual working memory: objects or Boolean maps?

The Basic Units of Working Memory Manipulation Are Boolean Maps, Not Objects

Determining the manipulation unit of working memory is one of the fundamental questions in understanding how working memory functions. The prevalent object-based theory in cognitive research predicts that memory manipulation is performed on the level of objects. Here we show instead that the basic units of working memory manipulation are Boolean maps, a data structure describing what can be perceived in an instant. We developed four new manipulation tasks (with data from 80 adults) and showed that manipulation times only increased when the number of Boolean maps manipulated increased. Increasing the number of orientations manipulated did not induce longer manipulation times, consistent with a key prediction of the Boolean map theory. Our results show that Boolean maps are the manipulation unit of working memory.

The transition from feature to object: Storage unit in visual working memory depends on task difficulty

Visual working memory (VWM) is a cognitive memory buffer for temporarily processing and storing visual information. Previous studies suggest that its capacity is severely limited, and there is an ongoing debate on whether the storage capacity is object-based or feature-based in VWM. In this study, a change-detection task was employed to investigate whether and how task difficulty can affect VWM, specifically, its capacity and the unit of storage. Task difficulty was manipulated through the set size of memory items, memory fidelity required by the resolution of representation and the type of feature tested. We examined two types of stimuli: the single-feature type, where each memory item was composed of a single feature (color or shape), and the conjunctive-feature type, where each item was composed of a conjunction of two features (colored shape). Experiment 1 replicated the previous findings that memory capacity for colors was larger than shapes, and decreased with the resolution demand regardless of the type of stimuli. In Experiment 2, we analyzed and compared the results from single-feature items and conjunctive-feature items in the low- and high-resolution conditions while controlling for the number of to-be-remembered features. By directly matching the estimated capacity based on an object-unit and a feature-unit with the theoretical prediction, the results showed that the unit storage in VWM tended to be feature-based with low task difficulty, and to be object-based with high task difficulty. This suggests that VWM is dynamic and flexible, dependent on the load of the current task.

The impact of visual working memory capacity on the filtering efficiency of emotional face distractors

Emotional faces can serve as distractors for visual working memory (VWM) tasks. An event-related potential called contralateral delay activity (CDA) can measure the filtering efficiency of face distractors. Previous studies have investigated the influence of VWM capacity on filtering efficiency of simple neutral distractors but not of face distractors. We measured the CDA indicative of emotional face filtering during a VWM task related to facial identity. VWM capacity was measured in a separate colour change detection task, and participants were divided to high- and low-capacity groups. The high-capacity group was able to filter out distractors similarly irrespective of its facial emotion. In contrast, the low-capacity group failed in filtering the neutral and angry face distractors, while the filtering was efficient for the happy face distractors. The results indicate that potentially threatening faces are particularly difficult to filter if VWM capacity is limited.

Gestalt grouping cues can improve filtering performance in visual working memory

As part of filtering irrelevant information from entering visual working memory (VWM) and selecting only the relevant information for further processing the system should first tag the pieces of information as relevant or irrelevant. We manipulated difficulty of tagging items as relevant or irrelevant by applying perceptual grouping cues to investigate if it can improve filtering performance in VWM. Participants performed a change-detection task with three targets, six targets, or three targets and three distractors (filtering condition) in the memory display, and were asked to remember the colors (Experiments 1-2) or the orientations (Experiments 3-5) of the targets and ignore the distractors. In the filtering conditions, either the targets (Experiments 1, 3, and 4) or the distractors (Experiments 2 and 5) formed an illusory object (a Kanizsa triangle), appeared in a triangle-like configuration (grouping by proximity), or appeared at random positions (non-grouping). Grouping the targets improved filtering performance relative to non-grouping. Moreover, the illusory object cue further improved filtering performance beyond a proximity cue, but only when the cue was compatible with the task. When the distractors were grouped, the proximity cue improved filtering performance, and the illusory object cue, despite being a potent grouping cue, failed to improve filtering performance when it was compatible with the task. We suggest that the grouping cues advanced tagging of the grouped items. Yet, when the grouping cue strongly enhanced processing of the distractors, the tagging failed, such that the preliminary process of estimating incoming items led to full processing of the grouped items.

Contralateral delay activity tracks the influence of Gestalt grouping principles on active visual working memory representations

Recent studies have demonstrated that factors influencing perception, such as Gestalt grouping cues, can influence the storage of information in visual working memory (VWM). In some cases, stationary cues, such as stimulus similarity, lead to superior VWM performance. However, the neural correlates underlying these benefits to VWM performance remain unclear. One neural index, the contralateral delay activity (CDA), is an event-related potential that shows increased amplitude according to the number of items held in VWM and asymptotes at an individual's VWM capacity limit. Here, we applied the CDA to determine whether previously reported behavioral benefits supplied by similarity, proximity, and uniform connectedness were reflected as a neural savings such that the CDA amplitude was reduced when these cues were present. We implemented VWM change-detection tasks with arrays including similarity and proximity (Experiment 1); uniform connectedness (Experiments 2a and 2b); and similarity/proximity and uniform connectedness (Experiment 3). The results indicated that when there was a behavioral benefit to VWM, this was echoed by a reduction in CDA amplitude, which suggests more efficient processing. However, not all perceptual grouping cues provided a VWM benefit in the same measure (e.g., accuracy) or of the same magnitude. We also found unexpected interactions between cues. We observed a mixed bag of effects, suggesting that these powerful perceptual grouping benefits are not as predictable in VWM. The current findings indicate that when grouping cues produce behavioral benefits, there is a parallel reduction in the neural resources required to maintain grouped items within VWM.

The effects of direction similarity in visual working memory: Behavioural and event-related potential studies

Object similarity can improve visual working memory (VWM) performance in the change-detection task, but impair the recognition performance when it occurs at retrieval of VWM in the recognition task. The effect of direction similarity is an issue that has not been well resolved. Furthermore, electrophysiological evidence in support of the mechanisms that underlie the effects of similarity is still scarce. In the current study, we conducted three behavioural experiments to examine the effects of direction similarity on memory performance with regard to both the encoding and retrieval phases of VWM and one event-related potential (ERP) experiment to explore the neural signatures of direction similarity in VWM. Our behavioural studies indicated that direction similarity improved performance when it occurred at the encoding phase but impaired performance when it occurred at the retrieval phase. Moreover, the ERP experiment showed that the amplitude of the contralateral delay activity (CDA) increased with the increasing set size for similar but not dissimilar directions. In addition, the CDA amplitude for similar directions was lower than that for dissimilar directions at set size 2. Taken together, these findings suggest that direction similarity at encoding has a positive effect on VWM performance and at retrieval has a negative effect. Given that VWM capacity depends on information load and the number of objects, the positive effect of similarity may be attributed to reduced information load of memory objects.

Organization principles in visual working memory: Evidence from sequential stimulus display

Although the mechanisms of visual working memory (VWM) have been studied extensively in recent years, the active property of VWM has received less attention. In the current study, we examined how VWM integrates sequentially presented stimuli by focusing on the role of Gestalt principles, which are important organizing principles in perceptual integration. We manipulated the level of Gestalt cues among three or four sequentially presented objects that were memorized. The Gestalt principle could not emerge unless all the objects appeared together. We distinguished two hypotheses: a perception-alike hypothesis and an encoding-specificity hypothesis. The former predicts that the Gestalt cue will play a role in information integration within VWM; the latter predicts that the Gestalt cue will not operate within VWM. In four experiments, we demonstrated that collinearity (Experiment 1) and closure (Experiment 2) cues significantly improved VWM performance, and this facilitation was not affected by the testing manner (Experiment 3) or by adding extra colors to the memorized objects (Experiment 4). Finally, we re-established the Gestalt cue benefit with similarity cues (Experiment 5). These findings together suggest that VWM realizes and uses potential Gestalt principles within the stored representations, supporting a perception-alike hypothesis.

The role of spatial configuration in multiple identity tracking

Background

The simultaneous tracking and identification of multiple moving objects encountered in everyday life requires one to correctly bind identities to objects. In the present study, we investigated the role of spatial configuration made by multiple targets when observers are asked to track multiple moving objects with distinct identities.

Methodology/Principal Findings

The overall spatial configuration made by the targets was manipulated: In the constant condition, the configuration remained as a virtual convex polygon throughout the tracking, and in the collapsed condition, one of the moving targets (critical target) crossed over an edge of the virtual polygon during tracking, destroying it. Identification performance was higher when the configuration remained intact than when it collapsed (Experiments 1a, 1b, and 2). Moreover, destroying the configuration affected the allocation of dynamic attention: the critical target captured more attention than did the other targets. However, observers were worse at identifying the critical target and were more likely to confuse it with the targets that formed the virtual crossed edge (Experiments 3–5). Experiment 6 further showed that the visual system constructs an overall configuration only by using the targets (and not the distractors); identification performance was not affected by whether the distractor violated the spatial configuration.

Conclusions/Significance

In sum, these results suggest that the visual system may integrate targets (but not distractors) into a spatial configuration during multiple identity tracking, which affects the distribution of dynamic attention and the updating of identity-location binding.

Coarse-to-fine construction for high-resolution representation in visual working memory

Background

This study explored whether the high-resolution representations created by visual working memory (VWM) are constructed in a coarse-to-fine or all-or-none manner. The coarse-to-fine hypothesis suggests that coarse information precedes detailed information in entering VWM and that its resolution increases along with the processing time of the memory array, whereas the all-or-none hypothesis claims that either both enter into VWM simultaneously, or neither does.

Methodology/Principal Findings

We tested the two hypotheses by asking participants to remember two or four complex objects. An ERP component, contralateral delay activity (CDA), was used as the neural marker. CDA is higher for four objects than for two objects when coarse information is primarily extracted; yet, this CDA difference vanishes when detailed information is encoded. Experiment 1 manipulated the comparison difficulty of the task under a 500-ms exposure time to determine a condition in which the detailed information was maintained. No CDA difference was found between two and four objects, even in an easy-comparison condition. Thus, Experiment 2 manipulated the memory array’s exposure time under the easy-comparison condition and found a significant CDA difference at 100 ms while replicating Experiment 1′s results at 500 ms. In Experiment 3, the 500-ms memory array was blurred to block the detailed information; this manipulation reestablished a significant CDA difference.

Conclusions/Significance

These findings suggest that the creation of high-resolution representations in VWM is a coarse-to-fine process.