Delineating resetting and updating in visual working memory based on the object-to-representation correspondence

The effect of context on pointer allocation in visual working memory

Visual working memory (VWM) can hold a limited amount of visual information and manipulate it. It encodes this information and forms representations of each one of the relevant objects. When an object changes, VWM can either update or reset its representation to account for this change. To access a specific representation VWM relies on a pointer system associating each representation with the corresponding object in the environment. While previous studies described these processes as reacting to a change in the object status, this study investigated the adaptability of the pointer system to the task context. We measured the contralateral delay activity (CDA; an electrophysiological marker of VWM) as a marker of updating and resetting. In two experiments we used a shape change detection task (similar to Balaban & Luria, 2017) and manipulated the proportion of the resetting and updating trials to create different task contexts. Experiment 1 indicated that VWM can adapt to a resetting mode in which it performs resetting in conditions that triggered updating in previous studies. However, Experiment 2 revealed that the pointer system cannot adapt to an updating mode and perform updating in conditions that trigger resetting. These results suggest that VWM can strategically perform resetting, but once a pointer is lost, it's impossible to update the representation and a resetting process is mandatory triggered regardless of the context.

Dissociable online integration processes in visual working memory

Visual working memory has severe capacity limits, creating a bottleneck for active processing. A key way of mitigating this limitation is by chunking, i.e. compressing several pieces of information into one visual working memory representation. However, despite decades of research, chunking efficiency remains debated because of mixed evidence. We propose that there are actually 2 integration mechanisms: Grouping combines several objects to one representation, and object-unification merges the parts of a single object. Critically, we argue that the fundamental distinction between the 2 processes is their differential use of the pointer system, the indexing process connecting visual working memory representations with perception. In grouping, the objects that are represented together still maintain independent pointers, making integration costly but highly flexible. Conversely, object-unification fuses the pointers as well as the representations, with the single pointer producing highly efficient integration but blocking direct access to individual parts. We manipulated integration cues via task-irrelevant movement, and monitored visual working memory's online electrophysiological marker. Uniquely colored objects were flexibly grouped and ungrouped via independent pointers (experiment 1). If objects turned uniformly black, object-integration could not be undone (experiment 2), requiring visual working memory to reset before re-individuation. This demonstrates 2 integration levels (representational-merging versus pointer-compression) and establishes the dissociation between visual working memory representations and their underlying pointers.

One turn at a time: Behavioral and ERP evidence for two types of rotations in the classical mental rotation task

We perform mental rotations in many everyday situations, such as reading a map or following furniture assembling instructions. In a classical mental rotation task, participants are asked to judge whether a rotated stimulus is presented in its mirrored form or its canonical form. Previous results have indicated a degree effect: RT is longer as the angle of rotation increases, and this effect is traditionally explained by arguing that this judgment requires rotating the stimulus back to its upright form. Importantly, in half of the trials, the stimuli are rotated on both the page plane and mirror plane. Namely, we argue that in previous research the task actually involved two different rotation processes. To provide a clear dissociation between these two rotations, we collected EEG data and used the Contralateral Delay Activity (CDA) as an indicator of visual working memory (VWM) load. The results of Experiment 1 suggested different VWM involvement according to the degrees rotations when the item was not mirrored, such that the CDA amplitude generally increased as the degree of rotation was higher. Mirrored trials were all at ceiling in terms of CDA, regardless of their rotation degree. Experiment 2 showed increased CDA amplitude uniquely related to the flip rotation. Thus, we provided ERP evidence that the canonical mental rotation task involves two types of rotations that can be dissociated based on the load they imposed on VWM.

A relational account of visual short-term memory (VSTM)

Visual short-term memory (VSTM) is an important resource that allows temporarily storing visual information. Current theories posit that elementary features (e.g., red, green) are encoded and stored independently of each other in VSTM. However, they have difficulty explaining the similarity effect, that similar items can be remembered better than dissimilar items. In Experiment 1, we tested (N = 20) whether the similarity effect may be due to storing items in a context-dependent manner in VSTM (e.g., as the reddest/yellowest item). In line with a relational account of VSTM, we found that the similarity effect is not due to feature similarity, but to an enhanced sensitivity for detecting changes when the relative colour of a to-be-memorised item changes (e.g., from reddest to not-reddest item; than when an item underwent the same change but retained its relative colour; e.g., still reddest). Experiment 2 (N = 20) showed that VSTM load, as indexed by the CDA amplitude in the EEG, was smaller when the colours were ordered so that they all had the same relationship than when the same colours were out-of-order, requiring encoding different relative colours. With this, we report two new effects in VSTM - a relational detection advantage that describes an enhanced sensitivity to relative changes in change detection, and a relational CDA effect, which reflects that VSTM load, as indexed by the CDA, scales with the number of (different) relative features between the memory items. These findings support a relational account of VSTM and question the view that VSTM stores features such as colours independently of each other.

Neural evidence for an object-based pointer system underlying working memory

To accomplish even rudimentary tasks, our cognitive system must update its representation of the changing environment. This process relies on visual working memory (VWM), which can actively modify its representations. We argue that this ability depends on a pointer system, such that each representation is stably and uniquely mapped to a specific stimulus. Without these pointers, VWM representations are inaccessible and therefore unusable. In three Electroencephalogram (EEG) experiments, we examined whether the pointers are allocated in an object-based, featural, or spatial manner: three factors that were confounded in previous studies. We used a feature change-detection task, in which objects moved and could separate into independently-moving parts. Despite the movement and separation being completely task-irrelevant, we found that the separation invalidated the pointers. This happened in a shape task, where the separation changed both the objects and the task-relevant features, but importantly, also in a color task, where the separation destroyed the objects while leaving the task-relevant features intact. Furthermore, even in a color task where all items had identical shapes, object-separation invalidated the pointers. This suggests that objects and not task-relevant features underlie the pointer system. Finally, when each object-part could be individuated already before the separation, the pointers were maintained, suggesting that the pointers are specifically tied to objects rather than locations. These results shed new light on the pointers which underlie VWM performance, demonstrating that the pointer system is object-based regardless of the task requirements.

For whom is social-network usage associated with anxiety? The moderating role of neural working-memory filtering of Facebook information

Is Facebook usage bad for mental health? Existing studies provide mixed results, and direct evidence for neural underlying moderators is lacking. We suggest that being able to filter social-network information from accessing working memory is essential to preserve limited cognitive resources to pursue relevant goals. Accordingly, among individuals with impaired neural social-network filtering ability, enhanced social-network usage would be associated with negative mental health. Specifically, participants performed a novel electrophysiological paradigm that isolates neural Facebook filtering ability. Participants' actual Facebook behavior and anxious symptomatology were assessed. Confirming evidence showed that enhanced Facebook usage was associated with anxious symptoms among individuals with impaired neural Facebook filtering ability. Although less robust and tentative, additional suggestive evidence indicated that this specific Facebook filtering impairment was not better explained by a general filtering deficit. These results involving a neural social-network filtering moderator, may help understand for whom increased online social-network usage is associated with negative mental health.

What happens to an individual visual working memory representation when it is interrupted?

This study tested the hypothesis that even the simplest cognitive tasks require the storage of information in working memory (WM), distorting any information that was previously stored in WM. Experiment 1 tested this hypothesis by requiring observers to perform a simple letter discrimination task while they were holding a single orientation in WM. We predicted that performing the task on the interposed letter stimulus would cause the orientation memory to become less precise and more categorical compared to when the letter was absent or when it was present but could be ignored. This prediction was confirmed. Experiment 2 tested the modality specificity of this effect by replacing the visual letter discrimination task with an auditory pitch discrimination task. Unlike the interposed visual stimulus, the interposed auditory stimulus produced little or no disruption of WM, consistent with the use of modality-specific representations. Thus, performing a simple visual discrimination task, but not a simple auditory discrimination task, distorts information about a single feature being maintained in visual WM. We suggest that the interposed task eliminates information stored within the focus of attention, leaving behind a WM representation outside the focus of attention that is relatively imprecise and categorical.