Perturbing Neural Representations of Working Memory with Task-irrelevant Interruption

Dimension-based retro-cue benefit in working memory does not require unfocused dimension removal

Introduction

Within the maintenance phase of visual working memory (VWM), previous researchers presented retro-cues orienting to a probed dimension across all multidimension stimuli and found a robust dimension-based retro-cue benefit (RCB): VWM performance for cued dimension was better than no/neutral-cue baseline. This improvement is often attributed to the prioritization of information related to the focused dimension and the removal of information related to the unfocused dimension from VWM. However, it remains unclear whether the removal of the uncued dimension is necessary to observe this dimension-based RCB.

Methods

In the current study, we first manipulated the number of retro-cues to investigate this question. We used colored, oriented bars as stimuli and two sequential retro-cues oriented to different dimensions in the double-cue condition. The last presented cue in each trial was always valid. Therefore, the unfocused dimension in the first cue display was probed in double-cue trials. Experiment 1 adopted change detection tasks and three cue type conditions (no-cue, single-cue, double-cue). Experiment 2 divided the single-cue condition into early- and late- cue conditions, using recall tasks to elevated probe precision. Experiment 3 further added double-neutral and double-same cue types and eliminated the different influences of post-memory masks on each dimension respectively.

Results

Results across these experiments showed a robust pattern of no worse performances for the double-cue condition than for the single-cue condition.

Discussion

Because the dimension-based single cue benefit was observed especially in early-cue trials, we supposed that the dimension-based RCB does not require removing the unfocused dimension from VWM.

Surprising sounds influence risky decision making

Adaptive behavior depends on appropriate responses to environmental uncertainty. Incidental sensory events might simply be distracting and increase errors, but alternatively can lead to stereotyped responses despite their irrelevance. To evaluate these possibilities, we test whether task-irrelevant sensory prediction errors influence risky decision making in humans across seven experiments (total n = 1600). Rare auditory sequences preceding option presentation systematically increase risk taking and decrease choice perseveration (i.e., increased tendency to switch away from previously chosen options). The risk-taking and perseveration effects are dissociable by manipulating auditory statistics: when rare sequences end on standard tones, including when rare sequences consist only of standard tones, participants are less likely to perseverate after rare sequences but not more likely to take risks. Computational modeling reveals that these effects cannot be explained by increased decision noise but can be explained by value-independent risky bias and perseveration parameters, decision biases previously linked to dopamine. Control experiments demonstrate that both surprise effects can be eliminated when tone sequences are presented in a balanced or fully predictable manner, and that surprise effects cannot be explained by erroneous beliefs. These findings suggest that incidental sounds may influence many of the decisions we make in daily life.

The differential impact of face distractors on visual working memory across encoding and delay stages

External distractions often occur when information must be retained in visual working memory (VWM)-a crucial element in cognitive processing and everyday activities. However, the distraction effects can differ if they occur during the encoding rather than the delay stages. Previous research on these effects used simple stimuli (e.g., color and orientation) rather than considering distractions caused by real-world stimuli on VWM. In the present study, participants performed a facial VWM task under different distraction conditions across the encoding and delay stages to elucidate the mechanisms of distraction resistance in the context of complex real-world stimuli. VWM performance was significantly impaired by delay-stage but not encoding-stage distractors (Experiment 1). In addition, the delay distraction effect arose primarily due to the absence of distractor process at the encoding stage rather than the presence of a distractor during the delay stage (Experiment 2). Finally, the impairment in the delay-distraction condition was not due to the abrupt appearance of distractors (Experiment 3). Taken together, these findings indicate that the processing mechanisms previously established for resisting distractions in VWM using simple stimuli can be extended to more complex real-world stimuli, such as faces.

Multifaceted consequences of visual distraction during natural behaviour

Visual distraction is a ubiquitous aspect of everyday life. Studying the consequences of distraction during temporally extended tasks, however, is not tractable with traditional methods. Here we developed a virtual reality approach that segments complex behaviour into cognitive subcomponents, including encoding, visual search, working memory usage, and decision-making. Participants copied a model display by selecting objects from a resource pool and placing them into a workspace. By manipulating the distractibility of objects in the resource pool, we discovered interfering effects of distraction across the different cognitive subcomponents. We successfully traced the consequences of distraction all the way from overall task performance to the decision-making processes that gate memory usage. Distraction slowed down behaviour and increased costly body movements. Critically, distraction increased encoding demands, slowed visual search, and decreased reliance on working memory. Our findings illustrate that the effects of visual distraction during natural behaviour can be rather focal but nevertheless have cascading consequences.

The reactivation of task rules triggers the reactivation of task-relevant items

Working memory (WM) describes the temporary storage of task-relevant items and procedural rules to guide action. Despite its central importance for goal-directed behavior, the interplay between WM and long-term memory (LTM) remains poorly understood. Recent studies have shown that repeated use of the same task-relevant item in WM results in a hand-off of the storage of that item to LTM, and switching to a new item reactivates WM. To further elucidate the rules governing WM-LTM interactions, we here planned to probe whether a change in task rules, independent of a switch in task-relevant items, would also lead to WM reactivation of maintained items. To this end, we used scalp-recorded electroencephalogram (EEG) data, specifically the contralateral delay activity (CDA), to track WM item storage while manipulating repetitions and changes in task rules and task-relevant items across trials in a visual WM task. We tested two rival hypotheses: If changes in task rules result in a reactivation of the target item representation, then the CDA should increase when a task change is cued even when the same target has been repeated across trials. However, if the reactivation of a task-relevant item only depends on the mnemonic availability of the item itself instead of the task it is used for, then only the changes in task-relevant items should reactivate the representations. Accordingly, the CDA amplitude should decrease for repeated task-relevant items independently of a task change. We found a larger CDA on task-switch compared to task-repeat trials, suggesting that the reactivation of task rules triggers the reactivation of task-relevant items in WM. By demonstrating that WM reactivation of LTM is interdependent for task rules and task-relevant items, this study informs our understanding of visual WM and its interplay with LTM. PREREGISTERED STAGE 1 PROTOCOL: https://osf.io/zp9e8 (date of in-principle acceptance: 19/12/2021).

Microsaccades Track Location-Based Object Rehearsal in Visual Working Memory

Besides controlling eye movements, the brain's oculomotor system has been implicated in the control of covert spatial attention and the rehearsal of spatial information in working memory. We investigated whether the oculomotor system also contributes to rehearsing visual objects in working memory when object location is never asked about. To address this, we tracked the incidental use of locations for mnemonic rehearsal via directional biases in microsaccades while participants maintained two visual objects (colored oriented gratings) in working memory. By varying the stimulus configuration (horizontal, diagonal, and vertical) at encoding, we could quantify whether microsaccades were more aligned with the configurational axis of the memory contents, as opposed to the orthogonal axis. Experiment 1 revealed that microsaccades continued to be biased along the axis of the memory content several seconds into the working memory delay. In Experiment 2, we confirmed that this directional microsaccade bias was specific to memory demands, ruling out lingering effects from passive and attentive encoding of the same visual objects in the same configurations. Thus, by studying microsaccade directions, we uncover oculomotor-driven rehearsal of visual objects in working memory through their associated locations.

Prioritization sharpens working memories but does not protect them from distraction

Perceptual distraction distorts visual working memory representations. Previous research has shown that memory responses are systematically biased toward visual distractors that are similar to the memoranda. However, it remains unclear whether the prioritization of one working memory representation over another reduces the impact of perceptual distractors. In five behavioral experiments, we used different forms of retrospective cues (indicating the likelihood of testing each item and/or the reward for responding correctly to each item) to manipulate the prioritization of items in working memory before visual distraction. We examined the effects of distraction with nonparametric analyses and a novel distractor intrusion model. We found that memory responses were more precise (lower absolute response errors and stronger memory signals) for items that were prioritized. However, these prioritized items were not immune to distraction, and their memory responses were biased toward the visual distractors to the same degree as were unprioritized items. Our findings demonstrate that the benefits associated with prioritization in working memory do not include protection from distraction biases. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Contralateral delay activity, but not alpha lateralization, indexes prioritization of information for working memory storage

Working memory is inherently limited, which makes it important to select and maintain only task-relevant information and to protect it from distraction. Previous research has suggested the contralateral delay activity (CDA) and lateralized alpha oscillations as neural candidates for such a prioritization process. While most of this work focused on distraction during encoding, we examined the effect of external distraction presented during memory maintenance. Participants memorized the orientations of three lateralized objects. After an initial distraction-free maintenance interval, distractors appeared in the same location as the targets or in the opposite hemifield. This distraction was followed by another distraction-free interval. Our results show that CDA amplitudes were stronger in the interval before compared with the interval after the distraction (i.e., CDA amplitudes were stronger in response to targets compared with distractors). This amplitude reduction in response to distractors was more pronounced in participants with higher memory accuracy, indicating prioritization and maintenance of relevant over irrelevant information. In contrast, alpha lateralization did not change from the interval before distraction compared with the interval after distraction, and we found no correlation between alpha lateralization and memory accuracy. These results suggest that alpha lateralization plays no direct role in either selective maintenance of task-relevant information or inhibition of distractors. Instead, alpha lateralization reflects the current allocation of spatial attention to the most salient information regardless of task-relevance. In contrast, CDA indicates flexible allocation of working memory resources depending on task-relevance.

Is the Contralateral Delay Activity (CDA) a robust neural correlate for Visual Working Memory (VWM) tasks? A reproducibility study

Visual working memory (VWM) allows us to actively store, update, and manipulate visual information surrounding us. While the underlying neural mechanisms of VWM remain unclear, contralateral delay activity (CDA), a sustained negativity over the hemisphere contralateral to the positions of visual items to be remembered, is often used to study VWM. To investigate if the CDA is a robust neural correlate for VWM tasks, we reproduced eight CDA-related studies with a publicly accessible EEG data set. We used the raw EEG data from these eight studies and analyzed all of them with the same basic pipeline to extract CDA. We were able to reproduce the results from all the studies and show that with a basic automated EEG pipeline we can extract a clear CDA signal. We share insights from the trends observed across the studies and raise some questions about the CDA decay and the CDA during the recall phase, which surprisingly, none of the eight studies did address. Finally, we also provide reproducibility recommendations based on our experience and challenges in reproducing these studies.

Visual-cortical enhancement by acoustic distractors: The effects of endogenous spatial attention and visual working memory load

Past work has shown that when a peripheral sound captures our attention, it activates the contralateral visual cortex as revealed by an event-related potential component labelled the auditory-evoked contralateral occipital positivity (ACOP). This cross-modal activation of the visual cortex has been observed even when the sounds were not relevant to the ongoing task (visual or auditory), suggesting that peripheral sounds automatically activate the visual cortex. However, it is unclear whether top-down factors such as visual working memory (VWM) load and endogenous attention, which modulate the impact of task-irrelevant information, may modulate this spatially-specific component. Here, we asked participants to perform a lateralized VWM task (change detection), whose performance is supported by both endogenous spatial attention and VWM storage. A peripheral sound that was unrelated to the ongoing task was delivered during the retention interval. The amplitude of sound-elicited ACOP was analyzed as a function of the spatial correspondence with the cued hemifield, and of the memory array set-size. The typical ACOP modulation was observed over parieto-occipital sites in the 280-500 ms time window after sound onset. Its amplitude was not affected by VWM load but was modulated when the location of the sound did not correspond to the hemifield (right or left) that was cued for the change detection task. Our results suggest that sound-elicited activation of visual cortices, as reflected in the ACOP modulation, is unaffected by visual working memory load. However, endogenous spatial attention affects the ACOP, challenging the hypothesis that it reflects an automatic process.

Alterations in working memory maintenance of fearful face distractors in depressed participants: An ERP study

Task-irrelevant threatening faces (e.g., fearful) are difficult to filter from visual working memory (VWM), but the difficulty in filtering non-threatening negative faces (e.g., sad) is not known. Depressive symptoms could also potentially affect the ability to filter different emotional faces. We tested the filtering of task-irrelevant sad and fearful faces by depressed and control participants performing a color-change detection task. The VWM storage of distractors was indicated by contralateral delay activity, a specific event-related potential index for the number of objects stored in VWM during the maintenance phase. The control group did not store sad face distractors, but they automatically stored fearful face distractors, suggesting that threatening faces are specifically difficult to filter from VWM in non-depressed individuals. By contrast, depressed participants showed no additional consumption of VWM resources for either the distractor condition or the non-distractor condition, possibly suggesting that neither fearful nor sad face distractors were maintained in VWM. Our control group results confirm previous findings of a threat-related filtering difficulty in the normal population while also suggesting that task-irrelevant non-threatening negative faces do not automatically load into VWM. The novel finding of the lack of negative distractors within VWM storage in participants with depressive symptoms may reflect a decreased overall responsiveness to negative facial stimuli. Future studies should investigate the mechanisms underlying distractor filtering in depressed populations.

Long-term memory and working memory compete and cooperate to guide attention

Multiple types of memory guide attention: Both long-term memory (LTM) and working memory (WM) effectively guide visual search. Furthermore, both types of memories can capture attention automatically, even when detrimental to performance. It is less clear, however, how LTM and WM cooperate or compete to guide attention in the same task. In a series of behavioral experiments, we show that LTM and WM reliably cooperate to guide attention: Visual search is faster when both memories cue attention to the same spatial location (relative to when only one memory can guide attention). LTM and WM competed to guide attention in more limited circumstances: Competition only occurred when these memories were in different dimensions - particularly when participants searched for a shape and held an accessory color in mind. Finally, we found no evidence for asymmetry in either cooperation or competition: There was no evidence that WM helped (or hindered) LTM-guided search more than the other way around. This lack of asymmetry was found despite differences in LTM-guided and WM-guided search overall, and differences in how two LTMs and two WMs compete or cooperate with each other to guide attention. This work suggests that, even if only one memory is currently task-relevant, WM and LTM can cooperate to guide attention; they can also compete when distracting features are salient enough. This work elucidates interactions between WM and LTM during attentional guidance, adding to the literature on costs and benefits to attention from multiple active memories.

Cat-astrophic effects of sudden interruptions on spatial auditory attention

Salient interruptions draw attention involuntarily. Here, we explored whether this effect depends on the spatial and temporal relationships between a target stream and interrupter. In a series of online experiments, listeners focused spatial attention on a target stream of spoken syllables in the presence of an otherwise identical distractor stream from the opposite hemifield. On some random trials, an interrupter (a cat "MEOW") occurred. Experiment 1 established that the interrupter, which occurred randomly in 25% of the trials in the hemifield opposite the target, degraded target recall. Moreover, a majority of participants exhibited this degradation for the first target syllable, which finished before the interrupter began. Experiment 2 showed that the effect of an interrupter was similar whether it occurred in the opposite or the same hemifield as the target. Experiment 3 found that the interrupter degraded performance slightly if it occurred before the target stream began but had no effect if it began after the target stream ended. Experiment 4 showed decreased interruption effects when the interruption frequency increased (50% of the trials). These results demonstrate that a salient interrupter disrupts recall of a target stream, regardless of its direction, especially if it occurs during a target stream.

Neural correlates of a load-dependent decline in visual working memory

Previous studies have shown that a rate of temporal decline in visual working memory (vWM) highly depends on a number of memory items. When people retain the information of many (≥ 4) stimuli simultaneously, their memory representations are fragile and rapidly degrade within 2–3 seconds after an offset (called the “competition” among memory items). When a memory load is low (1 or 2 items), in contrast, the fidelity of vWM is preserved for a longer time because focused attention to the small number of items prevents the temporal degradation. In the present study, we explored neural correlates of this load-dependent decline of vWM in the human brain. Using electroencephalography and a classical change-detection task, we recorded neural measures of vWM that have been reported previously, such as the contralateral delay activity (CDA) and a suppression of alpha power (8–12 Hz). Results indicated that the load-dependent decline of vWM was more clearly reflected in the change in power and speed of alpha/beta rhythm than CDA, suggesting a close relationship of those signals to an attention-based preservation of WM fidelity.

The impact of retro-cue validity on working memory representation: Evidence from electroencephalograms

Visual working memory (VWM) performance can be improved by retrospectively cueing an item. The validity of retro-cues has an impact on the mechanisms underlying the retro-cue effect, but how non-cued representations are handled under different retro-cue validity conditions is not yet clear. Here, we used electroencephalograms to investigate whether retro-cue validity can affect the fate of non-cued representations in VWM. The participants were required to perform a change-detection task using a retro-cue with 80% or 20% validity. Contralateral delay activity and the lateralized alpha power were used to assess memory storage and selective attention, respectively. The retro-cue could redirect selective attention to the cued item under both validity conditions; however, the participants maintained the non-cued representations under the low-validity condition but dropped them from VWM under the high-validity condition. These results suggest that the maintenance of non-cued representations in VWM is affected by the expectation of cue validity and may be partially strategically driven. DATA AVAILABILITY: The datasets generated/analyzed during this study and experimental script have been added to https://osf.io/qtwc9/.

Active Working Memory and Simple Cognitive Operations

Working memory is thought to serve as a buffer for ongoing cognitive operations, even in tasks that have no obvious memory requirements. This conceptualization has been supported by dual-task experiments, in which interference is observed between a primary task involving short-term memory storage and a secondary task that presumably requires the same buffer as the primary task. Little or no interference is typically observed when the secondary task is very simple. Here, we test the hypothesis that even very simple tasks require the working memory buffer, but interference can be minimized by using activity-silent representations to store the information from the primary task. We tested this hypothesis using dual-task paradigm in which a simple discrimination task was interposed in the retention interval of a change detection task. We used contralateral delay activity (CDA) to track the active maintenance of information for the change detection task. We found that the CDA was massively disrupted after the interposed task. Despite this disruption of active maintenance, we found that performance in the change detection task was only slightly impaired, suggesting that activity-silent representations were used to retain the information for the change detection task. A second experiment replicated this result and also showed that automated discriminations could be performed without producing a large CDA disruption. Together, these results suggest that simple but non-automated discrimination tasks require the same processes that underlie active maintenance of information in working memory.

Shielding working-memory representations from temporally predictable external interference

Protecting working-memory content from distracting external sensory inputs and intervening tasks is an ubiquitous demand in daily life. Here, we ask whether and how temporal expectations about external events can help mitigate effects of such interference during working-memory retention. We manipulated the temporal predictability of interfering items that occurred during the retention period of a visual working-memory task and report that temporal expectations reduce the detrimental influence of external interference on subsequent memory performance. Moreover, to determine if the protective effects of temporal expectations rely on distractor suppression or involve shielding of internal representations, we compared effects after irrelevant distractors that could be ignored vs. interrupters that required a response. Whereas distractor suppression may be sufficient to confer protection from predictable distractors, any benefits after interruption are likely to involve memory shielding. We found similar benefits of temporal expectations after both types of interference. We conclude that temporal expectations may play an important role in safeguarding behaviour based on working memory - acting through mechanisms that include the shielding of internal content from external interference.

Inter-electrode correlations measured with EEG predict individual differences in cognitive ability

Human brains share a broadly similar functional organization with consequential individual variation. This duality in brain function has primarily been observed when using techniques that consider the spatial organization of the brain, such as MRI. Here, we ask whether these common and unique signals of cognition are also present in temporally sensitive but spatially insensitive neural signals. To address this question, we compiled electroencephalogram (EEG) data from individuals of both sexes while they performed multiple working memory tasks at two different data-collection sites (n = 171 and 165). Results revealed that trial-averaged EEG activity exhibited inter-electrode correlations that were stable within individuals and unique across individuals. Furthermore, models based on these inter-electrode correlations generalized across datasets to predict participants' working memory capacity and general fluid intelligence. Thus, inter-electrode correlation patterns measured with EEG provide a signature of working memory and fluid intelligence in humans and a new framework for characterizing individual differences in cognitive abilities.

When the time is right: Temporal dynamics of brain activity in healthy aging and dementia

Brain activity and communications are complex phenomena that dynamically unfold over time. However, in contrast with the large number of studies reporting neuroanatomical differences in activation relative to young adults, changes of temporal dynamics of neural activity during normal and pathological aging have been grossly understudied and are still poorly known. Here, we synthesize the current state of knowledge from MEG and EEG studies that aimed at specifying the effects of healthy and pathological aging on local and network dynamics, and discuss the clinical and theoretical implications of these findings. We argue that considering the temporal dynamics of brain activations and networks could provide a better understanding of changes associated with healthy aging, and the progression of neurodegenerative disease. Recent research has also begun to shed light on the association of these dynamics with other imaging modalities and with individual differences in cognitive performance. These insights hold great potential for driving new theoretical frameworks and development of biomarkers to aid in identifying and treating age-related cognitive changes.

Controlling the Flow of Distracting Information in Working Memory

Visual working memory (WM) must maintain relevant information, despite the constant influx of both relevant and irrelevant information. Attentional control mechanisms help determine which of this new information gets access to our capacity-limited WM system. Previous work has treated attentional control as a monolithic process-either distractors capture attention or they are suppressed. Here, we provide evidence that attentional capture may instead be broken down into at least two distinct subcomponent processes: (1) Spatial capture, which refers to when spatial attention shifts towards the location of irrelevant stimuli and (2) item-based capture, which refers to when item-based WM representations of irrelevant stimuli are formed. To dissociate these two subcomponent processes of attentional capture, we utilized a series of electroencephalography components that track WM maintenance (contralateral delay activity), suppression (distractor positivity), item individuation (N2pc), and spatial attention (lateralized alpha power). We show that new, relevant information (i.e., a task-relevant distractor) triggers both spatial and item-based capture. Irrelevant distractors, however, only trigger spatial capture from which ongoing WM representations can recover more easily. This fractionation of attentional capture into distinct subcomponent processes provides a refined framework for understanding how distracting stimuli affect attention and WM.

Distraction in Visual Working Memory: Resistance is Not Futile

Over half a century of research focused on understanding how working memory is capacity constrained has overshadowed the fact that it is also remarkably resistant to interference. Protecting goal-relevant information from distraction is a cornerstone of cognitive function that involves a multifaceted collection of control processes and storage mechanisms. Here, we discuss recent advances in cognitive psychology and neuroscience that have produced new insights into the nature of visual working memory and its ability to resist distraction. We propose that distraction resistance should be an explicit component in any model of working memory and that understanding its behavioral and neural correlates is essential for building a comprehensive understanding of real-world memory function.

Salient distractors open the door of perception: alpha desynchronization marks sensory gating in a working memory task

Focusing attention on relevant information while ignoring distracting stimuli is essential to the efficacy of working memory. Alpha- and theta-band oscillations have been linked to the inhibition of anticipated and attentionally avoidable distractors. However, the neurophysiological background of the rejection of task-irrelevant stimuli appearing in the focus of attention is not fully understood. We aimed to examine whether theta and alpha-band oscillations serve as an indicator of successful distractor rejection. Twenty-four students were enrolled in the study. 64-channel EEG was recorded during a modified Sternberg working memory task where weak and strong (salient) distractors were presented during the retention period. Event-related spectral perturbation in the alpha frequency band was significantly modulated by the saliency of the distracting stimuli, while theta oscillation was modulated by the need for cognitive control. Moreover, stronger alpha desynchronization to strong relative to weak distracting stimuli significantly increased the probability of mistakenly identifying the presented distractor as a member of the memory sequence. Therefore, our results suggest that alpha activity reflects the vulnerability of attention to distracting salient stimuli.