A two-phase model of resource allocation in visual working memory

Comparing retro-cue benefit mechanisms in visual working memory: completely valid vs. highly valid retro-cues

Visual working memory (VWM) plays a crucial role in temporarily maintaining and manipulating visual information. Retro-cue benefit (RCB) refers to the enhancement of memory performance when attention is directed toward a subset of items in VWM after their initial encoding. Our recent electroencephalogram (EEG) studies indicate that cue validity affects the mechanisms underlying RCB formation. However, previous research has not thoroughly examined whether these mechanisms differ between completely valid and highly valid cue conditions. This study investigates the consistency of RCB mechanisms under conditions of complete (100%) and high (80%) retro-cue validity. We manipulated retro-cue validity and examined cognitive processing mechanisms under different validity conditions using EEG. Specifically, we focused on the N2pc component, which reflects attentional resource allocation, and the contralateral delay activity (CDA) component, which reflects the quantity of information retained in VWM. The results, encompassing both behavioral and event-related potential (ERP) findings, show that participants in both the 100% and 80% cue validity conditions exhibit robust RCB. Notably, the degree of RCB remains consistent across these conditions, indicating that participants utilize retro-cues to enhance VWM performance to the same extent. In the 80% cue validity condition, a significant retro-cue cost (RCC) was observed, indicating that participants selectively discarded uncued items from VWM. In invalid trials, response accuracy drops to chance levels, supporting the removal hypothesis. ERP results reveal that attentional resource allocation (N2pc) and the quantity of retained information (CDA) remain uniform across cue validity conditions. The mechanism responsible for RCB formation appears to involve an all-or-nothing process of discarding uncued information rather than a flexible resource allocation strategy. This study provides insights into attention allocation and information-processing mechanisms in VWM, suggesting that conclusions drawn from tasks with completely valid retro-cues can be integrated with findings from highly valid cue tasks. These findings also illuminate the flexibility of internal attentional resource allocation during RCB formation and contribute to our understanding of attention processes in VWM.

Comparison of working memory performance in athletes and non-athletes: a meta-analysis of behavioural studies

The relationship between sports expertise and working memory (WM) has garnered increasing attention in experimental research. However, no meta-analysis has compared WM performance between athletes and non-athletes. This study addresses this gap by comparing WM performance between these groups and investigating potential moderators. A comprehensive literature search identified 21 studies involving 1455 participants from seven databases, including PubMed, Embase, and ProQuest. Athletes primarily engaged in basketball, football, and fencing, while non-athletes included some identified as sedentary. The risk of bias assessment indicated low risk across most domains. Publication bias, assessed through a funnel plot and statistical tests, showed no significant evidence of bias. The forest plot, using a random effects model, revealed moderate heterogeneity. The overall effect size indicated a statistically significant, albeit small, advantage for athletes over non-athletes (Hedges' g = 0.30), persisting across sports types and performance levels. Notably, this advantage was more pronounced when athletes were contrasted with a sedentary population (Hedges' g = 0.63), compared to the analysis where the sedentary population was excluded from the non-athlete reference group (Hedges' g = 0.15). Our findings indicate a consistent link between sports expertise and improved WM performance, while sedentary lifestyles appear to be associated with WM disadvantages.

Influence of presentation duration on filtering of irrelevant stimuli in visual working memory

In environments teeming with distractions, the ability to selectively focus on relevant information is crucial for advanced cognitive processing. Existing research using event-related potential (ERP) technology has shown active suppression of irrelevant stimuli during the consolidation phase of visual working memory (VWM). In previous studies, participants have always been given sufficient time to consolidate VWM, while suppressing distracting information. However, it remains unclear whether the suppression of irrelevant distractors requires continuous effort throughout their presence or whether this suppression is only necessary after the consolidation of task-relevant information. To address this question, our study examines whether distractor suppression is necessary in scenarios where consolidation time is limited. This research investigates the effect of varying presentation durations on the filtering of distractors in VWM. We tasked participants with memorizing two color stimuli and ignoring four distractors, presented for either 50 ms or 200 ms. Using ERP technology, we discovered that the distractor-induced distractor positivity (PD) amplitude is larger during longer presentation durations compared to shorter ones. These findings underscore the significant impact of presentation duration on the efficacy of distractor suppression in VWM, as prolonged exposure results in a stronger suppression effect on distractors. This study sheds light on the temporal dynamics of attention and memory, emphasizing the critical role of stimulus timing in cognitive tasks. These findings provide valuable insights into the mechanisms underlying VWM and have significant implications for models of attention and memory.

The differential impact of face distractors on visual working memory across encoding and delay stages

External distractions often occur when information must be retained in visual working memory (VWM)-a crucial element in cognitive processing and everyday activities. However, the distraction effects can differ if they occur during the encoding rather than the delay stages. Previous research on these effects used simple stimuli (e.g., color and orientation) rather than considering distractions caused by real-world stimuli on VWM. In the present study, participants performed a facial VWM task under different distraction conditions across the encoding and delay stages to elucidate the mechanisms of distraction resistance in the context of complex real-world stimuli. VWM performance was significantly impaired by delay-stage but not encoding-stage distractors (Experiment 1). In addition, the delay distraction effect arose primarily due to the absence of distractor process at the encoding stage rather than the presence of a distractor during the delay stage (Experiment 2). Finally, the impairment in the delay-distraction condition was not due to the abrupt appearance of distractors (Experiment 3). Taken together, these findings indicate that the processing mechanisms previously established for resisting distractions in VWM using simple stimuli can be extended to more complex real-world stimuli, such as faces.

A negative emotional state impairs individuals' ability to filter distractors from working memory: an ERP study

Capacity-limited visual working memory (VWM) requires that individuals have sufficient memory space and the ability to filter distractors. Negative emotional states are known to impact VWM storage, yet their influence on distractor filtering within VWM remains underexplored. We conducted direct neural measurement of participants (n = 56) who conducted a lateralized change detection task with distractors, while manipulating the emotional state by presenting neutral or negative images before each trial. We found a detrimental effect of distractors on memory accuracy under both neutral and negative emotional states. Using the event-related potential (ERP) component, contralateral delay activity (CDA; sensitive to VWM load), to observe the VWM load in each condition, we found that in the neutral state, the participants showed significantly higher late CDA amplitudes when remembering 4 targets compared with 2 targets and 2 targets with 2 distractors but no significant difference when remembering 2 targets compared with 2 targets with 2 distractors. In the negative state, no significant CDA amplitude differences were evident when remembering 4 targets and 2 targets, but CDA was significantly higher when remembering 2 targets with 2 distractors compared with 2 targets. These results suggest that the maximum number of items participants could store in VWM was lower under negative emotional states than under neutral emotional states. Importantly, the participants could filter out distractors when in a neutral emotional state but not in a negative emotional state, indicating that negative emotional states impair their ability to filter out distractors in VWM.

Heterogeneous Forgetting Rates and Greedy Allocation in Slot-Based Memory Networks Promotes Signal Retention

A key question in the neuroscience of memory encoding pertains to the mechanisms by which afferent stimuli are allocated within memory networks. This issue is especially pronounced in the domain of working memory, where capacity is finite. Presumably the brain must embed some "policy" by which to allocate these mnemonic resources in an online manner in order to maximally represent and store afferent information for as long as possible and without interference from subsequent stimuli. Here, we engage this question through a top-down theoretical modeling framework. We formally optimize a gating mechanism that projects afferent stimuli onto a finite number of memory slots within a recurrent network architecture. In the absence of external input, the activity in each slot attenuates over time (i.e., a process of gradual forgetting). It turns out that the optimal gating policy consists of a direct projection from sensory activity to memory slots, alongside an activity-dependent lateral inhibition. Interestingly, allocating resources myopically (greedily with respect to the current stimulus) leads to efficient utilization of slots over time. In other words, later-arriving stimuli are distributed across slots in such a way that the network state is minimally shifted and so prior signals are minimally "overwritten." Further, networks with heterogeneity in the timescales of their forgetting rates retain stimuli better than those that are more homogeneous. Our results suggest how online, recurrent networks working on temporally localized objectives without high-level supervision can nonetheless implement efficient allocation of memory resources over time.

Mutual interplay between cognitive offloading and secondary task performance

Various modern tools, such as smartphones, allow for cognitive offloading (i.e., the externalization of cognitive processes). In this study, we examined the use and consequences of cognitive offloading in demanding situations in which people perform multiple tasks concurrently-mimicking the requirements of daily life. In a preregistered study, we adapted the dual-task paradigm so that one of the tasks allowed for cognitive offloading. As a primary task, our participants (N = 172) performed the pattern copy task-a highly demanding working memory task that allows for offloading at various degrees. In this task, we manipulated the temporal costs of offloading. Concurrently, half of the participants responded to a secondary N-back task. As our main research question, we investigated the impact of offloading behavior on secondary task performance. We observed that more pronounced offloading in the condition without temporal costs was accompanied by a more accurate performance in the N-back task. Furthermore, the necessity to respond to the N-back task increased offloading behavior. These results suggest an interplay between offloading and secondary task performance: in demanding situations, individuals increasingly use cognitive offloading, which releases internal resources that can then be devoted to improving performance in other, concurrent tasks.

The slow rate of working memory consolidation from vision is a structural limit

The speed with which information from vision is transformed into working memory (WM) representations that resist interference from ongoing perception and cognition is the subject of conflicting results. Using distinct paradigms, researchers have arrived at estimates of the consolidation time course ranging from 25 ms to 1 s - a range of more than an order of magnitude. However, comparisons of consolidation duration across very different estimation paradigms rely on the implicit assumption that WM consolidation speed is a stable, structural constraint of the WM system. The extremely large variation in WM consolidation speed estimates across measurement approaches motivated the current work's goal of determining whether consolidation speed truly is a stable structural constraint of WM encoding, or instead might be under strategic control as suggested by some accounts. By manipulating the relative task priority of WM encoding and a subsequent sensorimotor decision in a dual-task paradigm, the current experiments demonstrate that the long duration of WM consolidation does not change as a result of task-specific strategies. These results allow comparison of WM consolidation across estimation approaches, are consistent with recent multi-phase WM consolidation models, and are consistent with consolidation duration being an inflexible structural limit.

Reward influences the allocation but not the availability of resources in visual working memory

Visual working memory possesses capacity constraints limiting the availability of resources for encoding and maintaining information. Studies have shown that prospective rewards improve performance on visual working memory tasks, but it remains unclear whether rewards increase total resource availability or simply influence the allocation of resources. Participants performed a continuous report visual working memory task with oriented grating stimuli. On each trial, participants were presented with a priority cue, which signaled the item most likely to be probed, and a reward cue, which signaled the magnitude of a performance-contingent reward. We showed that rewards decreased recall error for cued items and increased recall error for noncued items. This tradeoff was due to a change in the probability of successfully encoding a cued versus a noncued item rather than a change in recall precision or the probability of binding errors. Rewards did not modulate performance when priority cues were retroactively presented after the stimulus presentation period, indicating that rewards only affect resource allocation when participants are able to engage proactive control before encoding. Additionally, reward had no effect on visual working memory performance when priority cues were absent and thus unable to guide resource allocation. These findings indicate that rewards influence the flexible allocation of resources during selection and encoding in visual working memory, but do not augment total capacity. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Causal evidence for the role of the sensory visual cortex in visual short-term memory maintenance

The role of the sensory visual cortex during visual short-term memory (VSTM) remains controversial. This controversy is possibly due to methodological issues in previous attempts to investigate the effects of transcranial magnetic stimulation (TMS) on VSTM. The aim of this study was to use TMS, while covering previous methodological deficits. Sixty-four young adults were recruited to participate in two experiments (Experiment 1: n = 36; Experiment 2: n = 28) using a VSTM orientation change-detection task under TMS. Monocular vision was ensured using red-blue goggles combined with red-blue stimuli. Double-pulse TMS was delivered at different times (Experiment 1: 0, 200 or 1000 ms; Experiment 2: 200, 1000 ms) during a 2 s maintenance phase, on one side of the occipital hemisphere. In Experiment 2, a sham TMS condition was introduced. Decreased detection sensitivity (d') in the ipsilateral occipital hemisphere to visual hemifield, and in the real TMS (compared with sham TMS) condition indicated inhibitory TMS effects, and thus, a causal involvement of the sensory visual cortex during early (200 ms) and late (1000 ms) maintenance in VSTM. These findings are aligned with sensory recruitment, which proposes that both perceptual and memory processes rely upon the same neural substrates in the sensory visual cortex. The methods used in this study were preregistered and had received in-principle acceptance on 6 June 2022 (Stage 1 protocol can be found in: https://doi.org/10.17605/OSF.IO/EMPDT).

The facilitating effect of identical objects in visual working memory

According to the associative network of memory representations proposed by embedded processes models, the links between related memory representations were automatically established, which rendered these representations more easily activated. The present study adopted color recall tasks to explore whether the memory performance of identical objects was enhanced via the strengthening links between them, producing facilitating effect of identical objects. In Experiment 1, the number of identical items was manipulated. The results evidenced the facilitating effect, which was positively related to the number of identical objects. Experiment 2 modulated the spatial location of identical objects, which suggested that the facilitating effect was absent when two pairs of identical objects were located diagonally. Furthermore, Experiment 3 suggested that the facilitating effect was observed for the identical items which were presented in the second and fourth quadrants, rather than the first and third quadrants. Together, these results evidenced the facilitating effect of identical objects, which, however, was affected by spatial bias.

Improving fluid intelligence of children through working memory training: The role of inhibition control

Intelligence is strongly associated with working memory. Working memory training can improve fluid intelligence, but the underlying mechanism requires further investigation. Because inhibition control may play a key role in working memory training, this study investigated this process from an electrophysiological perspective. In total, 40 children aged 9 to 11 years were enrolled and randomly divided into a training group (n = 20) and a control group (n = 20). The training group received 20 days of working memory training, whereas the control group did not receive any training. Before and after the training period, all participants were tested using Raven's Standard Progressive Matrices (RSPM), and electrophysiological indicators were recorded while they performed go/no-go and Stroop tasks. The results revealed that relative to the control group, the training group had significantly improved RSPM scores in the test conducted after their training. For the go/no-go tasks, the training group exhibited a significant decrease in N2 amplitude, a significant increase in P3 amplitude, a significant decrease in theta band energy, and an improvement in response inhibition ability. No significant change was observed for the Stroop task. Correlation analysis revealed that an improvement in individual response inhibition can positively predict an improvement in fluid intelligence. These results suggest that working memory training enhances the fluid intelligence of children by enhancing their response inhibition ability.

Encoding specificity instead of online integration of real-world spatial regularities for objects in working memory

Most objects show high degrees of spatial regularity (e.g. beach umbrellas appear above, not under, beach chairs). The spatial regularities of real-world objects benefit visual working memory (VWM), but the mechanisms behind this spatial regularity effect remain unclear. The "encoding specificity" hypothesis suggests that spatial regularity will enhance the visual encoding process but will not facilitate the integration of information online during VWM maintenance. The "perception-alike" hypothesis suggests that spatial regularity will function in both visual encoding and online integration during VWM maintenance. We investigated whether VWM integrates sequentially presented real-world objects by focusing on the existence of the spatial regularity effect. Throughout five experiments, we manipulated the presentation (simultaneous vs. sequential) and regularity (with vs. without regularity) of memory arrays among pairs of real-world objects. The spatial regularity of memory objects presented simultaneously, but not sequentially, improved VWM performance. We also examined whether memory load, verbal suppression and masking, and memory array duration hindered the spatial regularity effect in sequential presentation. We found a stable absence of the spatial regularity effect, suggesting that the participants were unable to integrate real-world objects based on spatial regularities online. Our results support the encoding specificity hypothesis, wherein the spatial regularity of real-world objects can enhance the efficiency of VWM encoding, but VWM cannot exploit spatial regularity to help organize sampled sequential information into meaningful integrations.

Sensory recruitment in visual short-term memory: A systematic review and meta-analysis of sensory visual cortex interference using transcranial magnetic stimulation

Sensory visual areas are involved in encoding information in visual short-term memory (VSTM). Yet it remains unclear whether sensory visual cortex is a necessary component of the brain network for maintenance of information in VSTM. Here, we aimed to systematically review studies that have investigated the role of the sensory visual cortex in VSTM using transcranial magnetic stimulation (TMS) and to quantitatively explore these effects using meta-analyses. Fourteen studies were identified and reviewed. Eight studies provided sufficient data for meta-analysis. Two meta-analyses, one regarding the VSTM encoding phase (17 effect sizes) and one regarding the VSTM maintenance phase (15 effect sizes), two meta-regressions (32 effect sizes in each), and one exploratory meta-analysis were conducted. Our results indicate that the sensory visual cortex is similarly involved in both the encoding and maintenance VSTM phase. We suggest that some cases where evidence did not show significant TMS effects was due to low memory or perceptual task demands. Overall, these findings support the idea that sensory visual areas are part of the brain network responsible for successfully maintaining information in VSTM.

Examining the relationship between working memory consolidation and long-term consolidation

An emerging area of research is focused on the relationship between working memory and long-term memory and the likely overlap between these processes. Of particular interest is how some information first maintained in working memory is retained for longer periods and eventually preserved in long-term memory. The process of stabilizing transient memory representations for lasting retention is referred to as consolidation in both the working memory and long-term memory literature, although these have historically been viewed as independent constructs. The present review aims to investigate the relationship between working memory consolidation and long-term memory consolidation, which both have rich, but distinct, histories. This review provides an overview of the proposed models and neural mechanisms of both types of consolidation, as well as clinical findings related to consolidation and potential approaches for the manipulation of consolidation. Finally, two hypotheses are proposed to explain the relationship between working memory consolidation and long-term memory consolidation.

Negative and Positive Bias for Emotional Faces: Evidence from the Attention and Working Memory Paradigms

Visual attention and visual working memory (VWM) are two major cognitive functions in humans, and they have much in common. A growing body of research has investigated the effect of emotional information on visual attention and VWM. Interestingly, contradictory findings have supported both a negative bias and a positive bias toward emotional faces (e.g., angry faces or happy faces) in the attention and VWM fields. We found that the classical paradigms-that is, the visual search paradigm in attention and the change detection paradigm in VWM-are considerably similar. The settings of these paradigms could therefore be responsible for the contradictory results. In this paper, we compare previous controversial results from behavioral and neuroscience studies using these two paradigms. We suggest three possible contributing factors that have significant impacts on the contradictory conclusions regarding different emotional bias effects; these factors are stimulus choice, experimental setting, and cognitive process. We also propose new research directions and guidelines for future studies.

The two-stage process in visual working memory consolidation

Two hypotheses have been proposed to explain the formation manner for visual working memory (VWM) representations during the consolidation process: an all-or-none process hypothesis and a coarse-to-fine process hypothesis. However, neither the all-or-none process hypothesis nor the coarse-to-fine process hypothesis can stipulate clearly how VWM representations are formed during the consolidation process. In the current study, we propose a two-stage process hypothesis to reconcile these hypotheses. The two-stage process hypothesis suggests that the consolidation of coarse information is an all-or-none process in the early consolidation stage, while the consolidation of detailed information is a coarse-to-fine process in the late consolidation stage. By systematically manipulating the encoding time of memory stimuli, we asked participants to memorize one (Experiment 1) or two (Experiment 2) orientations in different encoding time intervals. We found that the memory rate increased linearly as the encoding time increased. More importantly, VWM precision remained constant when the encoding time was short, while the precision increased linearly as the encoding time increased when the encoding time was sufficient. These results supported the two-stage process hypothesis, which reconciles previous conflicting findings in the literature.

Negative emotional state modulates visual working memory in the late consolidation phase

Although a considerable literature has grown up around the interactions between emotional state and visual working memory (VWM) performance, the mechanism underlying the impact of the negative emotional state on VWM remains unclear. The present study aimed to test whether the influence of emotional state is related to the early phase or late phase of VWM consolidation process. Across three experiments, we found that the negative emotional state did not affect VWM performance when the presentation time of stimuli was short. However, when the presentation time was long, the negative emotional state increased the VWM precision and reduced the VWM number. According to the two-phase model proposed by Ye et al. (2017. A two-phase model of resource allocation in visual working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43(10), 1557-1566. doi: 10.1037/xlm0000376), the results suggested that negative emotional state could affect the late phase of resource allocation in VWM consolidation process, but it has no impact on the early consolidation phase. The findings from this study make important contributions to the current literature regarding the emotional modulation of VWM.

Working memory capacity affects trade-off between quality and quantity only when stimulus exposure duration is sufficient: Evidence for the two-phase model

The relation between visual working memory (VWM) capacity and attention has attracted much interest. In this study, we investigated the correlation between the participants’ VWM capacity and their ability to voluntarily trade off the precision and number of items remembered. The two-phase resource allocation model proposed by Ye et al. (2017) suggests that for a given set size, it takes a certain amount of consolidation time for an individual to control attention to adjust the VWM resources to trade off the precision and number. To verify whether trade-off ability varies across VWM capacity, we measured each individual’s VWM capacity and then conducted a colour recall task to examine their trade-off ability. By manipulating the task requirement, participants were instructed to memorise either more colours in a low-precision way or fewer colours in a high-precision way. We conducted two experiments by adjusting stimulus duration to be longer than predicted critical value (Experiment 1) and duration shorter than predicted critical value (Experiment 2). While the results of Experiment 1 showed a positive correlation between the VWM capacity and trade-off ability, the results of Experiment 2 showed a lack of such correlation. These results are consistent with the prediction from the two-phase model.

The impact of visual working memory capacity on the filtering efficiency of emotional face distractors

Emotional faces can serve as distractors for visual working memory (VWM) tasks. An event-related potential called contralateral delay activity (CDA) can measure the filtering efficiency of face distractors. Previous studies have investigated the influence of VWM capacity on filtering efficiency of simple neutral distractors but not of face distractors. We measured the CDA indicative of emotional face filtering during a VWM task related to facial identity. VWM capacity was measured in a separate colour change detection task, and participants were divided to high- and low-capacity groups. The high-capacity group was able to filter out distractors similarly irrespective of its facial emotion. In contrast, the low-capacity group failed in filtering the neutral and angry face distractors, while the filtering was efficient for the happy face distractors. The results indicate that potentially threatening faces are particularly difficult to filter if VWM capacity is limited.