Drift in Neural Population Activity Causes Working Memory to Deteriorate Over Time

Dissecting the components of error in analogue report tasks

Over the last two decades, the analogue report task has become a standard method for measuring the fidelity of visual representations across research domains including perception, attention, and memory. Despite its widespread use, there has been no methodical investigation of the different task parameters that might contribute to response variability. To address this gap, we conducted two experiments manipulating components of a typical analogue report test of memory for colour hue. We found that human response errors were independently affected by changes in storage and maintenance requirements of the task, demonstrated by a strong effect of set size even in the absence of a memory delay. In contrast, response variability remained unaffected by physical size of the colour wheel, implying negligible contribution of motor noise to task performance, or by its chroma radius, highlighting non-uniformity of the standard colour space. Comparing analogue report to a matched forced-choice task, we found variation in adjustment criterion made a limited contribution to analogue report variability, becoming meaningful only with low representational noise. Our findings validate the analogue report task as a robust measure of representational fidelity for most purposes, while also quantifying non-representational sources of noise that would limit its reliability in specialized settings.

Perceptual encoding benefit of visual memorability on visual memory formation

Human observers often exhibit remarkable consistency in remembering specific visual details, such as certain face images. This phenomenon is commonly attributed to visual memorability, a collection of stimulus attributes that enhance the long-term retention of visual information. However, the exact contributions of visual memorability to visual memory formation remain elusive as these effects could emerge anywhere from early perceptual encoding to post-perceptual memory consolidation processes. To clarify this, we tested three key predictions from the hypothesis that visual memorability facilitates early perceptual encoding that supports the formation of visual short-term memory (VSTM) and the retention of visual long-term memory (VLTM). First, we examined whether memorability benefits in VSTM encoding manifest early, even within the constraints of a brief stimulus presentation (100-200 ms; Experiment 1). We achieved this by manipulating stimulus presentation duration in a VSTM change detection task using face images with high- or low-memorability while ensuring they were equally familiar to the participants. Second, we assessed whether this early memorability benefit increases the likelihood of VSTM retention, even with post-stimulus masking designed to interrupt post-perceptual VSTM consolidation processes (Experiment 2). Last, we investigated the durability of memorability benefits by manipulating memory retention intervals from seconds to 24 h (Experiment 3). Across experiments, our data suggest that visual memorability has an early impact on VSTM formation, persisting across variable retention intervals and predicting subsequent VLTM overnight. Combined, these findings highlight that visual memorability enhances visual memory within 100-200 ms following stimulus onset, resulting in robust memory traces resistant to post-perceptual interruption and long-term forgetting.

Linking Cognitive Integrity to Working Memory Dynamics in the Aging Human Brain

Aging is accompanied by a decline of working memory, an important cognitive capacity that involves stimulus-selective neural activity that persists after stimulus presentation. Here, we unraveled working memory dynamics in older human adults (male and female) including those diagnosed with mild cognitive impairment (MCI) using a combination of behavioral modeling, neuropsychological assessment, and MEG recordings of brain activity. Younger adults (male and female) were studied with behavioral modeling only. Participants performed a visuospatial delayed match-to-sample task under systematic manipulation of the delay and distance between sample and test stimuli. Their behavior (match/nonmatch decisions) was fit with a computational model permitting the dissociation of noise in the internal operations underlying the working memory performance from a strategic decision threshold. Task accuracy decreased with delay duration and sample/test proximity. When sample/test distances were small, older adults committed more false alarms than younger adults. The computational model explained the participants' behavior well. The model parameters reflecting internal noise (not decision threshold) correlated with the precision of stimulus-selective cortical activity measured with MEG during the delay interval. The model uncovered an increase specifically in working memory noise in older compared with younger participants. Furthermore, in the MCI group, but not in the older healthy controls, internal noise correlated with the participants' clinically assessed cognitive integrity. Our results are consistent with the idea that the stability of working memory contents deteriorates in aging, in a manner that is specifically linked to the overall cognitive integrity of individuals diagnosed with MCI.

Representation and computation in visual working memory

The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have advanced our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event. New experimental tools have revealed that observers possess richer information about the uncertainty in their memories and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visuospatial WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research on human WM from a computational perspective, including the neural mechanisms that support it.

A dynamic neural resource model bridges sensory and working memory

Probing memory of a complex visual image within a few hundred milliseconds after its disappearance reveals significantly greater fidelity of recall than if the probe is delayed by as little as a second. Classically interpreted, the former taps into a detailed but rapidly decaying visual sensory or 'iconic' memory (IM), while the latter relies on capacity-limited but comparatively stable visual working memory (VWM). While iconic decay and VWM capacity have been extensively studied independently, currently no single framework quantitatively accounts for the dynamics of memory fidelity over these time scales. Here, we extend a stationary neural population model of VWM with a temporal dimension, incorporating rapid sensory-driven accumulation of activity encoding each visual feature in memory, and a slower accumulation of internal error that causes memorized features to randomly drift over time. Instead of facilitating read-out from an independent sensory store, an early cue benefits recall by lifting the effective limit on VWM signal strength imposed when multiple items compete for representation, allowing memory for the cued item to be supplemented with information from the decaying sensory trace. Empirical measurements of human recall dynamics validate these predictions while excluding alternative model architectures. A key conclusion is that differences in capacity classically thought to distinguish IM and VWM are in fact contingent upon a single resource-limited WM store.

Stochastic attractor models of visual working memory

This paper investigates models of working memory in which memory traces evolve according to stochastic attractor dynamics. These models have previously been shown to account for response-biases that are manifest across multiple trials of a visual working memory task. Here we adapt this approach by making the stable fixed points correspond to the multiple items to be remembered within a single-trial, in accordance with standard dynamical perspectives of memory, and find evidence that this multi-item model can provide a better account of behavioural data from continuous-report tasks. Additionally, the multi-item model proposes a simple mechanism by which swap-errors arise: memory traces diffuse away from their initial state and are captured by the attractors of other items. Swap-error curves reveal the evolution of this process as a continuous function of time throughout the maintenance interval and can be inferred from experimental data. Consistent with previous findings, we find that empirical memory performance is not well characterised by a purely-diffusive process but rather by a stochastic process that also embodies error-correcting dynamics.

A signal-detection account of item-based and ensemble-based visual change detection: A reply to Harrison, McMaster, and Bays

Growing empirical evidence shows that ensemble information (e.g., the average feature or feature variance of a set of objects) affects visual working memory for individual items. Recently, Harrison, McMaster, and Bays (2021) used a change detection task to test whether observers explicitly rely on ensemble representations to improve their memory for individual objects. They found that sensitivity to simultaneous changes in all memorized items (which also globally changed set summary statistics) rarely exceeded a level predicted by the so-called optimal summation model within the signal-detection framework. This model implies simple integration of evidence for change from all individual items and no additional evidence coming from ensemble. Here, we argue that performance at the level of optimal summation does not rule out the use of ensemble information. First, in two experiments, we show that, even if evidence from only one item is available at test, the statistics of the whole memory set affect performance. Second, we argue that optimal summation itself can be conceptually interpreted as one of the strategies of holistic, ensemble-based decision. We also redefine the reference level for the item-based strategy as the so-called "minimum rule," which predicts performance far below the optimum. We found that that both our and Harrison et al. (2021)'s observers consistently outperformed this level. We conclude that observers can rely on ensemble information when performing visual change detection. Overall, our work clarifies and refines the use of signal-detection analysis in measuring and modeling working memory.

Corvids optimize working memory by categorizing continuous stimuli

Working memory (WM) is a crucial element of the higher cognition of primates and corvid songbirds. Despite its importance, WM has a severely limited capacity and is vulnerable to noise. In primates, attractor dynamics mitigate the effect of noise by discretizing continuous information. Yet, it remains unclear whether similar dynamics are seen in avian brains. Here, we show jackdaws (Corvus monedula) have similar behavioral biases as humans; memories are less precise and more biased as memory demands increase. Model-based analysis reveal discrete attractors are evenly spread across the stimulus space. Altogether, our comparative approach suggests attractor dynamics in primates and corvids mitigate the effect of noise by systematically drifting towards specific attractors. By demonstrating this effect in an evolutionary distant species, our results strengthen attractor dynamics as general, adaptive biological principle to efficiently use WM.

Learning efficient representations of environmental priors in working memory

Experience shapes our expectations and helps us learn the structure of the environment. Inference models render such learning as a gradual refinement of the observer's estimate of the environmental prior. For instance, when retaining an estimate of an object's features in working memory, learned priors may bias the estimate in the direction of common feature values. Humans display such biases when retaining color estimates on short time intervals. We propose that these systematic biases emerge from modulation of synaptic connectivity in a neural circuit based on the experienced stimulus history, shaping the persistent and collective neural activity that encodes the stimulus estimate. Resulting neural activity attractors are aligned to common stimulus values. Using recently published human response data from a delayed-estimation task in which stimuli (colors) were drawn from a heterogeneous distribution that did not necessarily correspond with reported population biases, we confirm that most subjects' response distributions are better described by experience-dependent learning models than by models with fixed biases. This work suggests systematic limitations in working memory reflect efficient representations of inferred environmental structure, providing new insights into how humans integrate environmental knowledge into their cognitive strategies.

Assessing the interaction between working memory and perception through time

Content maintained in visual working memory changes concurrent visual processing, suggesting that visual working memory may recruit an overlapping neural representation with visual perception. However, it remains unclear whether visual working memory representations persist as a sensory code through time, or are recoded later into an abstract code. Here, we directly contrasted a temporal decay + visual code account and a temporal decay + abstract code account within the temporal dynamics of the interaction between working memory and perception. By manipulating the ISI (inter-stimulus interval) between working memory encoding and a perceptual discrimination task, we found that task-relevant and therefore actively maintained perceptual information parametrically altered participants' ability to discriminate perceptual stimuli even 4 s after encoding, whereas task-irrelevant information caused only an acutely transient effect. While continuously present, the size of this shift in discrimination thresholds gradually decreased over time. Concomitantly, the size of the bias in working memory reports increased over time. The opposing directions of threshold and bias effects are consistent with the local maintenance of information in perceptual areas, explained by a temporal decay + visual code account. As the maintained representation decays over time, its ability to alter incoming perceptual signals decreases (reduced threshold effects) while its likelihood of being impacted by those same signals increases (increased bias effects). Altogether, these results suggest that the readout of working memory relies on a sensory representation at a cost of increased interference by ongoing perception.

Awareness of the relative quality of spatial working memory representations

Working memory (WM) is the ability to maintain and manipulate information no longer accessible in the environment. The brain maintains WM representations over delay periods in noisy population-level activation patterns, resulting in variability in WM representations across items and trials. It is established that participants can introspect aspects of the quality of WM representations, and that they can accurately compare which of several WM representations of stimulus features like orientation or color is better on each trial. However, whether this ability to evaluate and compare the quality of multiple WM representations extends to spatial WM tasks remains unknown. Here, we employed a memory-guided saccade task to test recall errors for remembered spatial locations when participants were allowed to choose the most precise representation to report. Participants remembered either one or two spatial locations over a delay and reported one item's location with a saccade. On trials with two spatial locations, participants reported either the spatial location of a randomly cued item, or the location of the stimulus they remembered best. We found a significant improvement in recall error and increase in response time (RT) when participants reported their best-remembered item compared with trials in which they were randomly cued. These results demonstrate that participants can accurately introspect the relative quality of neural WM representations for spatial position, consistent with previous observations for other stimulus features, and support a model of WM coding involving noisy representations across items and trials.

A spiking network model for clustering report in a visual working memory task

Introduction

Working memory (WM) plays a key role in many cognitive processes, and great interest has been attracted by WM for many decades. Recently, it has been observed that the reports of the memorized color sampled from a uniform distribution are clustered, and the report error for the stimulus follows a Gaussian distribution.

Methods

Based on the well-established ring model for visuospatial WM, we constructed a spiking network model with heterogeneous connectivity and embedded short-term plasticity (STP) to investigate the neurodynamic mechanisms behind this interesting phenomenon.

Results

As a result, our model reproduced the clustering report given stimuli sampled from a uniform distribution and the error of the report following a Gaussian distribution. Perturbation studies showed that the heterogeneity of connectivity and STP are necessary to explain experimental observations.

Conclusion

Our model provides a new perspective on the phenomenon of visual WM in experiments.

Working memory is updated by reallocation of resources from obsolete to new items

Visual working memory (VWM) resources are limited, placing constraints on how much visual information can be simultaneously retained. During visually guided activity, stored information can quickly become outdated, so updating mechanisms are needed to ensure the contents of memory remain relevant to current task goals. In particular, successful deallocation of resources from items that become obsolete is likely to be critical for maintaining the precision of those representations still in memory. The experiments in this study involved presenting two memory arrays of coloured disks in sequence. The appearance of the second array was a cue to replace, rehearse, or add a new colour to the colours in memory. We predicted that successful resource reallocation should result in comparable recall precision when an item was replaced or rehearsed, owing to the removal of pre-replacement features. In contrast, a failure to update WM should lead to comparable precision with a condition in which a new colour was added to memory. We identified a very small proportion (∼5%) of trials in which participants incorrectly reported a feature from the first array in place of its replacement in the second, which we interpreted as a failure to incorporate the information from the second display into memory. Once these trials were discounted, precision estimates were consistent with complete redistribution of resources in the case of updating a single item. We conclude that working memory can be efficiently updated when previous information becomes obsolete, but that this is a demanding active process that occasionally fails.

Underestimation in temporal numerosity judgments computationally explained by population coding model

The ability to judge numerosity is essential to an animal’s survival. Nevertheless, the number of signals presented in a sequence is often underestimated. We attempted to elucidate the mechanism for the underestimation by means of computational modeling based on population coding. In the model, the population of neurons which were selective to the logarithmic number of signals responded to sequential signals and the population activity was integrated by a temporal window. The total number of signals was decoded by a weighted average of the integrated activity. The model predicted well the general trends in the human data while the prediction was not fully sufficient for the novel aging effect wherein underestimation was significantly greater for the elderly than for the young in specific stimulus conditions. Barring the aging effect, we can conclude that humans judge the number of signals in sequence by temporally integrating the neural representations of numerosity.

Forgetting in visual working memory: Internal noise explains decay of feature representations

The precision of visual working memory (VWM) representations decreases as time passes. It is often assumed that VWM decay is random and caused by internal noise accumulation. However, forgetting in VWM could occur systematically, such that some features deteriorate more rapidly than others. There exist only a few studies testing these two models of forgetting, with conflicting results. Here, decay of features in VWM was thoroughly tested using signal detection theory methods: psychophysical classification images, internal noise estimation, and receiver operant characteristic (ROC). A modified same–different memory task was employed with two retention times (500 and 4000 ms). Experiment 1 investigated VWM decay using a compound grating memory task, and Experiment 2 tested shape memory using radial frequency patterns. Memory performance dropped some 15% with increasing retention time in both experiments. Interestingly, classification images showed virtually indistinguishable weighting of stimulus features at both retention times, suggesting that VWM decay is not feature specific. Instead, we found a 77% increase in stimulus-independent internal noise at the longer retention time. Finally, the slope of the ROC curve plotted as z-scores was shallower at the longer retention time, indicating that the amount of stimulus-independent internal noise increased. Together these findings provide strong support for the idea that VWM decay does not result from a systematic loss of some stimulus features but instead is caused by uniformly increasing random internal noise.

Motivation improves working memory by two processes: Prioritisation and retrieval thresholds

Motivation can improve performance when the potential rewards outweigh the cost of effort expended. In working memory (WM), people can prioritise rewarded items at the expense of unrewarded items, suggesting a fixed memory capacity. But can capacity itself change with motivation? Across four experiments (N = 30-34) we demonstrate motivational improvements in WM even when all items were rewarded. However, this was not due to better memory precision, but rather better selection of the probed item within memory. Motivational improvements operated independently of encoding, maintenance, or attention shifts between items in memory. Moreover, motivation slowed responses. This contrasted with the benefits of rewarding items unequally, which allowed prioritisation of one item over another. We conclude that motivation can improve memory recall, not via precision or capacity, but via speed-accuracy trade-offs when selecting the item to retrieve.

Strategy-dependent effects of working-memory limitations on human perceptual decision-making

Deliberative decisions based on an accumulation of evidence over time depend on working memory, and working memory has limitations, but how these limitations affect deliberative decision-making is not understood. We used human psychophysics to assess the impact of working-memory limitations on the fidelity of a continuous decision variable. Participants decided the average location of multiple visual targets. This computed, continuous decision variable degraded with time and capacity in a manner that depended critically on the strategy used to form the decision variable. This dependence reflected whether the decision variable was computed either: (1) immediately upon observing the evidence, and thus stored as a single value in memory; or (2) at the time of the report, and thus stored as multiple values in memory. These results provide important constraints on how the brain computes and maintains temporally dynamic decision variables.

Working memory, the brain’s ability to temporarily store and recall information, is a critical part of decision making – but it has its limits. The brain can only store so much information, for so long. Since decisions are not often acted on immediately, information held in working memory ‘degrades’ over time. However, it is unknown whether or not this degradation of information over time affects the accuracy of later decisions.

The tactics that people use, knowingly or otherwise, to store information in working memory also remain unclear. Do people store pieces of information such as numbers, objects and particular details? Or do they tend to compute that information, make some preliminary judgement and recall their verdict later? Does the strategy chosen impact people’s decision-making?

To investigate, Schapiro et al. devised a series of experiments to test whether the limitations of working memory, and how people store information, affect the accuracy of decisions they make. First, participants were shown an array of colored discs on a screen. Then, either immediately after seeing the disks or a few seconds later, the participants were asked to recall the position of one of the disks they had seen, or the average position of all the disks. This measured how much information degraded for a decision based on multiple items, and how much for a decision based on a single item. From this, the method of information storage used to make a decision could be inferred.

Schapiro et al. found that the accuracy of people’s responses worsened over time, whether they remembered the position of each individual disk, or computed their average location before responding. The greater the delay between seeing the disks and reporting their location, the less accurate people’s responses tended to be. Similarly, the more disks a participant saw, the less accurate their response became. This suggests that however people store information, if working memory reaches capacity, decision-making suffers and that, over time, stored information decays.

Schapiro et al. also noticed that participants remembered location information in different ways depending on the task and how many disks they were shown at once. This suggests people adopt different strategies to retain information momentarily.

In summary, these findings help to explain how people process and store information to make decisions and how the limitations of working memory impact their decision-making ability. A better understanding of how people use working memory to make decisions may also shed light on situations or brain conditions where decision-making is impaired.

Mechanisms of feature binding in visual working memory are stable over long delays

The ability to accurately retain the binding between the features of different objects is a critical element of visual working memory. The underlying mechanism can be elucidated by analyzing correlations of response errors in dual-report experiments, in which participants have to report two features of a single item from a previously viewed stimulus array. Results from separate previous studies using different cueing conditions have indicated that location takes a privileged role in mediating binding between other features, in that largely independent response errors have been observed when location was used as a cue, but errors were highly correlated when location was one of the reported features. Earlier results from change detection tasks likewise support such a special role of location, but they also suggest that this role is substantially reduced for longer retention intervals in favor of object-based representation. In the present study, we replicated the findings of previous dual-report tasks with different cueing conditions, using matched stimuli and procedures. Moreover, we show that the observed patterns of error correlations remain qualitatively unchanged with longer retention intervals. Fits with neural population models demonstrate that the behavioral results at long, as well as short, delays are best explained by memory representations in independent feature maps, in which an item's features are bound to each other only via their shared location.

A direct neural measure of variable precision representations in visual working memory

Visual working memory (VWM) is an active representation enabling the manipulation of item information even in the absence of visual input. A common way to investigate VWM is to analyze the performance at later recall. This approach, however, leaves uncertainties about whether the variation of recall performance is attributable to item encoding and maintenance or to the testing of memorized information. Here, we record the contralateral delay activity (CDA), an established electrophysiological measure of item storage and maintenance, in human subjects performing a delayed orientation precision estimation task. This allows us to link the fluctuation of recall precision directly to the process of item encoding and maintenance. We show that for two sequentially encoded orientation items, the CDA amplitude reflects the precision of orientation recall of both items, with higher precision being associated with a larger amplitude. Furthermore, we show that the CDA amplitudes for the items vary independently from each other, suggesting that the precision of memory representations fluctuates independently.NEW & NOTEWORTHY The present work demonstrates for the first time that the contralateral delay activity (CDA), an online electrophysiological measure of the number of representations maintained in memory, is also a reliable measure of the precision of memory representations. Furthermore, we show that the CDA fluctuates independently for individual items held in memory, thereby providing unambiguous direct neurophysiological support for independently fluctuating memory representations.

Working Memory: From Neural Activity to the Sentient Mind

Working memory (WM) is the ability to maintain and manipulate information in the conscious mind over a timescale of seconds. This ability is thought to be maintained through the persistent discharges of neurons in a network of brain areas centered on the prefrontal cortex, as evidenced by neurophysiological recordings in nonhuman primates, though both the localization and the neural basis of WM has been a matter of debate in recent years. Neural correlates of WM are evident in species other than primates, including rodents and corvids. A specialized network of excitatory and inhibitory neurons, aided by neuromodulatory influences of dopamine, is critical for the maintenance of neuronal activity. Limitations in WM capacity and duration, as well as its enhancement during development, can be attributed to properties of neural activity and circuits. Changes in these factors can be observed through training-induced improvements and in pathological impairments. WM thus provides a prototypical cognitive function whose properties can be tied to the spiking activity of brain neurons. © 2021 American Physiological Society. Compr Physiol 11:1-41, 2021.

Temporal dynamics of implicit memory underlying serial dependence

Serial dependence is the effect in which the immediately preceding trial influences participants' responses to the current stimulus. But for how long does this bias last in the absence of interference from other stimuli? Here, we had 20 healthy young adult participants (12 women) perform a coincident timing task using different inter-trial intervals to characterize the serial dependence effect as the time between trials increases. Our results show that serial dependence abruptly decreases from 0.1 s to 1 s inter-trial interval, but it remains pronounced after that for up to 8 s. In addition, participants' response variability slightly decreases over longer intervals. We discuss these results in light of recent models suggesting that serial dependence might rely on a short-term memory trace kept through changes in synaptic weights, which might explain its long duration and apparent stability over time.

Does the presence of more features in a bound representation in working memory require extra object-based attention?

Recent studies have examined the role of attention in retaining bound representations in working memory (WM) and found that object-based attention plays a pivotal role. However, no study has investigated whether maintaining bound representations with more features in WM requires extra object-based attention. We investigated this by examining whether a secondary task consuming object-based attention was more disruptive to the maintenance of bindings in WM when more features were stored per object. We instructed participants to memorize three bound representations in a WM task while manipulating the number of features (two vs. three features) contained in each representation. Moreover, we manipulated whether a secondary task consuming object-based attention was interpolated into the maintenance phase of WM. If extra object-based attention was required after the addition of an extra feature in the bound representation, the secondary task would result in a greater disruption of the three- rather than two-featured binding. In two experiments, we found that the added secondary task significantly impaired the binding performance, but the performance of the two- and three-featured bindings was disrupted to the same extent. These results suggest that the presence of more features in a bound representation in WM does not require extra object-based attention.

Attention fluctuations impact ongoing maintenance of information in working memory

Working memory maintains information in a readily accessible state and has been shown to degrade as the length of the retention interval increases. Previous research has suggested that this decline is attributable to changes in precision as well as sudden loss of item representations. Here, by measuring trial-to-trial variations in performance, we examined an orthogonal distinction between the maximum number of items that an individual can store, and the probability of achieving that maximum. Across two experiments, we replicated the finding that performance declines after long (10 s) retention intervals, as well as past observations that forgetting was due to probabilistic dropping of individual items rather than all-or-none losses of the stored memories. Critically, longer retention intervals did not reduce the maximum amount of information that could be stored in working memory. Instead, lower attentional control accounted for a decreased probability of maintaining the maximum number of items in working memory. Thus, longer retention intervals impact working memory storage via fluctuations in attentional control that lower the probability of achieving a stable maximum storage capacity.

Embodied working memory during ongoing input streams

Sensory stimuli endow animals with the ability to generate an internal representation. This representation can be maintained for a certain duration in the absence of previously elicited inputs. The reliance on an internal representation rather than purely on the basis of external stimuli is a hallmark feature of higher-order functions such as working memory. Patterns of neural activity produced in response to sensory inputs can continue long after the disappearance of previous inputs. Experimental and theoretical studies have largely invested in understanding how animals faithfully maintain sensory representations during ongoing reverberations of neural activity. However, these studies have focused on preassigned protocols of stimulus presentation, leaving out by default the possibility of exploring how the content of working memory interacts with ongoing input streams. Here, we study working memory using a network of spiking neurons with dynamic synapses subject to short-term and long-term synaptic plasticity. The formal model is embodied in a physical robot as a companion approach under which neuronal activity is directly linked to motor output. The artificial agent is used as a methodological tool for studying the formation of working memory capacity. To this end, we devise a keyboard listening framework to delineate the context under which working memory content is (1) refined, (2) overwritten or (3) resisted by ongoing new input streams. Ultimately, this study takes a neurorobotic perspective to resurface the long-standing implication of working memory in flexible cognition.

A new toolbox to distinguish the sources of spatial memory error

Studying the sources of errors in memory recall has proven invaluable for understanding the mechanisms of working memory (WM). While one-dimensional memory features (e.g., color, orientation) can be analyzed using existing mixture modeling toolboxes to separate the influence of imprecision, guessing, and misbinding (the tendency to confuse features that belong to different memoranda), such toolboxes are not currently available for two-dimensional spatial WM tasks. Here we present a method to isolate sources of spatial error in tasks where participants have to report the spatial location of an item in memory, using two-dimensional mixture models. The method recovers simulated parameters well and is robust to the influence of response distributions and biases, as well as number of nontargets and trials. To demonstrate the model, we fit data from a complex spatial WM task and show the recovered parameters correspond well with previous spatial WM findings and with recovered parameters on a one-dimensional analogue of this task, suggesting convergent validity for this two-dimensional modeling approach. Because the extra dimension allows greater separation of memoranda and responses, spatial tasks turn out to be much better for separating misbinding from imprecision and guessing than one-dimensional tasks. Code for these models is freely available in the MemToolbox2D package and is integrated to work with the commonly used MATLAB package MemToolbox.

Stochastic sampling provides a unifying account of visual working memory limits

Research into human working memory limits has been shaped by the competition between different formal models, with a central point of contention being whether internal representations are continuous or discrete. Here we describe a sampling approach derived from principles of neural coding as a framework to understand working memory limits. Reconceptualizing existing models in these terms reveals strong commonalities between seemingly opposing accounts, but also allows us to identify specific points of difference. We show that the discrete versus continuous nature of sampling is not critical to model fits, but that, instead, random variability in sample counts is the key to reproducing human performance in both single- and whole-report tasks. A probabilistic limit on the number of items successfully retrieved is an emergent property of stochastic sampling, requiring no explicit mechanism to enforce it. These findings resolve discrepancies between previous accounts and establish a unified computational framework for working memory that is compatible with neural principles.

Drifting codes within a stable coding scheme for working memory

Working memory (WM) is important to maintain information over short time periods to provide some stability in a constantly changing environment. However, brain activity is inherently dynamic, raising a challenge for maintaining stable mental states. To investigate the relationship between WM stability and neural dynamics, we used electroencephalography to measure the neural response to impulse stimuli during a WM delay. Multivariate pattern analysis revealed representations were both stable and dynamic: there was a clear difference in neural states between time-specific impulse responses, reflecting dynamic changes, yet the coding scheme for memorised orientations was stable. This suggests that a stable subcomponent in WM enables stable maintenance within a dynamic system. A stable coding scheme simplifies readout for WM-guided behaviour, whereas the low-dimensional dynamic component could provide additional temporal information. Despite having a stable subspace, WM is clearly not perfect-memory performance still degrades over time. Indeed, we find that even within the stable coding scheme, memories drift during maintenance. When averaged across trials, such drift contributes to the width of the error distribution.

The visual nonverbal memory trace is fragile when actively maintained, but endures passively for tens of seconds

Despite attempts at active maintenance in the focus of attention, the fragile nature of the visual nonverbal memory trace may be revealed when the retention interval between target memoranda and probed recall on a trial is extended. In contrast, a passively maintained or unattended visual memory trace may be revealed as persisting proactive interference extending across quite extended intervals between trials in a recent probes task. The present study, comprising five experiments, used this task to explore the persistence of such a passive visual memory trace over time. Participants viewed some target visual items (for example, abstract colored patterns) followed by a variable retention interval and a probe item. The task was to report whether the probe matched one of the targets or not. A decaying active memory trace was indicated by poorer performance as the memory retention interval was extended on a trial. However, when the probe was a member of the target set from the preceding trial, task performance was poorer than a comparison novel probe, demonstrating proactive interference. Manipulations of the intertrial interval revealed that the temporal persistence of the passive memory trace of an old target was impressive, and proactive interference was largely resilient to a simple ‘cued forgetting’ manipulation. These data support the proposed two-process memory conception (active–passive memory) contrasting fragile active memory traces decaying over a few seconds with robust passive traces extending to tens of seconds.

Scale-invariant Characteristics of Forgetting: Toward a Unifying Account of Hippocampal Forgetting across Short and Long Timescales

After over 100 years of relative silence in the cognitive literature, recent advances in the study of the neural underpinnings of memory-specifically, the hippocampus-have led to a resurgence of interest in the topic of forgetting. This review draws a theoretically driven picture of the effects of time on forgetting of hippocampus-dependent memories. We review evidence indicating that time-dependent forgetting across short and long timescales is reflected in progressive degradation of hippocampal-dependent relational information. This evidence provides an important extension to a growing body of research accumulated in recent years, showing that-in contrast to the once prevailing view that the hippocampus is exclusively involved in memory and forgetting over long timescales-the role of the hippocampus also extends to memory and forgetting over short timescales. Thus, we maintain that similar rules govern not only remembering but also forgetting of hippocampus-dependent information over short and long timescales.

Error-correcting dynamics in visual working memory

Working memory is critical to cognition, decoupling behavior from the immediate world. Yet, it is imperfect; internal noise introduces errors into memory representations. Such errors have been shown to accumulate over time and increase with the number of items simultaneously held in working memory. Here, we show that discrete attractor dynamics mitigate the impact of noise on working memory. These dynamics pull memories towards a few stable representations in mnemonic space, inducing a bias in memory representations but reducing the effect of random diffusion. Model-based and model-free analyses of human and monkey behavior show that discrete attractor dynamics account for the distribution, bias, and precision of working memory reports. Furthermore, attractor dynamics are adaptive. They increase in strength as noise increases with memory load and experiments in humans show these dynamics adapt to the statistics of the environment, such that memories drift towards contextually-predicted values. Together, our results suggest attractor dynamics mitigate errors in working memory by counteracting noise and integrating contextual information into memories.

Neural representations in working memory are susceptible to internal noise, which scales with memory load. Here, the authors show that attractor dynamics mitigate the influence of internal noise by pulling memories towards a few stable representations.

Not all information in visual working memory is forgotten equally

To improve maintenance of task-relevant information in visual working memory (VWM), previously encoded, but no longer relevant, information can be suppressed or forgotten. However, it is unclear whether a cue directing attention to a subset of stimuli leads to complete forgetting for non-cued stimuli. The current study utilized a novel method of testing to-be forgotten information to determine if the effectiveness of forgetting differs depending on the type of encoded stimuli. Participants performed a directed forgetting change detection task, and importantly, the changed stimulus could be a novel stimulus or a to-be-forgotten stimulus. Stimulus type (colors, objects, or shapes) was manipulated across two experiments. Results suggest that a cue benefits memory for to-be-remembered information, but performance is not equivalent to never encoding to-be-forgotten information. Furthermore, the type of encoded information impacts the extent of forgetting.

Recall of facial expressions and simple orientations reveals competition for resources at multiple levels of the visual hierarchy

Many studies of visual working memory have tested humans' ability to reproduce primary visual features of simple objects, such as the orientation of a grating or the hue of a color patch, following a delay. A consistent finding of such studies is that precision of responses declines as the number of items in memory increases. Here we compared visual working memory for primary features and high-level objects. We presented participants with memory arrays consisting of oriented gratings, facial expressions, or a mixture of both. Precision of reproduction for all facial expressions declined steadily as the memory load was increased from one to five faces. For primary features, this decline and the specific distributions of error observed, have been parsimoniously explained in terms of neural population codes. We adapted the population coding model for circular variables to the non-circular and bounded parameter space used for expression estimation. Total population activity was held constant according to the principle of normalization and the intensity of expression was decoded by drawing samples from the Bayesian posterior distribution. The model fit the data well, showing that principles of population coding can be applied to model memory representations at multiple levels of the visual hierarchy. When both gratings and faces had to be remembered, an asymmetry was observed. Increasing the number of faces decreased precision of orientation recall, but increasing the number of gratings did not affect recall of expression, suggesting that memorizing faces involves the automatic encoding of low-level features, in addition to higher-level expression information.

Not-so-working Memory: Drift in Functional Magnetic Resonance Imaging Pattern Representations during Maintenance Predicts Errors in a Visual Working Memory Task

Working memory (WM) is critical to many aspects of cognition, but it frequently fails. Much WM research has focused on capacity limits, but even for single, simple features, the fidelity of individual representations is limited. Why is this? One possibility is that, because of neural noise and interference, neural representations do not remain stable across a WM delay, nor do they simply decay, but instead, they may "drift" over time to a new, less accurate state. We tested this hypothesis in a functional magnetic resonance imaging study of a match/nonmatch WM recognition task for a single item with a single critical feature: orientation. We developed a novel pattern-based index of "representational drift" to characterize ongoing changes in brain activity patterns throughout the WM maintenance period, and we were successfully able to predict performance on the match/nonmatch recognition task using this representational drift index. Specifically, in trials where the target and probe stimuli matched, participants incorrectly reported more nonmatches when their activity patterns drifted away from the target. In trials where the target and probe did not match, participants incorrectly reported more matches when their activity patterns drifted toward the probe. On the basis of these results, we contend that neural noise does not cause WM errors merely by degrading representations and increasing random guessing; instead, one means by which noise introduces errors is by pushing WM representations away from the target and toward other meaningful (yet incorrect) configurations. Thus, we demonstrate that behaviorally meaningful drift within representation space can be indexed by neuroimaging.

Test of a dynamic neural field model: spatial working memory is biased away from distractors

Attention facilitates the encoding (e.g., Awh, Anllo-Vento, & Hillyard, J Cognit Neurosci 12(5), 840-847, 2000) and maintenance of locations in spatial working memory (Awh, Vogel, & Oh, Atten, Percept Psychophys 78(4), 1043-1063, 2006). When individuals shift their attention during the maintenance period of a spatial working memory task, their memory of a target location tends to be biased in the direction of the attentional shift (Johnson & Spencer, 2016). Dynamic field theory predicts that in certain conditions, inhibitory mechanisms will result in biases away from distractors presented during the maintenance period of the task. Specifically, dynamic field theory predicts that memory responses will be biased toward distractors that are near the target location and biased away from distractors that are farther from the target location. In two experiments, the current study tested adults in a spatial memory task that required memorization of a single target location. On a subset of trials, a distractor appeared during the memory delay at different distances and directions from the target location. In contrast to the prediction, memory responses were biased away from distractors that were near the target location and not biased by distractors that were far from the target location, providing challenges for, dynamic field theory and other theories of spatial working memory.

The Regulatory Role of the Human Mediodorsal Thalamus

The function of the human mediodorsal thalamic nucleus (MD) has so far eluded a clear definition in terms of specific cognitive processes and tasks. Although it was at first proposed to play a role in long-term memory, a set of recent studies in animals and humans has revealed a more complex, and broader, role in several cognitive functions. The MD seems to play a multifaceted role in higher cognitive functions together with the prefrontal cortex and other cortical and subcortical brain areas. Specifically, we propose that the MD is involved in the regulation of cortical networks especially when the maintenance and temporal extension of persistent activity patterns in the frontal lobe areas are required.

Efficient Coding in Visual Working Memory Accounts for Stimulus-Specific Variations in Recall

Recall of visual features from working memory varies in both bias and precision depending on stimulus parameters. Whereas a number of models can approximate the average distribution of recall error across target stimuli, attempts to model how error varies with the choice of target have been ad hoc. Here we adapt a neural model of working memory to provide a principled account of these stimulus-specific effects, by allowing each neuron's tuning function to vary according to the principle of efficient coding, which states that neural responses should be optimized with respect to the frequency of stimuli in nature. For orientation, this means incorporating a prior that favors cardinal over oblique orientations. While continuing to capture the changes in error distribution with set size, the resulting model accurately described stimulus-specific variations as well, better than a slot-based competitor. Efficient coding produces a repulsive bias away from cardinal orientations, a bias that ought to be sensitive to changes in the environmental statistics. We subsequently tested whether shifts in the stimulus distribution influenced response bias to uniformly sampled target orientations in human subjects (of either sex). Across adaptation blocks, we manipulated the distribution of nontarget items by sampling from a bimodal congruent (incongruent) distribution with peaks centered on cardinal (oblique) orientations. Preadaptation responses were repulsed away from the cardinal axes. However, exposure to the incongruent distribution produced systematic decreases in repulsion that persisted after adaptation. This result confirms the role of prior expectation in generating stimulus-specific effects and validates the neural framework.

SIGNIFICANCE STATEMENT Theories of neural coding have been used successfully to explain how errors in recall from working memory depend on the number of items stored. However, recall of visual features also shows stimulus-specific variation in bias and precision. Here we unify two previously unconnected theories, the neural resource model of working memory and the efficient coding framework, to provide a principled account of these stimulus-specific effects. Given the importance of working memory limitations to multiple aspects of human and animal behavior, and the recent high-profile advances in theories of efficient coding, our modeling framework provides a richer, yet parsimonious, description of how orientation encoding influences visual working memory performance.