Variability in the quality of visual working memory

Dissecting the components of error in analogue report tasks : Error in analogue report tasks

Over the last two decades, the analogue report task has become a standard method for measuring the fidelity of visual representations across research domains including perception, attention, and memory. Despite its widespread use, there has been no methodical investigation of the different task parameters that might contribute to response variability. To address this gap, we conducted two experiments manipulating components of a typical analogue report test of memory for colour hue. We found that human response errors were independently affected by changes in storage and maintenance requirements of the task, demonstrated by a strong effect of set size even in the absence of a memory delay. In contrast, response variability remained unaffected by physical size of the colour wheel, implying negligible contribution of motor noise to task performance, or by its chroma radius, highlighting non-uniformity of the standard colour space. Comparing analogue report to a matched forced-choice task, we found variation in adjustment criterion made a limited contribution to analogue report variability, becoming meaningful only with low representational noise. Our findings validate the analogue report task as a robust measure of representational fidelity for most purposes, while also quantifying non-representational sources of noise that would limit its reliability in specialized settings.

Neural mechanisms of resource allocation in working memory

To mitigate capacity limits of working memory, people allocate resources according to an item's relevance. However, the neural mechanisms supporting such a critical operation remain unknown. Here, we developed computational neuroimaging methods to decode and demix neural responses associated with multiple items in working memory with different priorities. In striate and extrastriate cortex, the gain of neural responses tracked the priority of memoranda. Higher-priority memoranda were decoded with smaller error and lower uncertainty. Moreover, these neural differences predicted behavioral differences in memory prioritization. Remarkably, trialwise variability in the magnitude of delay activity in frontal cortex predicted differences in decoded precision between low and high-priority items in visual cortex. These results suggest a model in which feedback signals broadcast from frontal cortex sculpt the gain of memory representations in visual cortex according to behavioral relevance, thus, identifying a neural mechanism for resource allocation.

EZ-CDM: Fast, simple, robust, and accurate estimation of circular diffusion model parameters

The investigation of cognitive processes that form the basis of decision-making in paradigms involving continuous outcomes has gained the interest of modeling researchers who aim to develop a dynamic decision theory that accounts for both speed and accuracy. One of the most important of these continuous models is the circular diffusion model (CDM, Smith. Psychological Review, 123(4), 425. 2016), which posits a noisy accumulation process mathematically described as a stochastic two-dimensional Wiener process inside a disk. Despite the considerable benefits of this model, its mathematical intricacy has limited its utilization among scholars. Here, we propose a straightforward and user-friendly method for estimating the CDM parameters and fitting the model to continuous-scale data using simple formulas that can be readily computed and do not require theoretical knowledge of model fitting or extensive programming. Notwithstanding its simplicity, we demonstrate that the aforementioned method performs with a level of accuracy that is comparable to that of the maximum likelihood estimation method. Furthermore, a robust version of the method is presented, which maintains its simplicity while exhibiting a high degree of resistance to contaminant responses. Additionally, we show that the approach is capable of reliably measuring the key parameters of the CDM, even when these values are subject to across-trial variability. Finally, we demonstrate the practical application of the method on experimental data. Specifically, an illustrative example is presented wherein the method is employed along with estimating the probability of guessing. It is hoped that the straightforward methodology presented here will, on the one hand, help narrow the divide between theoretical constructs and empirical observations on continuous response tasks and, on the other hand, inspire cognitive psychology researchers to shift their laboratory investigations towards continuous response paradigms.

Stochastic attractor models of visual working memory

This paper investigates models of working memory in which memory traces evolve according to stochastic attractor dynamics. These models have previously been shown to account for response-biases that are manifest across multiple trials of a visual working memory task. Here we adapt this approach by making the stable fixed points correspond to the multiple items to be remembered within a single-trial, in accordance with standard dynamical perspectives of memory, and find evidence that this multi-item model can provide a better account of behavioural data from continuous-report tasks. Additionally, the multi-item model proposes a simple mechanism by which swap-errors arise: memory traces diffuse away from their initial state and are captured by the attractors of other items. Swap-error curves reveal the evolution of this process as a continuous function of time throughout the maintenance interval and can be inferred from experimental data. Consistent with previous findings, we find that empirical memory performance is not well characterised by a purely-diffusive process but rather by a stochastic process that also embodies error-correcting dynamics.

Perceptual similarity judgments do not predict the distribution of errors in working memory

Population coding models provide a quantitative account of visual working memory (VWM) retrieval errors with a plausible link to the response characteristics of sensory neurons. Recent work has provided an important new perspective linking population coding to variables of signal detection, including d-prime, and put forward a new hypothesis: that the distribution of recall errors on, for example, a color wheel, is a consequence of the psychological similarity between points in that stimulus space, such that the exponential-like psychophysical distance scaling function can fulfil the role of population tuning and obviate the need to fit a tuning width parameter to recall data. Using four different visual feature spaces, we measured psychophysical similarity and memory errors in the same participants. Our results revealed strong evidence for a common source of variability affecting similarity judgments and recall estimates but did not support any consistent relationship between psychophysical similarity functions and VWM errors. At the group level, the responsiveness functions obtained from the psychophysical similarity task diverged strongly from those that provided the best fit to working memory errors. At the individual level, we found convincing evidence against an association between observed and best-fitting similarity functions. Finally, our results show that the newly proposed exponential-like responsiveness function has in general no advantage over the canonical von Mises (circular normal) function assumed by previous population coding models. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The role of recollection and familiarity in visual working memory: A mixture of threshold and signal detection processes

Whether working memory reflects a thresholded recollection process whereby only a limited number of items are maintained in memory, or a signal detection process in which each studied item is increased in familiarity strength, is a topic of considerable debate. A review of visual working memory studies that have examined receiver operating characteristics (ROCs) across a broad set of materials and test conditions indicates that both signal detection and threshold processes contribute to working memory. In addition, the role that these two processes play varies systematically across conditions, such that a threshold process plays a particularly critical role when binary old/new judgments are required, when changes are relatively discrete, and when the hippocampus does not contribute to performance. In contrast, a signal detection process plays a greater role when confidence judgments are required, when the materials or the changes are global in nature, and when the hippocampus contributes to performance. In addition, the ROC results indicate that in standard single-probe tests of working memory, items that are maintained in an active recollected state support both recall-to-accept and recall-to-reject responses; whereas in complex-probe tests, recollection preferentially supports recall-to-reject; and in item-recognition tests it preferentially supports recall-to-accept. Moreover, there is growing evidence that these threshold and strength-based processes are related to distinct states of conscious awareness whereby they support perceiving- and sensing-based responses, respectively. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Both mOTS-words and pOTS-words prefer emoji stimuli over text stimuli during a reading task

The visual word form area in the occipitotemporal sulcus (OTS), here referred to as OTS-words, responds more strongly to text than other visual stimuli and is crucial for reading. We hypothesized, that this text preference may be driven by a preference for reading tasks, as in most prior fMRI studies only the text stimuli were readable. Hence, we performed three fMRI experiments (N=15) and systematically varied the participant's task and the stimulus, investigating mOTS-words and pOTS-words subregions. In experiment 1, we contrasted text stimuli with non-readable visual stimuli (faces, limbs, houses, objects). Experiment 2 utilized an fMRI adaptation paradigm, presenting compound words in text or emoji formats. In experiment 3, participants performed a reading or a color task on compound words in text or emoji format. Using experiment 1 data, we identified mOTS-words and pOTS-words by contrasting texts with non-readable stimuli. In experiment 2, pOTS-words, but not mOTS-words, showed fMRI adaptation for compound words in both text and emoji formats. In experiment 3, surprisingly, both subregions showed higher responses to compound words in emoji than text format. Moreover, mOTS-words showed higher responses during the reading than the color task and a task-stimulus interaction. Multivariate analyses revealed that distributed responses in pOTS-words encode the visual stimulus, while responses in mOTS-words encode both stimulus and task. Together, our findings suggest that the function of the OTS-words subregions goes beyond the specific visual processing of text and that these regions are flexibly recruited whenever semantic meaning needs to be assigned to visual input.

The dynamics of attentional guidance by working memory contents

Working memory (WM) contents can guide attention toward matching sensory information in the environment, but there are mixed findings regarding whether only a single prioritized item or multiple items held in WM can guide attention. The present study examines the limit of WM-guided attention with a novel task procedure and mouse trajectory analysis. Specifically, we introduced a perceptual-matching task utilizing the continuous estimation procedure within the maintenance interval of a WM task for one or two colors. We found that the overall perceptual matching mouse trajectory were robustly biased toward the location of WM-match color on the color-wheel (i.e., attraction bias), but only at memory set size one. However, the analysis of circular mouse trajectory distributions, through hierarchical Bayesian modeling, revealed two separable central peaks at both memory set sizes. Furthermore, model-free analysis demonstrated that the perceptual matching mouse trajectory patterns were similar regardless of memory set sizes. Together, these results support the single-item account and highlight the utility of mouse trajectory analyses in hypothesis testing in experimental psychology.

Probabilistic and rich individual working memories revealed by a betting game

When asked to remember a color, do people remember a point estimate (e.g., a particular shade of red), a point estimate plus an uncertainty estimate, or are memory representations rich probabilistic distributions over feature space? We asked participants to report the color of a circle held in working memory. Rather than collecting a single report per trial, we had participants place multiple bets to create trialwise uncertainty distributions. Bet dispersion correlated with performance, indicating that internal uncertainty guided bet placement. While the first bet was on average the most precisely placed, the later bets systematically shifted the distribution closer to the target, resulting in asymmetrical distributions about the first bet. This resulted in memory performance improvements when averaging across bets, and overall suggests that memory representations contain more information than can be conveyed by a single response. The later bets contained target information even when the first response would generally be classified as a guess or report of an incorrect item, suggesting that such failures are not all-or-none. This paradigm provides multiple pieces of evidence that memory representations are rich and probabilistic. Crucially, standard discrete response paradigms underestimate the amount of information in memory representations.

Corvids optimize working memory by categorizing continuous stimuli

Working memory (WM) is a crucial element of the higher cognition of primates and corvid songbirds. Despite its importance, WM has a severely limited capacity and is vulnerable to noise. In primates, attractor dynamics mitigate the effect of noise by discretizing continuous information. Yet, it remains unclear whether similar dynamics are seen in avian brains. Here, we show jackdaws (Corvus monedula) have similar behavioral biases as humans; memories are less precise and more biased as memory demands increase. Model-based analysis reveal discrete attractors are evenly spread across the stimulus space. Altogether, our comparative approach suggests attractor dynamics in primates and corvids mitigate the effect of noise by systematically drifting towards specific attractors. By demonstrating this effect in an evolutionary distant species, our results strengthen attractor dynamics as general, adaptive biological principle to efficiently use WM.

Learning efficient representations of environmental priors in working memory

Experience shapes our expectations and helps us learn the structure of the environment. Inference models render such learning as a gradual refinement of the observer's estimate of the environmental prior. For instance, when retaining an estimate of an object's features in working memory, learned priors may bias the estimate in the direction of common feature values. Humans display such biases when retaining color estimates on short time intervals. We propose that these systematic biases emerge from modulation of synaptic connectivity in a neural circuit based on the experienced stimulus history, shaping the persistent and collective neural activity that encodes the stimulus estimate. Resulting neural activity attractors are aligned to common stimulus values. Using recently published human response data from a delayed-estimation task in which stimuli (colors) were drawn from a heterogeneous distribution that did not necessarily correspond with reported population biases, we confirm that most subjects' response distributions are better described by experience-dependent learning models than by models with fixed biases. This work suggests systematic limitations in working memory reflect efficient representations of inferred environmental structure, providing new insights into how humans integrate environmental knowledge into their cognitive strategies.

A spatially continuous diffusion model of visual working memory

We present results from five visual working memory (VWM) experiments in which participants were briefly shown between 2 and 6 colored squares. They were then cued to recall the color of one of the squares and they responded by choosing the color on a continuous color wheel. The experiments provided response proportions and response time (RT) measures as a function of angle for the choices. Current VWM models for this task include discrete models that assume an item is either within working memory or not and resource models that assume that memory strength varies as a function of the number of items. Because these models do not include processes that allow them to account for RT data, we implemented them within the spatially continuous diffusion model (SCDM, Ratcliff, 2018) and use the experimental data to evaluate these combined models. In the SCDM, evidence retrieved from memory is represented as a spatially continuous normal distribution and this drives the decision process until a criterion (represented as a 1-D line) is reached, which produces a decision. Noise in the accumulation process is represented by continuous Gaussian process noise over spatial position. The models that fit best from the discrete and resource-based classes converged on a common model that had a guessing component and that allowed the height of the normal memory-strength distribution to vary with number of items. The guessing component was implemented as a regular decision process driven by a flat evidence distribution, a zero-drift process. The combination of choice and RT data allows models that were not identifiable based on choice data alone to be discriminated.

Reward influences the allocation but not the availability of resources in visual working memory

Visual working memory possesses capacity constraints limiting the availability of resources for encoding and maintaining information. Studies have shown that prospective rewards improve performance on visual working memory tasks, but it remains unclear whether rewards increase total resource availability or simply influence the allocation of resources. Participants performed a continuous report visual working memory task with oriented grating stimuli. On each trial, participants were presented with a priority cue, which signaled the item most likely to be probed, and a reward cue, which signaled the magnitude of a performance-contingent reward. We showed that rewards decreased recall error for cued items and increased recall error for noncued items. This tradeoff was due to a change in the probability of successfully encoding a cued versus a noncued item rather than a change in recall precision or the probability of binding errors. Rewards did not modulate performance when priority cues were retroactively presented after the stimulus presentation period, indicating that rewards only affect resource allocation when participants are able to engage proactive control before encoding. Additionally, reward had no effect on visual working memory performance when priority cues were absent and thus unable to guide resource allocation. These findings indicate that rewards influence the flexible allocation of resources during selection and encoding in visual working memory, but do not augment total capacity. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Distinguishing guesses from fuzzy memories: Further evidence for item limits in visual working memory

There is consistent debate over whether capacity in working memory (WM) is subject to an item limit, or whether an unlimited number of items can be held in this online memory system. The item limit hypothesis clearly predicts guessing responses when capacity is exceeded, and proponents of this view have highlighted evidence for guessing in visual working memory tasks. Nevertheless, various models that deny item limits can explain the same empirical patterns by asserting extremely low fidelity representations that cannot be distinguished from guesses. To address this ambiguity, we employed a task for which guess responses elicited a qualitatively distinct pattern from low fidelity memories. Inspired by work from Rouder et al. (2014), we employed an orientation WM task that required subjects to recall the precise orientation of each of six memoranda presented 1 s earlier. The orientation stimuli were created by rotating the position of a "clock hand" inside a circular region that was demarcated by four colored quadrants. Critically, when observers guess with these stimuli, the distribution of responses is biased towards the center of these quadrants, creating a "banded" pattern that cannot be explained by a low precision memory. We confirmed the presence of this guessing pattern using formal model comparisons, and we show that the prevalence of this pattern matches observers' own reports of when they thought they were guessing. Thus, these findings provide further evidence for guessing behaviors predicted by item limit models of WM capacity.

Predicted utility modulates working memory fidelity in the brain

The predicted utility of information stored in working memory (WM) is hypothesized to influence the strategic allocation of WM resources. Prior work has shown that when information is prioritized, it is remembered with greater precision relative to other remembered items. However, these paradigms often complicate interpretation of the effects of predicted utility on item fidelity due to a concurrent memory load. Likewise, no fMRI studies have examined whether the predicted utility of an item modulates fidelity in the neural representation of items during the memory delay without a concurrent load. In the current study, we used fMRI to investigate whether predicted utility influences fidelity of WM representations in the brain. Using a generative model multivoxel analysis approach to estimate the quality of remembered representations across predicted utility conditions, we observed that items with greater predicted utility are maintained in memory with greater fidelity, even when they are the only item being maintained. Further, we found that this pattern follows a parametric relationship where more predicted utility corresponded to greater fidelity. These precision differences could not be accounted for based on a redistribution of resources among already-remembered items. Rather, we interpret these results in terms of a gating mechanism that allows for pre-allocation of resources based on predicted value alone. This evidence supports a theoretical distinction between resource allocation that occurs as a result of load and resource pre-allocation that occurs as a result of predicted utility.

Examining the role of attentional allocation in working memory precision with pupillometry in children and adults

Working memory (WM) precision, or the fidelity with which items can be remembered, is an important aspect of WM capacity that increases over childhood. Why individuals are more or less precise from moment to moment and why WM becomes more stable with age are not yet fully understood. Here, we examined the role of attentional allocation in visual WM precision in children aged 8 to 13 years and young adults aged 18 to 27 years, as measured by fluctuations in pupil dilation during stimulus encoding and maintenance. Using mixed models, we examined intraindividual links between change in pupil diameter and WM precision across trials and the role of developmental differences in these associations. Through probabilistic modeling of error distributions and the inclusion of a visuomotor control task, we isolated mnemonic precision from other cognitive processes. We found an age-related increase in mnemonic precision that was independent of guessing behavior, serial position effects, fatigue or loss of motivation across the experiment, and visuomotor processes. Trial-by-trial analyses showed that trials with smaller changes in pupil diameter during encoding and maintenance predicted more precise responses than trials with larger changes in pupil diameter within individuals. At encoding, this relationship was stronger for older participants. Furthermore, the pupil-performance coupling grew across the delay period-particularly or exclusively for adults. These results suggest a functional link between pupil fluctuations and WM precision that grows over development; visual details may be stored more faithfully when attention is allocated efficiently to a sequence of objects at encoding and throughout a delay period.

Spatial and non-spatial feature binding impairments in visual working memory in schizophrenia

Working memory (WM) impairments are well recognized in schizophrenia patients (PSZ) and contribute to poor psycho-social outcomes in this population. Distinct neural networks underlay the ability to encode and recall visual and spatial information raising the possibility that profile of visual working memory performance may help pinpoint dysfunctional neural correlates in schizophrenia. This study assessed the resolution and associative aspects of visual working memory deficits in schizophrenia and whether these deficits arise during encoding or maintenance processes. A total of 60 participants (30 PSZ and 30 healthy controls) matched in age, gender and education assessed on a modified object in place (OiPT), a delayed non-match-to-sample (DNMST) and a delayed spatial estimation (DSET) task. Patients demonstrated lower accuracy than controls in binding visual features of the same object and recognizing novel objects as well as lower precision recalling the location of a memorized target. Moreover, response choice set size affected recognition accuracy more in PSZ than controls. However, delay duration affected spatial recall precisions, binding, and recognition accuracy equally in the two groups. Our results suggest that visual working memory (vWM) impairments in schizophrenia predominantly reflect spatial and non-spatial binding deficits, with largely preserved discrete feature information. Moreover, these impairments likely arise more during encoding than during maintenance. These binding deficits may reflect impaired effective neural functional connectivity observed in schizophrenia.

Awareness of the relative quality of spatial working memory representations

Working memory (WM) is the ability to maintain and manipulate information no longer accessible in the environment. The brain maintains WM representations over delay periods in noisy population-level activation patterns, resulting in variability in WM representations across items and trials. It is established that participants can introspect aspects of the quality of WM representations, and that they can accurately compare which of several WM representations of stimulus features like orientation or color is better on each trial. However, whether this ability to evaluate and compare the quality of multiple WM representations extends to spatial WM tasks remains unknown. Here, we employed a memory-guided saccade task to test recall errors for remembered spatial locations when participants were allowed to choose the most precise representation to report. Participants remembered either one or two spatial locations over a delay and reported one item's location with a saccade. On trials with two spatial locations, participants reported either the spatial location of a randomly cued item, or the location of the stimulus they remembered best. We found a significant improvement in recall error and increase in response time (RT) when participants reported their best-remembered item compared with trials in which they were randomly cued. These results demonstrate that participants can accurately introspect the relative quality of neural WM representations for spatial position, consistent with previous observations for other stimulus features, and support a model of WM coding involving noisy representations across items and trials.

No effect of monetary reward in a visual working memory task

Previous work has shown that humans distribute their visual working memory (VWM) resources flexibly across items: the higher the importance of an item, the better it is remembered. A related, but much less studied question is whether people also have control over the total amount of VWM resource allocated to a task. Here, we approach this question by testing whether increasing monetary incentives results in better overall VWM performance. In three experiments, subjects performed a delayed-estimation task on the Amazon Turk platform. In the first two experiments, four groups of subjects received a bonus payment based on their performance, with the maximum bonus ranging from $0 to $10 between groups. We found no effect of the amount of bonus on intrinsic motivation or on VWM performance in either experiment. In the third experiment, reward was manipulated on a trial-by-trial basis using a within-subjects design. Again, no evidence was found that VWM performance depended on the magnitude of potential reward. These results suggest that encoding quality in visual working memory is insensitive to monetary reward, which has implications for resource-rational theories of VWM.

Audiovisual interaction with rate-varying signals

A task-irrelevant, amplitude-modulating sound influences perception of a size-modulating visual stimulus. To probe the limits of this audiovisual interaction we vary the second temporal derivative of object size and of sound amplitude. In the study's first phase subjects see a visual stimulus size-modulating with f ″ ( x ) > 0, 0, or <0, and judge each one's rate as increasing, constant, or decreasing. Visual stimuli are accompanied by a steady, non-modulated auditory stimulus. The novel combination of multiple stimuli and multi-alternative responses allows subjects' similarity space to be estimated from the stimulus-response confusion matrix. In the study's second phase, rate-varying visual stimuli are presented in concert with auditory stimuli whose second derivative also varied. Subjects identified each visual stimuli as one of the three types, while trying to ignore the accompanying sound. Unlike some previous results with f ″ ( x ) fixed at 0, performance benefits relatively little when visual and auditory stimuli share the same directional change in modulation. However, performance does drop when visual and auditory stimului differ in their directions of rate change. Our task's computational demands may make it particularly vulnerable to the effects of a dynamic task-irrelevant stimulus.

Measuring memory is harder than you think: How to avoid problematic measurement practices in memory research

We argue that critical areas of memory research rely on problematic measurement practices and provide concrete suggestions to improve the situation. In particular, we highlight the prevalence of memory studies that use tasks (like the "old/new" task: "have you seen this item before? yes/no") where quantifying performance is deeply dependent on counterfactual reasoning that depends on the (unknowable) distribution of underlying memory signals. As a result of this difficulty, different literatures in memory research (e.g., visual working memory, eyewitness identification, picture memory, etc.) have settled on a variety of fundamentally different metrics to get performance measures from such tasks (e.g., A', corrected hit rate, percent correct, d', diagnosticity ratios, K values, etc.), even though these metrics make different, contradictory assumptions about the distribution of latent memory signals, and even though all of their assumptions are frequently incorrect. We suggest that in order for the psychology and neuroscience of memory to become a more cumulative, theory-driven science, more attention must be given to measurement issues. We make a concrete suggestion: The default memory task for those simply interested in performance should change from old/new ("did you see this item'?") to two-alternative forced-choice ("which of these two items did you see?"). In situations where old/new variants are preferred (e.g., eyewitness identification; theoretical investigations of the nature of memory signals), receiver operating characteristic (ROC) analysis should be performed rather than a binary old/new task.

Working memory is updated by reallocation of resources from obsolete to new items

Visual working memory (VWM) resources are limited, placing constraints on how much visual information can be simultaneously retained. During visually guided activity, stored information can quickly become outdated, so updating mechanisms are needed to ensure the contents of memory remain relevant to current task goals. In particular, successful deallocation of resources from items that become obsolete is likely to be critical for maintaining the precision of those representations still in memory. The experiments in this study involved presenting two memory arrays of coloured disks in sequence. The appearance of the second array was a cue to replace, rehearse, or add a new colour to the colours in memory. We predicted that successful resource reallocation should result in comparable recall precision when an item was replaced or rehearsed, owing to the removal of pre-replacement features. In contrast, a failure to update WM should lead to comparable precision with a condition in which a new colour was added to memory. We identified a very small proportion (∼5%) of trials in which participants incorrectly reported a feature from the first array in place of its replacement in the second, which we interpreted as a failure to incorporate the information from the second display into memory. Once these trials were discounted, precision estimates were consistent with complete redistribution of resources in the case of updating a single item. We conclude that working memory can be efficiently updated when previous information becomes obsolete, but that this is a demanding active process that occasionally fails.

Swap errors in visual working memory are fully explained by cue-feature variability

In cue-based recall from working memory, incorrectly reporting features of an uncued item may be referred to as a “swap” error. One account of these errors ascribes them to variability in memory for the cue features leading to erroneous selection of a non-target item, especially if it is similar to the target in the cue-feature dimension. However, alternative accounts of swap errors include cue-independent misbinding, and strategic guessing when the cued item is not in memory. Here we investigated the cause of swap errors by manipulating the variability with which either cue or report features (orientations in Exp 1; motion directions in Exp 2) were encoded. We found that swap errors increased with increasing variability in memory for the cue features, and their changing frequency could be quantitatively predicted based on recall variability when the same feature was used for report. These results are inconsistent with the hypothesis that swaps are a strategic response to forgotten items, and suggest that swap errors could be wholly accounted for by confusions due to cue-dimension variability. In a third experiment we examined whether spatial configuration of memory arrays in tasks with spatial cueing has an influence on swap error frequency. We observed a specific tendency to make swap errors to non-targets located precisely opposite to the cued location, suggesting that stimulus positions are partially encoded in a non-metric format.

Trial-by-trial mouse trajectory predicts variance in precision across working memory representations: A critical reanalysis of Hao et al. (2021)

Multiple representations in visual working memory (VWM) can vary in mnemonic precision. This inhomogeneity of VWM precision has received some support from recent studies with the whole-report procedure, in which all memory items are recalled in free or forced orders. Recently, Hao et al. (2021, Cognition, 214, 104739) added a novel item-selection stage before each memory recall and found smaller between-trial variance in mouse trajectory during the selection stage in free-recall condition as compared with forced recall, which was taken as evidence for less between-item interference and the resulting precision benefit under free recall. Here, we reanalyzed the original dataset with a different analytic approach and attempted independent hypothesis testing focusing on within-trial trajectory deviations. We found that the direction of trial-by-trial trajectory bias for the first to-be-recalled item was predictive of the relative mnemonic precision of the remaining items. Critically, this relationship was only present for forced recall but not for free recall. Hierarchical Bayesian modeling of recall errors further identified that this relationship was selectively driven by VWM precision. Together, our reanalysis provides evidence for the source of between-item interference and its direct association with variable precision of VWM representations, and further highlights the novel methodological benefits of probing memory decisional processes using mouse trajectory data.

Capacity and Allocation across Sensory and Short-Term Memories

Human memory consists of sensory memory (SM), short-term memory (STM), and long-term memory (LTM). SM enables a large capacity, but decays rapidly. STM has limited capacity, but lasts longer. The traditional view of these memory systems resembles a leaky hourglass, the large top and bottom portions representing the large capacities of SM and LTM, whereas the narrow portion in the middle represents the limited capacity of STM. The “leak” in the top part of the hourglass depicts the rapid decay of the contents of SM. However, recently, it was shown that major bottlenecks for motion processing exist prior to STM, and the “leaky hourglass” model was replaced by a “leaky flask” model with a narrower top part to capture bottlenecks prior to STM. The leaky flask model was based on data from one study, and the first goal of the current paper was to test if the leaky flask model would generalize by using a different set of data. The second goal of the paper was to explore various block diagram models for memory systems and determine the one best supported by the data. We expressed these block diagram models in terms of statistical mixture models and, by using the Bayesian information criterion (BIC), found that a model with four components, viz., SM, attention, STM, and guessing, provided the best fit to our data. In summary, we generalized previous findings about early qualitative and quantitative bottlenecks, as expressed in the leaky flask model and showed that a four-process model can provide a good explanation for how visual information is processed and stored in memory.

The hippocampus supports high-precision binding in visual working memory

It is well established that the hippocampus is critical for long-term episodic memory, but a growing body of research suggests that it also plays a critical role in supporting memory over very brief delays as measured in tests of working memory (WM). However, the circumstances under which the hippocampus is necessary for WM and the specific processes that it supports remain controversial. We propose that the hippocampus supports WM by binding together high-precision properties of an event, and we test this claim by examining the precision of color-location bindings in a visual WM task in which participants report the precise color of studied items using a continuous color wheel. Amnestic patients with hippocampal damage were significantly impaired at retrieving these colors after a 1-s delay, and these impairments reflected a reduction in the precision of those memories rather than increases in total memory failures or binding errors. Moreover, a parallel fMRI study in healthy subjects revealed that neural activity in the head and body of the hippocampus was directly related to the precision of visual WM decisions. Together, these results indicate that the hippocampus is critical in complex high-precision binding that supports memory over brief delays.

How Working Memory and Reinforcement Learning Are Intertwined: A Cognitive, Neural, and Computational Perspective

Reinforcement learning and working memory are two core processes of human cognition and are often considered cognitively, neuroscientifically, and algorithmically distinct. Here, we show that the brain networks that support them actually overlap significantly and that they are less distinct cognitive processes than often assumed. We review literature demonstrating the benefits of considering each process to explain properties of the other and highlight recent work investigating their more complex interactions. We discuss how future research in both computational and cognitive sciences can benefit from one another, suggesting that a key missing piece for artificial agents to learn to behave with more human-like efficiency is taking working memory's role in learning seriously. This review highlights the risks of neglecting the interplay between different processes when studying human behavior (in particular when considering individual differences). We emphasize the importance of investigating these dynamics to build a comprehensive understanding of human cognition.

Joint representation of working memory and uncertainty in human cortex

Neural representations of visual working memory (VWM) are noisy, and thus, decisions based on VWM are inevitably subject to uncertainty. However, the mechanisms by which the brain simultaneously represents the content and uncertainty of memory remain largely unknown. Here, inspired by the theory of probabilistic population codes, we test the hypothesis that the human brain represents an item maintained in VWM as a probability distribution over stimulus feature space, thereby capturing both its content and uncertainty. We used a neural generative model to decode probability distributions over memorized locations from fMRI activation patterns. We found that the mean of the probability distribution decoded from retinotopic cortical areas predicted memory reports on a trial-by-trial basis. Moreover, in several of the same mid-dorsal stream areas, the spread of the distribution predicted subjective trial-by-trial uncertainty judgments. These results provide evidence that VWM content and uncertainty are jointly represented by probabilistic neural codes.

Visual Short-Term Memory for Coherent and Sequential Motion: A rTMS Investigation

We investigated the role of the human medio-temporal complex (hMT+) in the memory encoding and storage of a sequence of four coherently moving random dot kinematograms (RDKs), by applying repetitive transcranial magnetic stimulation (rTMS) during an early or late phase of the retention interval. Moreover, in a second experiment, we also tested whether disrupting the functional integrity of hMT+ during the early phase impaired the precision of the encoded motion directions. Overall, results showed that both recognition accuracy and precision were worse in middle serial positions, suggesting the occurrence of primacy and recency effects. We found that rTMS delivered during the early (but not the late) phase of the retention interval was able to impair not only recognition of RDKs, but also the precision of the retained motion direction. However, such impairment occurred only for RDKs presented in middle positions along the presented sequence, where performance was already closer to chance level. Altogether these findings suggest an involvement of hMT+ in the memory encoding of visual motion direction. Given that both position sequence and rTMS modulated not only recognition but also the precision of the stored information, these findings are in support of a model of visual short-term memory with a variable resolution of each stored item, consistent with the assigned amount of memory resources, and that such item-specific memory resolution is supported by the functional integrity of area hMT+.

A direct neural measure of variable precision representations in visual working memory

Visual working memory (VWM) is an active representation enabling the manipulation of item information even in the absence of visual input. A common way to investigate VWM is to analyze the performance at later recall. This approach, however, leaves uncertainties about whether the variation of recall performance is attributable to item encoding and maintenance or to the testing of memorized information. Here, we record the contralateral delay activity (CDA), an established electrophysiological measure of item storage and maintenance, in human subjects performing a delayed orientation precision estimation task. This allows us to link the fluctuation of recall precision directly to the process of item encoding and maintenance. We show that for two sequentially encoded orientation items, the CDA amplitude reflects the precision of orientation recall of both items, with higher precision being associated with a larger amplitude. Furthermore, we show that the CDA amplitudes for the items vary independently from each other, suggesting that the precision of memory representations fluctuates independently.NEW & NOTEWORTHY The present work demonstrates for the first time that the contralateral delay activity (CDA), an online electrophysiological measure of the number of representations maintained in memory, is also a reliable measure of the precision of memory representations. Furthermore, we show that the CDA fluctuates independently for individual items held in memory, thereby providing unambiguous direct neurophysiological support for independently fluctuating memory representations.

To see, not to see or to see poorly: Perceptual quality and guess rate as a function of electroencephalography (EEG) brain activity in an orientation perception task

Detection of visual stimuli fluctuates over time, and these fluctuations have been shown to correlate with time domain evoked activity and frequency-domain periodic activity. However, it is unclear if these fluctuations are related to a change in guess rate, perceptual quality or both. Here we determined whether the quality of perception randomly varies across trials or is fixed so that the variability is the same. Then we estimated how perceptual quality and guess rate on an orientation perception task relate to electroencephalography (EEG) activity. Response errors were fitted to variable precision models and the standard mixture model to determine whether perceptual quality is from a varying or fixed distribution. Overall, the best fit was the standard mixture model that assumes that response variability can be defined by a fixed distribution. The power and phase of 2-7 Hz post-target activities were found to vary along with task performance in that more accurate trials had greater power, and the preferred phase differed significantly between accurate and guess trials. Guess rate and σ were significantly lower on trials with high 2- to 3-Hz power than low, and the difference started around 250-ms post-target. These effects coincide with changes in the P3 event-related potential (ERP): There was a more positive deflection in the accurate trials versus guesses. These results suggest that the spread of errors (perceptual quality) can be characterised by a fixed range of values. Where the errors fall within that range is modulated by the post-target power in the lower-frequency bands and their analogous ERPs.

Uncertainty is maintained and used in working memory

What are the contents of working memory? In both behavioral and neural computational models, a working memory representation is typically described by a single number, namely, a point estimate of a stimulus. Here, we asked if people also maintain the uncertainty associated with a memory and if people use this uncertainty in subsequent decisions. We collected data in a two-condition orientation change detection task; while both conditions measured whether people used memory uncertainty, only one required maintaining it. For each condition, we compared an optimal Bayesian observer model, in which the observer uses an accurate representation of uncertainty in their decision, to one in which the observer does not. We find that this “Use Uncertainty” model fits better for all participants in both conditions. In the first condition, this result suggests that people use uncertainty optimally in a working memory task when that uncertainty information is available at the time of decision, confirming earlier results. Critically, the results of the second condition suggest that this uncertainty information was maintained in working memory. We test model variants and find that our conclusions do not depend on our assumptions about the observer's encoding process, inference process, or decision rule. Our results provide evidence that people have uncertainty that reflects their memory precision on an item-specific level, maintain this information over a working memory delay, and use it implicitly in a way consistent with an optimal observer. These results challenge existing computational models of working memory to update their frameworks to represent uncertainty.

Free-recall benefit, inhomogeneity and between-item interference in working memory

We investigated visual working memory (VWM) with a whole-report task, where participants were asked to sequentially recall all the items in an order either chosen by themselves (free recall) or randomly chosen by the computer (forced recall). Comparisons between free and forced recalls helped us understand important but largely neglected aspects of VWM, such as inhomogeneity (different levels of precision) and between-item interference. One unique part of our task was the introduction of a separate item-selection stage before each recall, during which participants located the next item to recall. Their mouse trajectory was recorded and served as a dynamic measure of between-item interference over time. We found a free-recall benefit: the overall precision of all items is higher in free recall than in forced recall. Meanwhile, during item-selection, free recall is associated with faster localization of the target and less interference from the other items in memory. We also found evidence for inhomogeneity and discuss the connection of inhomogeneity and between-item interference to the free-recall benefit.

Location-independent feature binding in visual working memory for sequentially presented objects

Spatial location is believed to have a privileged role in binding features held in visual working memory. Supporting this view, Pertzov and Husain (Attention, Perception, & Psychophysics, 76(7), 1914–1924, 2014) reported that recall of bindings between visual features was selectively impaired when items were presented sequentially at the same location compared to sequentially at different locations. We replicated their experiment, but additionally tested whether the observed impairment could be explained by perceptual interference during encoding. Participants viewed four oriented bars in highly discriminable colors presented sequentially either at the same or different locations, and after a brief delay were cued with one color to reproduce the associated orientation. When we used the same timing as the original study, we reproduced its key finding of impaired binding memory in the same-location condition. Critically, however, this effect was significantly modulated by the duration of the inter-stimulus interval, and disappeared if memoranda were presented with longer delays between them. In a second experiment, we tested whether the effect generalized to other visual features, namely reporting of colors cued by stimulus shape. While we found performance deficits in the same-location condition, these did not selectively affect binding memory. We argue that the observed effects are best explained by encoding interference, and that memory for feature binding is not necessarily impaired when memoranda share the same location.

Serial reproduction reveals the geometry of visuospatial representations

A primary function of human vision is to encode and recall spatial information about visual scenes. We developed an experimental paradigm that reveals the structure of human spatial memory priors in unprecedented detail. We ran a series of 85 large-scale online experiments with 9,202 participants that paint an intricate picture of these priors. Our results suggest a way to understand visuospatial representations as reflecting the efficient allocation of coding resources. In a radical departure from traditional theory, we introduce a model that reinterprets spatial memory priors as reflecting an optimal allocation of perceptual resources. We validate the predictions of the model experimentally by showing that perceptual biases are correlated with variations in discrimination accuracy.

An essential function of the human visual system is to locate objects in space and navigate the environment. Due to limited resources, the visual system achieves this by combining imperfect sensory information with a belief state about locations in a scene, resulting in systematic distortions and biases. These biases can be captured by a Bayesian model in which internal beliefs are expressed in a prior probability distribution over locations in a scene. We introduce a paradigm that enables us to measure these priors by iterating a simple memory task where the response of one participant becomes the stimulus for the next. This approach reveals an unprecedented richness and level of detail in these priors, suggesting a different way to think about biases in spatial memory. A prior distribution on locations in a visual scene can reflect the selective allocation of coding resources to different visual regions during encoding (“efficient encoding”). This selective allocation predicts that locations in the scene will be encoded with variable precision, in contrast to previous work that has assumed fixed encoding precision regardless of location. We demonstrate that perceptual biases covary with variations in discrimination accuracy, a finding that is aligned with simulations of our efficient encoding model but not the traditional fixed encoding view. This work demonstrates the promise of using nonparametric data-driven approaches that combine crowdsourcing with the careful curation of information transmission within social networks to reveal the hidden structure of shared visual representations.

Allocation of resources in working memory: Theoretical and empirical implications for visual search

Recently, working memory (WM) has been conceptualized as a limited resource, distributed flexibly and strategically between an unlimited number of representations. In addition to improving the precision of representations in WM, the allocation of resources may also shape how these representations act as attentional templates to guide visual search. Here, we reviewed recent evidence in favor of this assumption and proposed three main principles that govern the relationship between WM resources and template-guided visual search. First, the allocation of resources to an attentional template has an effect on visual search, as it may improve the guidance of visual attention, facilitate target recognition, and/or protect the attentional template against interference. Second, the allocation of the largest amount of resources to a representation in WM is not sufficient to give this representation the status of attentional template and thus, the ability to guide visual search. Third, the representation obtaining the status of attentional template, whether at encoding or during maintenance, receives an amount of WM resources proportional to its relevance for visual search. Thus defined, the resource hypothesis of visual search constitutes a parsimonious and powerful framework, which provides new perspectives on previous debates and complements existing models of template-guided visual search.

Functional annotation of human cognitive states using deep graph convolution

A key goal in neuroscience is to understand brain mechanisms of cognitive functions. An emerging approach is "brain decoding", which consists of inferring a set of experimental conditions performed by a participant, using pattern classification of brain activity. Few works so far have attempted to train a brain decoding model that would generalize across many different cognitive tasks drawn from multiple cognitive domains. To tackle this problem, we proposed a multidomain brain decoder that automatically learns the spatiotemporal dynamics of brain response within a short time window using a deep learning approach. We evaluated the decoding model on a large population of 1200 participants, under 21 different experimental conditions spanning six different cognitive domains, acquired from the Human Connectome Project task-fMRI database. Using a 10s window of fMRI response, the 21 cognitive states were identified with a test accuracy of 90% (chance level 4.8%). Performance remained good when using a 6s window (82%). It was even feasible to decode cognitive states from a single fMRI volume (720ms), with the performance following the shape of the hemodynamic response. Moreover, a saliency map analysis demonstrated that the high decoding performance was driven by the response of biologically meaningful brain regions. Together, we provide an automated tool to annotate human brain activity with fine temporal resolution and fine cognitive granularity. Our model shows potential applications as a reference model for domain adaptation, possibly making contributions in a variety of domains, including neurological and psychiatric disorders.

Stimulus-Specific Visual Working Memory Representations in Human Cerebellar Lobule VIIb/VIIIa

fMRI research has revealed that cerebellar lobule VIIb/VIIIa exhibits load-dependent activity that increases with the number of items held in visual working memory (VWM). However, it remains unclear whether these cerebellar responses reflect processes specific to VWM or more general visual attentional mechanisms. To investigate this question, we examined whether cerebellar activity during the delay period of a VWM task is selective for stimuli held in working memory. A sample of male and female human subjects performed a VWM continuous report task in which they were retroactively cued to remember the direction of motion of moving dot stimuli. Cerebellar lobule VIIb/VIIIa delay-period activation accurately decoded the direction of the remembered stimulus, as did frontal and parietal regions of the dorsal attention network. Arguing against a motor explanation, no other cerebellar area exhibited stimulus specificity, including the oculomotor vermis, a key area associated with eye movement control. Finer-scale analysis revealed that the medial portion of lobule VIIb and to a lesser degree the lateral most portion of lobules VIIb and VIIIa, which exhibit robust resting state connectivity with frontal and parietal regions of the dorsal attention network, encoded the identity of the remembered stimulus, while intermediate portions of lobule VIIb/VIIIa did not. These findings of stimulus-specific coding of VWM within lobule VIIb/VIIIa indicate for the first time that the distributed network responsible for the encoding and maintenance of mnemonic representations extends to the cerebellum.SIGNIFICANCE STATEMENT There is considerable debate concerning where in the brain the contents of visual working memory (VWM) are stored. To date, this literature has primarily focused on the role of regions located within cerebral cortex. There is growing evidence for cerebellar involvement in higher-order cognitive functions including working memory. While the cerebellum has been previously shown to be recruited by VWM paradigms, it is unclear whether any portion of cerebellum actively encodes and maintains mnemonic representations. The present study demonstrates that cerebellar lobule VIIb/VIIIa activity patterns are selective for remembered stimuli and that this selectivity persists in the absence of perceptual input. These findings provide novel evidence for the participation of cerebellar structures in the persistent storage of visual information.

Statistical learning as a reference point for memory distortions: Swap and shift errors

Humans use regularities in the environment to facilitate learning, often without awareness or intent. How might such regularities distort long-term memory? Here, participants studied and reported the colors of objects in a long-term memory paradigm, uninformed that certain colors were sampled more frequently overall. When participants misreported an object's color, these errors were often centered around the average studied color (i.e., "Rich" color), demonstrating swap errors in long-term memory due to imposed statistical regularities. We observed such swap errors regardless of memory load, explicit knowledge, or the distance in color space between the correct color of the tested object and the Rich color. An explicit guessing strategy where participants intentionally made swap errors when uncertain could not fully account for our results. We discuss other potential sources of observed swap errors such as false memory and implicit biased guessing. Although less robust than swap errors, evidence was also observed for subtle shift errors towards or away from the Rich color dependent on the color distance between the correct color and the Rich color. Together, these findings of swap and shift errors provide converging evidence for memory distortion mechanisms induced by a reference point, bridging a gap in the literature between how attention to regularities similarly influences visual working memory and visual long-term memory.

A new toolbox to distinguish the sources of spatial memory error

Studying the sources of errors in memory recall has proven invaluable for understanding the mechanisms of working memory (WM). While one-dimensional memory features (e.g., color, orientation) can be analyzed using existing mixture modeling toolboxes to separate the influence of imprecision, guessing, and misbinding (the tendency to confuse features that belong to different memoranda), such toolboxes are not currently available for two-dimensional spatial WM tasks. Here we present a method to isolate sources of spatial error in tasks where participants have to report the spatial location of an item in memory, using two-dimensional mixture models. The method recovers simulated parameters well and is robust to the influence of response distributions and biases, as well as number of nontargets and trials. To demonstrate the model, we fit data from a complex spatial WM task and show the recovered parameters correspond well with previous spatial WM findings and with recovered parameters on a one-dimensional analogue of this task, suggesting convergent validity for this two-dimensional modeling approach. Because the extra dimension allows greater separation of memoranda and responses, spatial tasks turn out to be much better for separating misbinding from imprecision and guessing than one-dimensional tasks. Code for these models is freely available in the MemToolbox2D package and is integrated to work with the commonly used MATLAB package MemToolbox.

The interaction between language and working memory: a systematic review of fMRI studies in the past two decades

Language processing involves other cognitive domains, including Working Memory (WM). Much detail about the neural correlates of language and WM interaction remains unclear. This review summarizes the evidence for the interaction between WM and language obtained via functional Magnetic Resonance Imaging (fMRI) in the past two decades. The search was limited to PubMed, Google Scholar, Science direct and Neurosynth for working memory, language, fMRI, neuroimaging, cognition, attention, network, connectome keywords. The exclusion criteria consisted of studies including children, older adults, bilingual or multilingual population, clinical cases, music, sign language, speech, motor processing, review papers, meta-analyses, electroencephalography/event-related potential, and positron emission tomography. A total of 20 articles were included and discussed in four categories: language comprehension, language production, syntax, and networks. Studies on neural correlates of WM and language interaction are rare. Language tasks that involve WM activate common neural systems. Activated areas can be associated with cognitive concepts proposed by Baddeley and Hitch (1974), including the phonological loop of WM (mainly Broca and Wernicke's areas), other prefrontal cortex and right hemispheric regions linked to the visuospatial sketchpad. There is a clear, dynamic interaction between language and WM, reflected in the involvement of subcortical structures, particularly the basal ganglia (caudate), and of widespread right hemispheric regions. WM involvement is levered by cognitive demand in response to task complexity. High WM capacity readers draw upon buffer memory systems in midline cortical areas to decrease the WM demands for efficiency. Different dynamic networks are involved in WM and language interaction in response to the task in hand for an ultimate brain function efficiency, modulated by language modality and attention.

Insights into visual working memory precision at the feature- and object-level from a hemispheric encoding manipulation

Mnemonic precision is an important aspect of visual working memory (WM). Here, we probed mechanisms that affect precision for spatial (size) and non-spatial (colour) features of an object, and whether these features are encoded and/or stored separately in WM. We probed precision at the feature-level—that is, whether different features of a single object are represented separately or together in WM—and the object-level—that is, whether different features across a set of sequentially presented objects are represented in the same or different WM stores. By manipulating whether stimuli were encoded by the left and/or right hemisphere, we gained further insights into how objects are represented in WM. At the feature-level, we tested whether recall fidelity for the two features of an object fluctuated in tandem from trial to trial. We observed no significant coupling under either central or lateralized encoding, supporting the claim of parallel feature channels at encoding. At the level of WM storage of a set of objects, we found asymmetric feature interference under central encoding, whereby an increase in colour load led to a decrease in size precision. When objects were encoded by a single hemisphere, however, we found largely independent feature stores. Precision for size was more resistant to interference from the size of another object under right-hemisphere encoding; by contrast, precision for colour did not differ across hemispheres, suggesting a more distributed WM store. These findings suggest that distinct features of a single object are represented separately but are then partially integrated during maintenance of a set of sequentially presented objects.

Cognitive aging and verbal labeling in continuous visual memory

The decline of working memory (WM) is a common feature of general cognitive decline, and visual and verbal WM capacity appear to decline at different rates with age. Visual material may be remembered via verbal codes or visual traces, or both. Souza and Skóra, Cognition, 166, 277-297 (2017) found that labeling boosted memory in younger adults by activating categorical visual long-term memory (LTM) knowledge. Here, we replicated this and tested whether it held in healthy older adults. We compared performance in silence, under instructed overt labeling (participants were asked to say color names out loud), and articulatory suppression (repeating irrelevant syllables to prevent labeling) in the delayed estimation paradigm. Overt labeling improved memory performance in both age groups. However, comparing the effect of overt labeling and suppression on the number of coarse, categorical representations in the two age groups suggested that older adults used verbal labels subvocally more than younger adults, when performing the task in silence. Older adults also appeared to benefit from labels differently than younger adults. In younger adults labeling appeared to improve visual, continuous memory, suggesting that labels activated visual LTM representations. However, for older adults, labels did not appear to enhance visual, continuous representations, but instead boosted memory via additional verbal (categorical) memory traces. These results challenged the assumption that visual memory paradigms measure the same cognitive ability in younger and older adults, and highlighted the importance of controlling differences in age-related strategic preferences in visual memory tasks.

Stochastic sampling provides a unifying account of visual working memory limits

Research into human working memory limits has been shaped by the competition between different formal models, with a central point of contention being whether internal representations are continuous or discrete. Here we describe a sampling approach derived from principles of neural coding as a framework to understand working memory limits. Reconceptualizing existing models in these terms reveals strong commonalities between seemingly opposing accounts, but also allows us to identify specific points of difference. We show that the discrete versus continuous nature of sampling is not critical to model fits, but that, instead, random variability in sample counts is the key to reproducing human performance in both single- and whole-report tasks. A probabilistic limit on the number of items successfully retrieved is an emergent property of stochastic sampling, requiring no explicit mechanism to enforce it. These findings resolve discrepancies between previous accounts and establish a unified computational framework for working memory that is compatible with neural principles.

Set size effects on working memory precision are not due to an averaging of slots

Visual working memory is often characterized as a discrete system, where an item is either stored in memory or it is lost completely. As this theory predicts, increasing memory load primarily affects the probability that an item is in memory. However, the precision of items successfully stored in memory also decreases with memory load. The prominent explanation for this effect is the "slots-plus-averaging" model, which proposes that an item can be stored in replicate across multiple memory slots. Here, however, precision declined with set size even in iconic memory tasks that did not require working memory storage, ruling out such storage accounts. Moreover, whereas the slots-plus-averaging model predicts that precision effects should plateau at working memory capacity limits, precision continued to decline well beyond these limits in an iconic memory task, where the number of items available at test was far greater than working memory capacity. Precision also declined in tasks that did not require study items to be encoded simultaneously, ruling out perceptual limitations as the cause of set size effects on memory precision. Taken together, these results imply that set size effects on working memory precision do not stem from working memory storage processes, such as an averaging of slots, and are not due to perceptual limitations. This rejection of the prominent slots-plus-averaging model has implications for how contemporary models of discrete capacities theories can be improved, and how they might be rejected.

Lateralized alpha oscillations are irrelevant for the behavioral retro-cueing benefit in visual working memory

The limited capacity of visual working memory (vWM) necessitates the efficient allocation of available resources by prioritizing relevant over irrelevant items. Retro-cues, which inform about the future relevance of items after encoding has already finished, can improve the quality of memory representations of the relevant items. A candidate mechanism of this retro-cueing benefit is lateralization of neural oscillations in the alpha-band, but its precise role is still debated. The relative decrease of alpha power contralateral to the relevant items has been interpreted as supporting inhibition of irrelevant distractors or as supporting maintenance of relevant items. Here, we aimed at resolving this debate by testing how the magnitude of alpha-band lateralization affects behavioral performance: does stronger lateralization improve the precision of the relevant memory or does it reduce the biasing influence of the irrelevant distractor? We found that it does neither: while the data showed a clear retro-cue benefit and a biasing influence of non-target items as well as clear cue-induced alpha-band lateralization, the magnitude of this lateralization was not correlated with any performance parameter. This finding may indicate that alpha-band lateralization, which is typically observed in response to mnemonic cues, indicates an automatic shift of attention that only coincides with, but is not directly involved in mnemonic prioritization.

Expansion and Contraction Modulate Visual Short-Term Memory

We examined the influence of expansion and contraction on visual short-term memory (VSTM) using a change-detection task. In each trial, several expanding/contracting line segments with various orientations were presented in two successive displays. The orientation of objects in the second display was either identical to, or different from, that of the first display. Observers were asked to judge the presence or absence of an orientation change in successive displays. Results showed that memory performance for expanding objects was higher than for contracting objects (expansion benefit: Experiments 1A and 1B). Further experiments focused on VSTM processing (encoding, storage, and retrieval). Regarding the retrieval stage, an expansion benefit was replicated only when the direction of motion was consistent between two successive displays (Experiment 2A). A cueing stimulus enhanced the memory performance for both expanding and contracting motions and eliminated the expansion benefit (Experiment 2B). Regarding the storage stage, we found the expansion benefit occurred only for shorter blank intervals between the two displays (Experiment 3). Regarding the encoding stage, the expansion benefit was observed regardless of presentation times (Experiment 4). These results indicate the possibility that expanding and contracting motions modulate VSTM.

Humans incorporate trial-to-trial working memory uncertainty into rewarded decisions

Information stored in working memory (WM) is incorporated into many daily decisions and actions, and many complex decisions involve WM; however, there has been little work on investigating what WM information is used in memory decisions. Here we try to draw connections between WM and decision making by manipulating prior beliefs in a standard WM task with rewards. We use this paradigm to show that WM contains a representation of the trial-by-trial uncertainty of visual stimuli. This uncertainty is incorporated into rewarded decisions along with other information, such as expectations about the environment. By studying WM in parallel with decision making, we can gain new insight into how these systems work together.

Working memory (WM) plays an important role in action planning and decision making; however, both the informational content of memory and how that information is used in decisions remain poorly understood. To investigate this, we used a color WM task in which subjects viewed colored stimuli and reported both an estimate of a stimulus color and a measure of memory uncertainty, obtained through a rewarded decision. Reported memory uncertainty is correlated with memory error, showing that people incorporate their trial-to-trial memory quality into rewarded decisions. Moreover, memory uncertainty can be combined with other sources of information; after inducing expectations (prior beliefs) about stimuli probabilities, we found that estimates became shifted toward expected colors, with the shift increasing with reported uncertainty. The data are best fit by models in which people incorporate their trial-to-trial memory uncertainty with potential rewards and prior beliefs. Our results suggest that WM represents uncertainty information, and that this can be combined with prior beliefs. This highlights the potential complexity of WM representations and shows that rewarded decision can be a powerful tool for examining WM and informing and constraining theoretical, computational, and neurobiological models of memory.

Distinguishing target biases and strategic guesses in visual working memory

Previous studies have shown that when there are statistical regularities in the items stored in visual working memory, the responses are biased toward the ensemble average. This statistical-regularity-induced bias could happen in two ways: (1) a target bias, where the individual memory representations are pulled toward the ensemble average; or (2) a strategic guess, for items that are not memorized, other information in the ensemble (e.g., another item) is reported as a substitute. Here, these two mechanisms are distinguished on the basis of a three-part model (target responses + swap responses + random guesses; e.g., Bays, Catalao, & Husain, 2009, Journal of Vision, 9, 7). The strategic guess is operationalized as swap responses, whereas the target bias is reflected by a bias parameter in the target responses. This model was applied on 8 data sets (22 observers each). In this model, contributions of target biases and strategic guesses can be clearly distinguished from each other because they lead to distinctive patterns in the distribution of responses. In the present results, strategic guesses always contributed substantially to the statistical-regularity-induced biases, whereas target biases were limited to specific conditions. All in all, the Bayesian inference in visual working memory is much more limited than what is previously advocated.