Efficient Coding in Visual Working Memory Accounts for Stimulus-Specific Variations in Recall

Color category and inter-item interaction influence color working memory codependently

Our brains do not always encode visual information in a veridical way. Visual working memory (WM) for features such as color can be biased. WM bias comes from several sources. Category priors can lead to WM bias. For example, color WM is biased toward or away from category prototypes. In addition to category knowledge, contextual factors can induce and modulate WM bias; however, these biases of different sources have usually been investigated independently with different tasks. The present study sought to explore how color WM is influenced by both color category and concurrent distractor. Specifically, we asked participants to retain two color items in WM to investigate how the WM representation of the target color is biased by learned category knowledge and contextual inter-item interactions. Our study found that the WM representation of the target color is biased toward or away from the category prototypes and away from the distractor color that is simultaneously held in WM, indicating that both color category and concurrent distractor bias color WM. More importantly, the weight of these two biases depends on the specific color category, suggesting that category priors and inter-item interaction biases are not simply additive but flexible. Furthermore, we revealed that both types of biases arise from perceptual processes.

Dissecting the components of error in analogue report tasks

Over the last two decades, the analogue report task has become a standard method for measuring the fidelity of visual representations across research domains including perception, attention, and memory. Despite its widespread use, there has been no methodical investigation of the different task parameters that might contribute to response variability. To address this gap, we conducted two experiments manipulating components of a typical analogue report test of memory for colour hue. We found that human response errors were independently affected by changes in storage and maintenance requirements of the task, demonstrated by a strong effect of set size even in the absence of a memory delay. In contrast, response variability remained unaffected by physical size of the colour wheel, implying negligible contribution of motor noise to task performance, or by its chroma radius, highlighting non-uniformity of the standard colour space. Comparing analogue report to a matched forced-choice task, we found variation in adjustment criterion made a limited contribution to analogue report variability, becoming meaningful only with low representational noise. Our findings validate the analogue report task as a robust measure of representational fidelity for most purposes, while also quantifying non-representational sources of noise that would limit its reliability in specialized settings.

Repulsion bias is insensitive to spatial attention, yet expands during active working memory maintenance

Our brain sometimes represents visual information in a biased manner. Multiple visual features presented simultaneously or sequentially may interact with each other when we perceive them or maintain them in visual working memory (WM), giving rise to report bias. How goal-directed attention influences target representation is not fully understood, especially concerning whether attention towards distractors modulates report bias for the target. Our study investigated the WM biases of the target when it is concurrent with (1) one attended distractor only, (2) one unattended distractor only, and (3) both kinds of distractors during perception. It was found that the target WM is reported as being repelled away from concurrent distractors, attended or unattended, suggesting attention is not necessary for the occurrence of repulsion bias during perception. Furthermore, goal-directed attention towards the distractors modulates the strength of interitem interaction, and the repulsion bias was found to be stronger when attention was directed toward the distractor than when it was not. However, the exaggerated repulsion associated with the attended distractor is likely due to increased relevance to the memory task and (or) WM load instead of spatial attention. In contrast, spatial attention towards the distractor increases the chances of misreporting the distractor for the target.

Representation and computation in visual working memory

The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have advanced our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event. New experimental tools have revealed that observers possess richer information about the uncertainty in their memories and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visuospatial WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research on human WM from a computational perspective, including the neural mechanisms that support it.

Perceptual similarity judgments do not predict the distribution of errors in working memory

Population coding models provide a quantitative account of visual working memory (VWM) retrieval errors with a plausible link to the response characteristics of sensory neurons. Recent work has provided an important new perspective linking population coding to variables of signal detection, including d-prime, and put forward a new hypothesis: that the distribution of recall errors on, for example, a color wheel, is a consequence of the psychological similarity between points in that stimulus space, such that the exponential-like psychophysical distance scaling function can fulfil the role of population tuning and obviate the need to fit a tuning width parameter to recall data. Using four different visual feature spaces, we measured psychophysical similarity and memory errors in the same participants. Our results revealed strong evidence for a common source of variability affecting similarity judgments and recall estimates but did not support any consistent relationship between psychophysical similarity functions and VWM errors. At the group level, the responsiveness functions obtained from the psychophysical similarity task diverged strongly from those that provided the best fit to working memory errors. At the individual level, we found convincing evidence against an association between observed and best-fitting similarity functions. Finally, our results show that the newly proposed exponential-like responsiveness function has in general no advantage over the canonical von Mises (circular normal) function assumed by previous population coding models. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Geometry of visuospatial working memory information in miniature gaze patterns

Stimulus-dependent eye movements have been recognized as a potential confound in decoding visual working memory information from neural signals. Here we combined eye-tracking with representational geometry analyses to uncover the information in miniature gaze patterns while participants (n = 41) were cued to maintain visual object orientations. Although participants were discouraged from breaking fixation by means of real-time feedback, small gaze shifts (<1°) robustly encoded the to-be-maintained stimulus orientation, with evidence for encoding two sequentially presented orientations at the same time. The orientation encoding on stimulus presentation was object-specific, but it changed to a more object-independent format during cued maintenance, particularly when attention had been temporarily withdrawn from the memorandum. Finally, categorical reporting biases increased after unattended storage, with indications of biased gaze geometries already emerging during the maintenance periods before behavioural reporting. These findings disclose a wealth of information in gaze patterns during visuospatial working memory and indicate systematic changes in representational format when memory contents have been unattended.

The influence of natural image statistics on upright orientation judgements

Humans have well-documented priors for many features present in nature that guide visual perception. Despite being putatively grounded in the statistical regularities of the environment, scene priors are frequently violated due to the inherent variability of visual features from one scene to the next. However, these repeated violations do not appreciably challenge visuo-cognitive function, necessitating the broad use of priors in conjunction with context-specific information. We investigated the trade-off between participants' internal expectations formed from both longer-term priors and those formed from immediate contextual information using a perceptual inference task and naturalistic stimuli. Notably, our task required participants to make perceptual inferences about naturalistic images using their own internal criteria, rather than making comparative judgements. Nonetheless, we show that observers' performance is well approximated by a model that makes inferences using a prior for low-level image statistics, aggregated over many images. We further show that the dependence on this prior is rapidly re-weighted against contextual information, even when misleading. Our results therefore provide insight into how apparent high-level interpretations of scene appearances follow from the most basic of perceptual processes, which are grounded in the statistics of natural images.

Learning efficient representations of environmental priors in working memory

Experience shapes our expectations and helps us learn the structure of the environment. Inference models render such learning as a gradual refinement of the observer's estimate of the environmental prior. For instance, when retaining an estimate of an object's features in working memory, learned priors may bias the estimate in the direction of common feature values. Humans display such biases when retaining color estimates on short time intervals. We propose that these systematic biases emerge from modulation of synaptic connectivity in a neural circuit based on the experienced stimulus history, shaping the persistent and collective neural activity that encodes the stimulus estimate. Resulting neural activity attractors are aligned to common stimulus values. Using recently published human response data from a delayed-estimation task in which stimuli (colors) were drawn from a heterogeneous distribution that did not necessarily correspond with reported population biases, we confirm that most subjects' response distributions are better described by experience-dependent learning models than by models with fixed biases. This work suggests systematic limitations in working memory reflect efficient representations of inferred environmental structure, providing new insights into how humans integrate environmental knowledge into their cognitive strategies.

Neural tuning instantiates prior expectations in the human visual system

Perception is often modelled as a process of active inference, whereby prior expectations are combined with noisy sensory measurements to estimate the structure of the world. This mathematical framework has proven critical to understanding perception, cognition, motor control, and social interaction. While theoretical work has shown how priors can be computed from environmental statistics, their neural instantiation could be realised through multiple competing encoding schemes. Using a data-driven approach, here we extract the brain's representation of visual orientation and compare this with simulations from different sensory coding schemes. We found that the tuning of the human visual system is highly conditional on stimulus-specific variations in a way that is not predicted by previous proposals. We further show that the adopted encoding scheme effectively embeds an environmental prior for natural image statistics within the sensory measurement, providing the functional architecture necessary for optimal inference in the earliest stages of cortical processing.

A spatially continuous diffusion model of visual working memory

We present results from five visual working memory (VWM) experiments in which participants were briefly shown between 2 and 6 colored squares. They were then cued to recall the color of one of the squares and they responded by choosing the color on a continuous color wheel. The experiments provided response proportions and response time (RT) measures as a function of angle for the choices. Current VWM models for this task include discrete models that assume an item is either within working memory or not and resource models that assume that memory strength varies as a function of the number of items. Because these models do not include processes that allow them to account for RT data, we implemented them within the spatially continuous diffusion model (SCDM, Ratcliff, 2018) and use the experimental data to evaluate these combined models. In the SCDM, evidence retrieved from memory is represented as a spatially continuous normal distribution and this drives the decision process until a criterion (represented as a 1-D line) is reached, which produces a decision. Noise in the accumulation process is represented by continuous Gaussian process noise over spatial position. The models that fit best from the discrete and resource-based classes converged on a common model that had a guessing component and that allowed the height of the normal memory-strength distribution to vary with number of items. The guessing component was implemented as a regular decision process driven by a flat evidence distribution, a zero-drift process. The combination of choice and RT data allows models that were not identifiable based on choice data alone to be discriminated.

Levels of orientation bias differ across digital content categories: Implications for visual perception

With the continued growth of digital device use, a greater portion of the visual world experienced daily by many people has shifted towards digital environments. The "oblique effect" denotes a bias for horizontal and vertical (canonical) contours over oblique contours, which is derived from a disproportionate exposure to canonical content. Carpentered environments have been shown to possess proportionally more canonical than oblique contours, leading to perceptual bias in those who live in "built" environments. Likewise, there is potential for orientation sensitivity to be shaped by frequent exposure to digital content. The potential influence of digital content on the oblique effect was investigated by measuring the degree of orientation anisotropy from a range of digital scenes using Fourier analysis. Content from popular cartoons, video games, and social communication websites was compared to real-life nature, suburban, and urban scenes. Findings suggest that digital content varies widely in orientation anisotropy, but pixelated video games and social communication websites were found to exhibit a degree of orientation anisotropy substantially exceeding that observed in all measured categories of real-world environments. Therefore, the potential may exist for digital content to induce an even greater shift in orientation bias than has been observed in previous research. This potential, and implications of such a shift, is discussed.

Working memory is updated by reallocation of resources from obsolete to new items

Visual working memory (VWM) resources are limited, placing constraints on how much visual information can be simultaneously retained. During visually guided activity, stored information can quickly become outdated, so updating mechanisms are needed to ensure the contents of memory remain relevant to current task goals. In particular, successful deallocation of resources from items that become obsolete is likely to be critical for maintaining the precision of those representations still in memory. The experiments in this study involved presenting two memory arrays of coloured disks in sequence. The appearance of the second array was a cue to replace, rehearse, or add a new colour to the colours in memory. We predicted that successful resource reallocation should result in comparable recall precision when an item was replaced or rehearsed, owing to the removal of pre-replacement features. In contrast, a failure to update WM should lead to comparable precision with a condition in which a new colour was added to memory. We identified a very small proportion (∼5%) of trials in which participants incorrectly reported a feature from the first array in place of its replacement in the second, which we interpreted as a failure to incorporate the information from the second display into memory. Once these trials were discounted, precision estimates were consistent with complete redistribution of resources in the case of updating a single item. We conclude that working memory can be efficiently updated when previous information becomes obsolete, but that this is a demanding active process that occasionally fails.

Prior Expectations in Visual Speed Perception Predict Encoding Characteristics of Neurons in Area MT

Bayesian inference provides an elegant theoretical framework for understanding the characteristic biases and discrimination thresholds in visual speed perception. However, the framework is difficult to validate because of its flexibility and the fact that suitable constraints on the structure of the sensory uncertainty have been missing. Here, we demonstrate that a Bayesian observer model constrained by efficient coding not only well explains human visual speed perception but also provides an accurate quantitative account of the tuning characteristics of neurons known for representing visual speed. Specifically, we found that the population coding accuracy for visual speed in area MT ("neural prior") is precisely predicted by the power-law, slow-speed prior extracted from fitting the Bayesian observer model to psychophysical data ("behavioral prior") to the point that the two priors are indistinguishable in a cross-validation model comparison. Our results demonstrate a quantitative validation of the Bayesian observer model constrained by efficient coding at both the behavioral and neural levels.SIGNIFICANCE STATEMENT Statistical regularities of the environment play an important role in shaping both neural representations and perceptual behavior. Most previous work addressed these two aspects independently. Here we present a quantitative validation of a theoretical framework that makes joint predictions for neural coding and behavior, based on the assumption that neural representations of sensory information are efficient but also optimally used in generating a percept. Specifically, we demonstrate that the neural tuning characteristics for visual speed in brain area MT are precisely predicted by the statistical prior expectations extracted from psychophysical data. As such, our results provide a normative link between perceptual behavior and the neural representation of sensory information in the brain.

Luminance and Contrast of Images in the THINGS Database

The THINGS database is a freely available stimulus set that has the potential to facilitate the generation of theory that bridges multiple areas within cognitive neuroscience. The database consists of 26,107 high quality digital photos that are sorted into 1,854 concepts. While a valuable resource, relatively few technical details relevant to the design of studies in cognitive neuroscience have been described. We present an analysis of two key low-level properties of THINGS images, luminance and luminance contrast. These image statistics are known to influence common physiological and neural correlates of perceptual and cognitive processes. In general, we found that the distributions of luminance and contrast are in close agreement with the statistics of natural images reported previously. However, we found that image concepts are separable in their luminance and contrast: we show that luminance and contrast alone are sufficient to classify images into their concepts with above chance accuracy. We describe how these factors may confound studies using the THINGS images, and suggest simple controls that can be implemented a priori or post-hoc. We discuss the importance of using such natural images as stimuli in psychological research.

Effects of Attention Direction and Perceptual Distraction Within Visual Working Memory

Although substantial evidence demonstrates that directing attention to specific items is important for improving the performance of visual working memory (VWM), it is still not clear whether the attended items were better protected. The present study, thus, adopted a pre-cueing paradigm to examine the effect of attention direction and perceptual distractor on VWM. The results showed that a valid visual cue improved the individuals’ VWM performances and reduced their reaction time compared to the invalid and neutral cues. However, the VWM performances in the valid and neutral cue conditions were more disrupted by a post-stimuli distractor compared to the invalid cue condition. The findings suggest that although directing attention can improve the VWM performance, it is not efficient in protecting information from being distracted.

More Prominent Nonlinear Mixed Selectivity in the Dorsolateral Prefrontal than Posterior Parietal Cortex

Neurons in the dorsolateral prefrontal cortex (dlPFC) and posterior parietal cortex (PPC) are activated by different cognitive tasks and respond differently to the same stimuli depending on task. The conjunctive representations of multiple tasks in nonlinear fashion in single neuron activity, is known as nonlinear mixed selectivity (NMS). Here, we compared NMS in a working memory task in areas 8a and 46 of the dlPFC and 7a and lateral intraparietal cortex (LIP) of the PPC in macaque monkeys. NMS neurons were more frequent in dlPFC than in PPC and this was attributed to more cells gaining selectivity in the course of a trial. Additionally, in our task, the subjects' behavioral performance improved within a behavioral session as they learned the session-specific statistics of the task. The magnitude of NMS in the dlPFC also increased as a function of time within a single session. On the other hand, we observed minimal rotation of population responses and no appreciable differences in NMS between correct and error trials in either area. Our results provide direct evidence demonstrating a specialization in NMS between dlPFC and PPC and reveal mechanisms of neural selectivity in areas recruited in working memory tasks.

Visual working memory in aphantasia: Retained accuracy and capacity with a different strategy

Visual working memory paradigms involve retaining and manipulating visual information in mind over a period of seconds. Evidence suggests that visual imagery (sensory recruitment) is a strategy used by many to retain visual information during such tasks, leading some researchers to propose that visual imagery and visual working memory may be one and the same. If visual imagery is essential to visual working memory task performance there should be large ramifications for a special population of individuals who do not experience visual imagery, aphantasia. Here we assessed visual working memory task performance in this population using a number of different lab and clinical working memory tasks. We found no differences in capacity limits for visual, general number or spatial working memory for aphantasic individuals compared to controls. Further, aphantasic individuals showed no significant differences in performance on visual components of clinical working memory tests as compared to verbal components. However, there were significant differences in the reported strategies used by aphantasic individuals across all memory tasks. Additionally, aphantasic individual's visual memory accuracy did not demonstrate a significant oblique orientation effect, which is proposed to occur due to sensory recruitment, further supporting their non-visual imagery strategy reports. Taken together these data demonstrate that aphantasic individuals are not impaired on visual working memory tasks, suggesting visual imagery and working memory are not one and the same, with imagery (and sensory recruitment) being just one of the tools that can be used to solve visual working memory tasks.

Perceptual History Acts in World-Centred Coordinates

Serial dependence effects have been observed using a variety of stimuli and tasks, revealing that the recent past can bias current percepts, leading to increased similarity between two. The aim of this study is to determine whether this temporal integration occurs in egocentric or allocentric coordinates. We asked participants to perform an orientation reproduction task using grating stimuli while the head was kept at a fixed position, or after a 40° yaw rotation between trials, from left (-20°) to right (+20°), putting the egocentric and allocentric cues in conflict. Under these conditions, allocentric cues prevailed.

Conscious perception can be both graded and discrete

The attentional blink (AB) paradigm has been used to address an enduring debate about the nature of conscious perception: Does the temporary impairment in conscious perception of the second (T2) of two serially presented targets result from a probabilistic all-or-none loss of information, or does T2 transition into consciousness along a continuum of perceptual fidelity? To investigate this question, we presented noisy orientation patterns as targets embedded in a rapid serial sequence of nonoriented noise distractors, and evaluated perception of T2 orientation using a continuous report paradigm. Using discrete mixture models and variable resource models, we evaluated the effects of manipulating both perceptual and central demands on the precision of T2 responses and the estimated frequency of random guessing. When perceptual competition between targets was emphasized by their sharing of a common visual feature (i.e., orientation), the attentional blink was associated with degraded precision of T2 perception. By contrast, when the task required switching between different attended features across two visually distinct targets, T2 awareness was impaired in an all-or-none manner as evidenced by significant increases in guessing responses. Both statistical and model comparison analyses indicated that loss of target information can be graded or discrete, depending on whether perceptual or higher central stages are taxed by processing demands. Our findings provide new insights into the mechanisms underlying the attentional blink and help reconcile conflicting views regarding how information can be lost from awareness. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Neural evidence for categorical biases in location and orientation representations in a working memory task

Previous research demonstrated that visual representations in working memory exhibit biases with respect to the categorical structure of the stimulus space. However, a majority of those studies used behavioral measures of working memory, and it is not clear whether the working memory representations per se are influenced by the categorical structure or whether the biases arise in decision or response processes during the report. Here, I applied a multivariate decoding technique to EEG data collected during working memory tasks to determine whether neural activity associated with the representations in working memory is categorically biased prior to the report. I found that the decoding of spatial working memory was biased away from the nearest cardinal location, consistent with the biases observed in the behavioral responses. In a follow-up experiment which was designed to prevent the use of a response preparation strategy, I found that the decoding still exhibited categorical biases. Together, these results provide neural evidence that working memory representations themselves are categorically biased, imposing important constraints on the models of working memory representations.

Balancing Flexibility and Interference in Working Memory

Working memory is central to cognition, flexibly holding the variety of thoughts needed for complex behavior. Yet, despite its importance, working memory has a severely limited capacity, holding only three to four items at once. In this article, I review experimental and computational evidence that the flexibility and limited capacity of working memory reflect the same underlying neural mechanism. I argue that working memory relies on interactions between high-dimensional, integrative representations in the prefrontal cortex and structured representations in the sensory cortex. Together, these interactions allow working memory to flexibly maintain arbitrary representations. However, the distributed nature of working memory comes at the cost of causing interference between items in memory, resulting in a limited capacity. Finally, I discuss several mechanisms used by the brain to reduce interference and maximize the effective capacity of working memory. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Delay-period activity in frontal, parietal, and occipital cortex tracks noise and biases in visual working memory

Working memory is imprecise, and these imprecisions can be explained by the combined influences of random diffusive error and systematic drift toward a set of stable states ("attractors"). However, the neural correlates of diffusion and drift remain unknown. Here, we investigated how delay-period activity in frontal and parietal cortex, which is known to correlate with the decline in behavioral memory precision observed with increasing memory load, might relate to diffusion and drift. We analyzed data from an existing experiment in which subjects performed delayed recall for line orientation, at different loads, during functional magnetic resonance imaging (fMRI) scanning. To quantify the influence of drift and diffusion, we modeled subjects' behavior using a discrete attractor model and calculated within-subject correlation between frontal and parietal delay-period activity and whole-trial estimates of drift and diffusion. We found that although increases in frontal and parietal activity were associated with increases in both diffusion and drift, diffusion explained the most variance in frontal and parietal delay-period activity. In comparison, a subsequent whole-brain regression analysis showed that drift, rather than diffusion, explained the most variance in delay-period activity in lateral occipital cortex. These results are consistent with a model of the differential recruitment of general frontoparietal mechanisms in response to diffusive noise and of stimulus-specific biases in occipital cortex.

Stochastic sampling provides a unifying account of visual working memory limits

Research into human working memory limits has been shaped by the competition between different formal models, with a central point of contention being whether internal representations are continuous or discrete. Here we describe a sampling approach derived from principles of neural coding as a framework to understand working memory limits. Reconceptualizing existing models in these terms reveals strong commonalities between seemingly opposing accounts, but also allows us to identify specific points of difference. We show that the discrete versus continuous nature of sampling is not critical to model fits, but that, instead, random variability in sample counts is the key to reproducing human performance in both single- and whole-report tasks. A probabilistic limit on the number of items successfully retrieved is an emergent property of stochastic sampling, requiring no explicit mechanism to enforce it. These findings resolve discrepancies between previous accounts and establish a unified computational framework for working memory that is compatible with neural principles.

Set size effects on working memory precision are not due to an averaging of slots

Visual working memory is often characterized as a discrete system, where an item is either stored in memory or it is lost completely. As this theory predicts, increasing memory load primarily affects the probability that an item is in memory. However, the precision of items successfully stored in memory also decreases with memory load. The prominent explanation for this effect is the "slots-plus-averaging" model, which proposes that an item can be stored in replicate across multiple memory slots. Here, however, precision declined with set size even in iconic memory tasks that did not require working memory storage, ruling out such storage accounts. Moreover, whereas the slots-plus-averaging model predicts that precision effects should plateau at working memory capacity limits, precision continued to decline well beyond these limits in an iconic memory task, where the number of items available at test was far greater than working memory capacity. Precision also declined in tasks that did not require study items to be encoded simultaneously, ruling out perceptual limitations as the cause of set size effects on memory precision. Taken together, these results imply that set size effects on working memory precision do not stem from working memory storage processes, such as an averaging of slots, and are not due to perceptual limitations. This rejection of the prominent slots-plus-averaging model has implications for how contemporary models of discrete capacities theories can be improved, and how they might be rejected.

Theory of neural coding predicts an upper bound on estimates of memory variability

Observers reproducing elementary visual features from memory after a short delay produce errors consistent with the encoding-decoding properties of neural populations. While inspired by electrophysiological observations of sensory neurons in cortex, the population coding account of these errors is based on a mathematical idealization of neural response functions that abstracts away most of the heterogeneity and complexity of real neuronal populations. Here we examine a more physiologically grounded model based on the tuning of a large set of neurons recorded in macaque V1 and show that key predictions of the idealized model are preserved. Both models predict long-tailed distributions of error when memory resources are taxed, as observed empirically in behavioral experiments and commonly approximated with a mixture of normal and uniform error components. Specifically, for an idealized homogeneous neural population, the width of the fitted normal distribution cannot exceed the average tuning width of the component neurons, and this also holds to a good approximation for more biologically realistic populations. Examining eight published studies of orientation recall, we find a consistent pattern of results suggestive of a median tuning width of approximately 20°, which compares well with neurophysiological observations. The finding that estimates of variability obtained by the normal-plus-uniform mixture method are bounded from above leads us to reevaluate previous studies that interpreted a saturation in width of the normal component as evidence for fundamental limits on the precision of perception, working memory, and long-term memory.

Recall of facial expressions and simple orientations reveals competition for resources at multiple levels of the visual hierarchy

Many studies of visual working memory have tested humans' ability to reproduce primary visual features of simple objects, such as the orientation of a grating or the hue of a color patch, following a delay. A consistent finding of such studies is that precision of responses declines as the number of items in memory increases. Here we compared visual working memory for primary features and high-level objects. We presented participants with memory arrays consisting of oriented gratings, facial expressions, or a mixture of both. Precision of reproduction for all facial expressions declined steadily as the memory load was increased from one to five faces. For primary features, this decline and the specific distributions of error observed, have been parsimoniously explained in terms of neural population codes. We adapted the population coding model for circular variables to the non-circular and bounded parameter space used for expression estimation. Total population activity was held constant according to the principle of normalization and the intensity of expression was decoded by drawing samples from the Bayesian posterior distribution. The model fit the data well, showing that principles of population coding can be applied to model memory representations at multiple levels of the visual hierarchy. When both gratings and faces had to be remembered, an asymmetry was observed. Increasing the number of faces decreased precision of orientation recall, but increasing the number of gratings did not affect recall of expression, suggesting that memorizing faces involves the automatic encoding of low-level features, in addition to higher-level expression information.