Sudden death and gradual decay in visual working memory

The relationship between visual working memory and attention: retention of precise colour information in the absence of effects on perceptual selection

We examined the conditions under which a feature value in visual working memory (VWM) recruits visual attention to matching stimuli. Previous work has suggested that VWM supports two qualitatively different states of representation: an active state that interacts with perceptual selection and a passive (or accessory) state that does not. An alternative hypothesis is that VWM supports a single form of representation, with the precision of feature memory controlling whether or not the representation interacts with perceptual selection. The results of three experiments supported the dual-state hypothesis. We established conditions under which participants retained a relatively precise representation of a parcticular colour. If the colour was immediately task relevant, it reliably recruited attention to matching stimuli. However, if the colour was not immediately task relevant, it failed to interact with perceptual selection. Feature maintenance in VWM is not necessarily equivalent with feature-based attentional selection.

Source accuracy data reveal the thresholded nature of human episodic memory

Episodic recollection supports conscious retrieval of past events. It is unknown why recollected memories are often vivid, but at other times we struggle to remember. Such experiences might reflect a recollection threshold: Either the threshold is exceeded and information is retrieved, or recollection fails completely. Alternatively, retrieval failure could reflect weak memory: Recollection could behave as a continuous signal, always yielding some variable degree of information. Here we reconcile these views, using a novel source memory task that measures retrieval accuracy directly. We show that recollection is thresholded, such that retrieval sometimes simply fails. Our technique clarifies a fundamental property of memory and allows responses to be accurately measured, without recourse to subjective introspection. These findings raise new questions about how successful retrieval is determined and why it declines with age and disease.

Working memory management and predicted utility

Given the limited capacity of working memory (WM), its resources should be allocated strategically. One strategy is filtering, whereby access to WM is granted preferentially to items with the greatest utility. However, reallocation of WM resources might be required if the utility of maintained information subsequently declines. Here, we present behavioral, computational, and neuroimaging evidence that human participants track changes in the predicted utility of information in WM. First, participants demonstrated behavioral costs when the utility of items already maintained in WM declined and resources should be reallocated. An adapted Q-learning model indicated that these costs scaled with the historical utility of individual items. Finally, model-based neuroimaging demonstrated that frontal cortex tracked the utility of items to be maintained in WM, whereas ventral striatum tracked changes in the utility of items maintained in WM to the degree that these items are no longer useful. Our findings suggest that frontostriatal mechanisms track the utility of information in WM, and that these dynamics may predict delays in the removal of information from WM.

Task probability and report of feature information: what you know about what you 'see' depends on what you expect to need

We investigated the influence of dimensional set on report of object feature information using an immediate memory probe task. Participants viewed displays containing up to 36 coloured geometric shapes which were presented for several hundred milliseconds before one item was abruptly occluded by a probe. A cue presented simultaneously with the probe instructed participants to report either about the colour or shape of the probe item. A dimensional set towards the colour or shape of the presented items was induced by manipulating task probability - the relative probability with which the two feature dimensions required report. This was done across two participant groups: One group was given trials where there was a higher report probability of colour, the other a higher report probability of shape. Two experiments showed that features were reported most accurately when they were of high task probability, though in both cases the effect was largely driven by the colour dimension. Importantly the task probability effect did not interact with display set size. This is interpreted as tentative evidence that this manipulation influences feature processing in a global manner and at a stage prior to visual short term memory.

Sleep deprivation accelerates delay-related loss of visual short-term memories without affecting precision

STUDY OBJECTIVES

Visual short-term memory (VSTM) is an important measure of information processing capacity and supports many higher-order cognitive processes. We examined how sleep deprivation (SD) and maintenance duration interact to influence the number and precision of items in VSTM using an experimental design that limits the contribution of lapses at encoding.

DESIGN

For each trial, participants attempted to maintain the location and color of three stimuli over a delay. After a retention interval of either 1 or 10 seconds, participants reported the color of the item at the cued location by selecting it on a color wheel. The probability of reporting the probed item, the precision of report, and the probability of reporting a nonprobed item were determined using a mixture-modeling analysis. Participants were studied twice in counterbalanced order, once after a night of normal sleep and once following a night of sleep deprivation.

SETTING

Sleep laboratory.

PARTICIPANTS

Nineteen healthy college age volunteers (seven females) with regular sleep patterns.

INTERVENTIONS

Approximately 24 hours of total SD.

MEASUREMENTS AND RESULTS

SD selectively reduced the number of integrated representations that can be retrieved after a delay, while leaving the precision of object information in the stored representations intact. Delay interacted with SD to lower the rate of successful recall.

CONCLUSIONS

Visual short-term memory is compromised during sleep deprivation, an effect compounded by delay. However, when memories are retrieved, they tend to be intact.

Visual Long-Term Memory Has the Same Limit on Fidelity as Visual Working Memory

Visual long-term memory can store thousands of objects with surprising visual detail, but just how detailed are these representations, and how can one quantify this fidelity? Using the property of color as a case study, we estimated the precision of visual information in long-term memory, and compared this with the precision of the same information in working memory. Observers were shown real-world objects in random colors and were asked to recall the colors after a delay. We quantified two parameters of performance: the variability of internal representations of color (fidelity) and the probability of forgetting an object’s color altogether. Surprisingly, the fidelity of color information in long-term memory was comparable to the asymptotic precision of working memory. These results suggest that long-term memory and working memory may be constrained by a common limit, such as a bound on the fidelity required to retrieve a memory representation.

The benefit of forgetting

Recent research using change-detection tasks has shown that a directed-forgetting cue, indicating that a subset of the information stored in memory can be forgotten, significantly benefits the other information stored in visual working memory. How do these directed-forgetting cues aid the memory representations that are retained? We addressed this question in the present study by using a recall paradigm to measure the nature of the retained memory representations. Our results demonstrated that a directed-forgetting cue leads to higher-fidelity representations of the remaining items and a lower probability of dropping these representations from memory. Next, we showed that this is made possible by the to-be-forgotten item being expelled from visual working memory following the cue, allowing maintenance mechanisms to be focused on only the items that remain in visual working memory. Thus, the present findings show that cues to forget benefit the remaining information in visual working memory by fundamentally improving their quality relative to conditions in which just as many items are encoded but no cue is provided.

Visual working memory

Visual working memory (VWM), the system of storing, manipulating, and utilizing, visual information is fundamental to many cognitive acts. Exploring the limitations of this system is essential to understand the characteristics of higher-order cognition, since at a basic level, VWM is the interface through which we interact with our environment. Given its important function, this system has become a very active area of research in the recent years. Here, we examine current models of VWM, along with the proposed reasons for what limits its capacity. This is followed by a short description of recent neural findings that have helped constrain models of VWM. In closing, we focus on work exploring individual differences in working memory capacity, and what these findings reveal about the intimate relationship between VWM and attention. WIREs Cogn Sci 2013, 4:179-190. doi: 10.1002/wcs.1219 For further resources related to this article, please visit the WIREs website.

Attention restores discrete items to visual short-term memory

When a memory is forgotten, is it lost forever? Our study shows that selective attention can restore forgotten items to visual short-term memory (VSTM). In our two experiments, all stimuli presented in a memory array were designed to be equally task relevant during encoding. During the retention interval, however, participants were sometimes given a cue predicting which of the memory items would be probed at the end of the delay. This shift in task relevance improved recall for that item. We found that this type of cuing improved recall for items that otherwise would have been irretrievable, providing critical evidence that attention can restore forgotten information to VSTM. Psychophysical modeling of memory performance has confirmed that restoration of information in VSTM increases the probability that the cued item is available for recall but does not improve the representational quality of the memory. We further suggest that attention can restore discrete items to VSTM.

Working memory inefficiency: minimal information is utilized in visual recognition tasks

Can people make perfect use of task-relevant information in working memory (WM)? Specifically, when questioned about an item in an array that does not happen to be in WM, can participants take into account other items that are in WM, eliminating them as response candidates? To address this question, an ideal-responder model that assumes perfect use of items in a capacity-limited WM was tested against a minimal-responder model that assumes use of only information about the queried item. Three different WM tasks were adopted: change detection, identity recognition, and location recognition. The change-detection task produced benchmark WM results. The 2 novel tasks showed that only the minimal responder model provided convergence with this benchmark. This finding was replicable even when the change-detection task was replaced by a feature-switch detection task. Thus, it appears that people do not make full use of information in WM.

Obligatory encoding of task-irrelevant features depletes working memory resources

Selective attention is often considered the "gateway" to visual working memory (VWM). However, the extent to which we can voluntarily control which of an object's features enter memory remains subject to debate. Recent research has converged on the concept of VWM as a limited commodity distributed between elements of a visual scene. Consequently, as memory load increases, the fidelity with which each visual feature is stored decreases. Here we used changes in recall precision to probe whether task-irrelevant features were encoded into VWM when individuals were asked to store specific feature dimensions. Recall precision for both color and orientation was significantly enhanced when task-irrelevant features were removed, but knowledge of which features would be probed provided no advantage over having to memorize both features of all items. Next, we assessed the effect an interpolated orientation-or color-matching task had on the resolution with which orientations in a memory array were stored. We found that the presence of orientation information in the second array disrupted memory of the first array. The cost to recall precision was identical whether the interfering features had to be remembered, attended to, or could be ignored. Therefore, it appears that storing, or merely attending to, one feature of an object is sufficient to promote automatic encoding of all its features, depleting VWM resources. However, the precision cost was abolished when the match task preceded the memory array. So, while encoding is automatic, maintenance is voluntary, allowing resources to be reallocated to store new visual information.

The precision of visual memory for a complex contour shape measured by a freehand drawing task

Contour information is an important source for object perception and memory. Three experiments examined the precision of visual short-term memory for complex contour shapes. All used a new procedure that assessed recall memory for holistic information in complex contour shapes: Participants studied, then reproduced (without cues), a contoured shape by freehand drawing. In Experiment 1 memory precision was measured by comparing Fourier descriptors for studied and reproduced contours. Results indicated survival of lower (holistic) frequency information (i.e., ⩽5cycles/perimeter) and loss of higher (detail) frequency information. Secondary tasks placed demands on either verbal memory (Experiment 2) or visual spatial memory (Experiment 3). Neither secondary task interfered with recall of complex contour shapes, suggesting that the memory system maintaining holistic shape information was independent of both the verbal memory system and the visual spatial memory subsystem of visual short-term memory. The nature of memory for complex contour shape is discussed.

Introspective judgments predict the precision and likelihood of successful maintenance of visual working memory

Working memory serves as an essential workspace for the mind, allowing for the active maintenance of information to support short-term cognitive goals. Although people can readily report the contents of working memory, it is unknown whether they might have reliable metacognitive knowledge regarding the accuracy of their own memories. We investigated this question to better understand the core properties of the visual working memory system. Observers were briefly presented with displays of three or six oriented gratings, after which they were cued to report the orientation of a specific grating from memory as well as their subjective confidence in their memory. We used a mixed-model approach to obtain separate estimates of the probability of successful memory maintenance and the precision of memory for successfully remembered items. Confidence ratings strongly predicted the likelihood that the cued grating was successfully maintained, and furthermore revealed trial-to-trial variations in the visual precision of memory itself. Our findings provide novel evidence indicating that the precision of visual working memory is variable in nature. These results inform an ongoing debate regarding whether this working memory system relies on discrete slots with fixed visual resolution or on representations with variable precision, as might arise from variability in the amount of resources assigned to individual items on each trial.

Rapid forgetting prevented by retrospective attention cues

Recent studies have demonstrated that memory performance can be enhanced by a cue which indicates the item most likely to be subsequently probed, even when that cue is delivered seconds after a stimulus array is extinguished. Although such retro-cuing has attracted considerable interest, the mechanisms underlying it remain unclear. Here, we tested the hypothesis that retro-cues might protect an item from degradation over time. We employed two techniques that previously have not been deployed in retro-cuing tasks. First, we used a sensitive, continuous scale for reporting the orientation of a memorized item, rather than binary measures (change or no change) typically used in previous studies. Second, to investigate the stability of memory across time, we also systematically varied the duration between the retro-cue and report. Although accuracy of reporting uncued objects rapidly declined over short intervals, retro-cued items were significantly more stable, showing negligible decline in accuracy across time and protection from forgetting. Retro-cuing an object’s color was just as advantageous as spatial retro-cues. These findings demonstrate that during maintenance, even when items are no longer visible, attention resources can be selectively redeployed to protect the accuracy with which a cued item can be recalled over time, but with a corresponding cost in recall for uncued items.

Forgetting what was where: the fragility of object-location binding

Although we frequently take advantage of memory for objects locations in everyday life, understanding how an object’s identity is bound correctly to its location remains unclear. Here we examine how information about object identity, location and crucially object-location associations are differentially susceptible to forgetting, over variable retention intervals and memory load. In our task, participants relocated objects to their remembered locations using a touchscreen. When participants mislocalized objects, their reports were clustered around the locations of other objects in the array, rather than occurring randomly. These ‘swap’ errors could not be attributed to simple failure to remember either the identity or location of the objects, but rather appeared to arise from failure to bind object identity and location in memory. Moreover, such binding failures significantly contributed to decline in localization performance over retention time. We conclude that when objects are forgotten they do not disappear completely from memory, but rather it is the links between identity and location that are prone to be broken over time.

Are multiple visual short-term memory storages necessary to explain the retro-cue effect?

Recent research has shown that change detection performance is enhanced when, during the retention interval, attention is cued to the location of the upcoming test item. This retro-cue advantage has led some researchers to suggest that visual short-term memory (VSTM) is divided into a durable, limited-capacity storage and a more fragile, high-capacity storage. Consequently, performance is poor on the no-cue trials because fragile VSTM is overwritten by the test display and only durable VSTM is accessible under these conditions. In contrast, performance is improved in the retro-cue condition because attention keeps fragile VSTM accessible. The aim of the present study was to test the assumptions underlying this two-storage account. Participants were asked to encode an array of colors for a change detection task involving no-cue and retro-cue trials. A retro-cue advantage was found even when the cue was presented after a visual (Experiment 1) or a central (Experiment 2) interference. Furthermore, the magnitude of the interference was comparable between the no-cue and retro-cue trials. These data undermine the main empirical support for the two-storage account and suggest that the presence of a retro-cue benefit cannot be used to differentiate between different VSTM storages.

An ideal observer analysis of visual working memory

Limits in visual working memory (VWM) strongly constrain human performance across many tasks. However, the nature of these limits is not well understood. In this article we develop an ideal observer analysis of human VWM by deriving the expected behavior of an optimally performing but limited-capacity memory system. This analysis is framed around rate-distortion theory, a branch of information theory that provides optimal bounds on the accuracy of information transmission subject to a fixed information capacity. The result of the ideal observer analysis is a theoretical framework that provides a task-independent and quantitative definition of visual memory capacity and yields novel predictions regarding human performance. These predictions are subsequently evaluated and confirmed in 2 empirical studies. Further, the framework is general enough to allow the specification and testing of alternative models of visual memory (e.g., how capacity is distributed across multiple items). We demonstrate that a simple model developed on the basis of the ideal observer analysis-one that allows variability in the number of stored memory representations but does not assume the presence of a fixed item limit-provides an excellent account of the empirical data and further offers a principled reinterpretation of existing models of VWM.

Directed forgetting and directed remembering in visual working memory

A defining characteristic of visual working memory is its limited capacity. This means that it is crucial to maintain only the most relevant information in visual working memory. However, empirical research is mixed as to whether it is possible to selectively maintain a subset of the information previously encoded into visual working memory. Here we examined the ability of participants to use cues to either forget or remember a subset of the information already stored in visual working memory. In Experiment 1, participants were cued to either forget or remember 1 of 2 groups of colored squares during a change-detection task. We found that both types of cues aided performance in the visual working memory task but that observers benefited more from a cue to remember than a cue to forget a subset of the objects. In Experiment 2, we show that the previous findings, which indicated that directed-forgetting cues are ineffective, were likely due to the presence of invalid cues that appeared to cause observers to disregard such cues as unreliable. In Experiment 3, we recorded event-related potentials and show that an electrophysiological index of focused maintenance is elicited by cues that indicate which subset of information in visual working memory needs to be remembered, ruling out alternative explanations of the behavioral effects of retention-interval cues. The present findings demonstrate that observers can focus maintenance mechanisms on specific objects in visual working memory based on cues indicating future task relevance.

From distributed resources to limited slots in multiple-item working memory: a spiking network model with normalization

Recent behavioral studies have given rise to two contrasting models for limited working memory capacity: a "discrete-slot" model in which memory items are stored in a limited number of slots, and a "shared-resource" model in which the neural representation of items is distributed across a limited pool of resources. To elucidate the underlying neural processes, we investigated a continuous network model for working memory of an analog feature. Our model network fundamentally operates with a shared resource mechanism, and stimuli in cue arrays are encoded by a distributed neural population. On the other hand, the network dynamics and performance are also consistent with the discrete-slot model, because multiple objects are maintained by distinct localized population persistent activity patterns (bump attractors). We identified two phenomena of recurrent circuit dynamics that give rise to limited working memory capacity. As the working memory load increases, a localized persistent activity bump may either fade out (so the memory of the corresponding item is lost) or merge with another nearby bump (hence the resolution of mnemonic representation for the merged items becomes blurred). We identified specific dependences of these two phenomena on the strength and tuning of recurrent synaptic excitation, as well as network normalization: the overall population activity is invariant to set size and delay duration; therefore, a constant neural resource is shared by and dynamically allocated to the memorized items. We demonstrate that the model reproduces salient observations predicted by both discrete-slot and shared-resource models, and propose testable predictions of the merging phenomenon.

Lacking control over the trade-off between quality and quantity in visual short-term memory

Visual short-term memory (VSTM) is limited in the quantity and quality of items that can be retained over time. Importantly, these two mnemonic parameters interact: increasing the number of items in VSTM reduces the quality with which they are represented. Here, we ask whether this trade-off is under top-down control. Specifically, we test whether participants can strategically optimise the trade-off between quality and quantity for VSTM according to task demands. We manipulated strategic trade-off by varying expectations about the number of to-be-remembered items (Experiments 1-2) or the precision required for the memory-based judgement (Experiment 3). In a final experiment, we manipulated both variables in a complementary way to maximise the motivation to strategically control the balance between number and the quality of items encoded into VSTM. In different blocks, performance would benefit most either by encoding a large number of items with low precision or by encoding a small number of items with high precision (Experiment 4). In all experiments, we compared VSTM performance on trials matched for mnemonic demand, but within contexts emphasising the quality or quantity of VSTM representations. Across all four experiments, we found no evidence to suggest that participants use this contextual information to bias the balance between the number and precision of items in VSTM. Rather, our data suggest that the trade-off may be determined primarily by stimulus-driven factors at encoding.

Seven-month-old infants chunk items in memory

Although working memory has a highly constrained capacity limit of three or four items, both adults and toddlers can increase the total amount of stored information by "chunking" object representations in memory. To examine the developmental origins of chunking, we used a violation-of-expectation procedure to ask whether 7-month-old infants, whose working memory capacity is still maturing, also can chunk items in memory. In Experiment 1, we found that in the absence of chunking cues, infants failed to remember three identical hidden objects. In Experiments 2 and 3, we found that infants successfully remembered three hidden objects when provided with overlapping spatial and featural chunking cues. In Experiment 4, we found that infants did not chunk when provided with either spatial or featural chunking cues alone. Finally, in Experiment 5, we found that infants also failed to chunk when spatial and featural cues specified different chunks (i.e., were pitted against each other). Taken together, these results suggest that chunking is available before working memory capacity has matured but still may undergo important development over the first year of life.

Short-term storage capacity for visual objects depends on expertise

Visual short-term memory (VSTM) has traditionally been thought to have a very limited capacity of around 3-4 objects. However, recently several researchers have argued that VSTM may be limited in the amount of information retained rather than by a specific number of objects. Here we present a study of the effect of long-term practice on VSTM capacity. We investigated four age groups ranging from pre-school children to adults and measured the change in VSTM capacity for letters and pictures. We found a clear increase in VSTM capacity for letters with age but not for pictures. Our results indicate that VSTM capacity is dependent on the level of expertise for specific types of stimuli.

A Power-Law Model of Psychological Memory Strength in Short- and Long-Term Recognition

A classic law of cognition is that forgetting curves are closely approximated by power functions. This law describes relations between different empirical dependent variables and the retention interval, and the precise form of the functional relation depends on the scale used to measure each variable. In the research reported here, we conducted a recognition task involving both short- and long-term probes. We discovered that formal memory-strength parameters from an exemplar-recognition model closely followed a power function of the lag between studied items and a test probe. The model accounted for rich sets of response time (RT) data at both individual-subject and individual-lag levels. Because memory strengths were derived from model fits to choices and RTs from individual trials, the psychological power law was independent of the scale used to summarize the forgetting functions. Alternative models that assumed different functional relations or posited a separate fixed-strength working memory store fared considerably worse than the power-law model did in predicting the data.

Development of visual working memory precision in childhood

Visual working memory (VWM) is the facility to hold in mind visual information for brief periods of time. Developmental studies have suggested an increase during childhood in the maximum number of complete items that can simultaneously be stored in VWM. Here, we exploit a recent theoretical and empirical innovation to investigate instead the precision with which items are stored in VWM, where precision is a continuous measure reflecting VWM resolution. Ninety boys aged 7 to 13 years completed one-item and three-item VWM tasks in which stimuli were coloured bars varying in orientation. On each trial, participants used a rotating dial to reproduce the probed stimulus from memory. Results show linear age-related improvement in recall precision for both one-item and three-item VWM tasks. However, even the youngest age group stored a significant amount of information about all three items on the difficult 3-item VWM task. Importantly, the development of VWM precision was not accounted for by development on a sensorimotor control task. Whereas storage of a single complete item was previously thought to be well within the capacity limitations of the current age range, these results suggest protracted development during childhood and early adolescence in the resolution with which single and multiple items are stored in VWM. Probabilistic modelling of response distribution data suggests that improvement in VWM performance is attributable to a specific decrease in variability of stored feature representations, rather than to a decrease in misbinding or random noise. As such, we highlight a novel, potentially developmentally plausible mechanism that may underlie developmental improvement in VWM performance, independent of any alterations in the maximum number of complete items which can be stored.

Statistical learning and its consequences

Statistical learning refers to an unconscious cognitive process in which repeated patterns, or regularities, are extracted from the sensory environment. In this chapter, I describe what is currently known about statistical learning. First, I classify types of regularities that exist in the visual environment. Second, I introduce a family of experimental paradigms that have been used to study statistical learning in the laboratory. Third, I review a series of behavioral and functional neuroimaging studies that seek to uncover the underlying nature of statistical learning. Finally, I consider ways in which statistical learning may be important for perception, attention, and visual search. The goals of this chapter are thus to highlight the prevalence of regularities, to explain how they are extracted by the mind and brain, and to suggest that the resulting knowledge has widespread consequences for other aspects of cognition.

Contrasting Predictions of Low- and High-Threshold Models for the Detection of Changing Visual Features

Change blindness is the failure of observers to notice otherwise obvious changes to a visual scene when those changes are masked in some way (eg by blotches or a blanking of the screen). Typically, change blindness is taken as evidence that our representation of the visual world is capacity limited. The locus of this capacity limit is thought to be visual short-term memory (vSTM). The capacity of vSTM is usually estimated with a high-threshold model which assumes that each element in the stimulus array is either fully encoded or not encoded at all, and, furthermore, that false alarms can arise only by guessing, not by noise. Low-threshold models, by contrast, suggest that false alarms can arise by noise at the level of detection/discrimination and/or decision. In this study, we use a well-controlled stimulus display in which a single element changes over a blanking of the screen and contrast predictions from a popular high-threshold model of vSTM with the predictions of a low-threshold model (specifically, the sample-size model) of visual search and vSTM. The data were better predicted by the low-threshold model.

Precision of working memory for visual motion sequences and transparent motion surfaces

Recent studies investigating working memory for location, color, and orientation support a dynamic resource model. We examined whether this might also apply to motion, using random dot kinematograms (RDKs) presented sequentially or simultaneously. Mean precision for motion direction declined as sequence length increased, with precision being lower for earlier RDKs. Two alternative models of working memory were compared specifically to distinguish between the contributions of different sources of error that corrupt memory (W. Zhang & S. J. Luck, 2008 vs. P. M. Bays, R. F. G. Catalao, & M. Husain, 2009). The latter provided a significantly better fit for the data, revealing that decrease in memory precision for earlier items is explained by an increase in interference from other items in a sequence rather than random guessing or a temporal decay of information. Misbinding feature attributes is an important source of error in working memory. Precision of memory for motion direction decreased when two RDKs were presented simultaneously as transparent surfaces, compared to sequential RDKs. However, precision was enhanced when one motion surface was prioritized, demonstrating that selective attention can improve recall precision. These results are consistent with a resource model that can be used as a general conceptual framework for understanding working memory across a range of visual features.

The number and quality of representations in working memory

Flexible-resource theories characterize working memory as a flexible resource that can store either a large number of low-quality representations or a small number of high-quality representations. In contrast, limited-item theories propose that the number of items that can be stored in working memory is strictly limited and cannot be increased by decreasing the quality of the representations. We tested these fundamentally different conceptualizations of working memory capacity by determining whether observers could trade quality for quantity in working memory when given incentives to do so. We found no evidence that observers could increase the number of representations by decreasing their quality in working memory, but observers could make such a trade-off at earlier processing stages. Our results show that the capacity limit of working memory is best characterized as a limit on the number of items that can be stored and not as a limit on a finely divisible resource that simultaneously determines the number and quality of the representations.

Object features fail independently in visual working memory: evidence for a probabilistic feature-store model

The world is composed of features and objects and this structure may influence what is stored in working memory. It is widely believed that the content of memory is object-based: Memory stores integrated objects, not independent features. We asked participants to report the color and orientation of an object and found that memory errors were largely independent: Even when one of the object's features was entirely forgotten, the other feature was often reported. This finding contradicts object-based models and challenges fundamental assumptions about the organization of information in working memory. We propose an alternative framework involving independent self-sustaining representations that may fail probabilistically and independently for each feature. This account predicts that the degree of independence in feature storage is determined by the degree of overlap in neural coding during perception. Consistent with this prediction, we found that errors for jointly encoded dimensions were less independent than errors for independently encoded dimensions.

Qualitative similarities in the visual short-term memory of pigeons and people

Visual short-term memory plays a key role in guiding behavior, and individual differences in visual short-term memory capacity are strongly predictive of higher cognitive abilities. To provide a broader evolutionary context for understanding this memory system, we directly compared the behavior of pigeons and humans on a change detection task. Although pigeons had a lower storage capacity and a higher lapse rate than humans, both species stored multiple items in short-term memory and conformed to the same basic performance model. Thus, despite their very different evolutionary histories and neural architectures, pigeons and humans have functionally similar visual short-term memory systems, suggesting that the functional properties of visual short-term memory are subject to similar selective pressures across these distant species.

Recoding between two types of STM representation revealed by the dynamics of memory search

Visual STM (VSTM) is thought to be related to visual attention in several ways. Attention controls access to VSTM during memory encoding and plays a role in the maintenance of stored information by strengthening memorized content. We investigated the involvement of visual attention in recall from VSTM. In two experiments, we measured electrophysiological markers of attention in a memory search task with varying intervals between VSTM encoding and recall, and so we were able to track recoding of representations in memory. Results confirmed the involvement of attention in VSTM recall. However, the amplitude of the N2pc and N3rs components, which mark orienting of attention and search within VSTM, decreased as a function of delay. Conversely, the amplitude of the P3 and sustained posterior contralateral negativity components increased as a function of delay, effectively the opposite of the N2pc and N3rs modulations. These effects were only observed when verbal memory was not taxed. Thus, the results suggested that gradual recoding from visuospatial orienting of attention into verbal recall mechanisms takes place from short to long retention intervals. Interestingly, recall at longer delays was faster than at short delays, indicating that verbal representation is coupled with faster responses. These results extend the orienting-of-attention hypothesis by including an account of representational recoding during short-term consolidation and its consequences for recall from VSTM.

Coarse-to-fine encoding of spatial frequency information into visual short-term memory for faces but impartial decay

Face perception studies investigated how spatial frequencies (SF) are extracted from retinal display while forming a perceptual representation, or their selective use during task-imposed categorization. Here we focused on the order of encoding low-spatial frequencies (LSF) and high-spatial frequencies (HSF) from perceptual representations into visual short-term memory (VSTM). We also investigated whether different SF-ranges decay from VSTM at different rates during a study-test stimulus-onset asynchrony. An old/new VSTM paradigm was used in which two broadband faces formed the positive set and the probes preserved either low or high SF ranges. Exposure time of 500 ms was sufficient to encode both HSF and LSF in the perceptual representation (experiment 1). Nevertheless, when the positive-set was exposed for 500 ms, LSF-probes were better recognized in VSTM compared with HSF-probes; this effect vanished at 800-ms exposure time (experiment 2). Backward masking the positive set exposed for 800 ms re-established the LSF-probes advantage (experiment 3). The speed of decay up to 10 seconds was similar for LSF- and HSF-probes (experiment 4). These results indicate that LSF are extracted and consolidated into VSTM faster than HSF, supporting a coarse-to-fine order, while the decay from VSTM is not governed by SF.

EEG correlates of attentional load during multiple object tracking

While human subjects tracked a subset of ten identical, randomly-moving objects, event-related potentials (ERPs) were evoked at parieto-occipital sites by task-irrelevant flashes that were superimposed on either tracked (Target) or non-tracked (Distractor) objects. With ERPs as markers of attention, we investigated how allocation of attention varied with tracking load, that is, with the number of objects that were tracked. Flashes on Target discs elicited stronger ERPs than did flashes on Distractor discs; ERP amplitude (0-250 ms) decreased monotonically as load increased from two to three to four (of ten) discs. Amplitude decreased more rapidly for Target discs than Distractor discs. As a result, with increasing tracking loads, the difference between ERPs to Targets and Distractors diminished. This change in ERP amplitudes with load accords well with behavioral performance, suggesting that successful tracking depends upon the relationship between the neural signals associated with attended and non-attended objects.

Competition in working memory reduces frontal guidance of visual selection

Working memory (WM) representations can bias visual selection to matching stimuli in the field. WM biases can, however, be modulated by the level of cognitive load, with WM guidance reduced as memory load increases. Here, we used functional magnetic resonance imaging to distinguish between competing hypotheses for the reduction of WM guidance under load: 1) poor neural representations of memory contents under high load, 2) strategic control at high loads to direct attention away from search distracters matching the WM content, and 3) reduction of frontal top-down biasing of visual areas with increasing memory loads. We show that matching between WM contents and the visual display appeared to be well represented in visual areas under high memory loads, despite a lack of WM guidance at the behavioral level. There was little engagement of "cognitive control" areas in the prefrontal cortex during search at high loads. More importantly, WM guidance at low loads engaged a set of frontal regions in the superior and inferior ventral frontal cortex. Functional connectivity analyses revealed frontal regions working in concert with occipital areas at low memory loads, but this coupling was disrupted by increased memory load. We discuss the implications for understanding the mechanisms supporting the interplay between WM and attention.

Dynamic updating of working memory resources for visual objects

Recent neurophysiological and imaging studies have investigated how neural representations underlying working memory (WM) are dynamically updated for objects presented sequentially. Although such studies implicate information encoded in oscillatory activity across distributed brain networks, interpretation of findings depends crucially on the underlying conceptual model of how memory resources are distributed. Here, we quantify the fidelity of human memory for sequences of colored stimuli of different orientation. The precision with which each orientation was recalled declined with increases in total memory load, but also depended on when in the sequence it appeared. When one item was prioritized, its recall was enhanced, but with corresponding decrements in precision for other objects. Comparison with the same number of items presented simultaneously revealed an additional performance cost for sequential display that could not be explained by temporal decay. Memory precision was lower for sequential compared with simultaneous presentation, even when each item in the sequence was presented at a different location. Importantly, stochastic modeling established this cost for sequential display was due to misbinding object features (color and orientation). These results support the view that WM resources can be dynamically and flexibly updated as new items have to be stored, but redistribution of resources with the addition of new items is associated with misbinding object features, providing important constraints and a framework for interpreting neural data.

Flexible attention allocation to visual and auditory working memory tasks: manipulating reward induces a trade-off

Prominent roles for general attention resources are posited in many models of working memory, but the manner in which these can be allocated differs between models or is not sufficiently specified. We varied the payoffs for correct responses in two temporally-overlapping recognition tasks, a visual array comparison task and a tone sequence comparison task. In the critical conditions, an increase in reward for one task corresponded to a decrease in reward for the concurrent task, but memory load remained constant. Our results show patterns of interference consistent with a trade-off between the tasks, suggesting that a shared resource can be flexibly divided, rather than only fully allotted to either of the tasks. Our findings support a role for a domain-general resource in models of working memory, and furthermore suggest that this resource is flexibly divisible.

Storage and binding of object features in visual working memory

An influential conception of visual working memory is of a small number of discrete memory "slots", each storing an integrated representation of a single visual object, including all its component features. When a scene contains more objects than there are slots, visual attention controls which objects gain access to memory. A key prediction of such a model is that the absolute error in recalling multiple features of the same object will be correlated, because features belonging to an attended object are all stored, bound together. Here, we tested participants' ability to reproduce from memory both the color and orientation of an object indicated by a location cue. We observed strong independence of errors between feature dimensions even for large memory arrays (6 items), inconsistent with an upper limit on the number of objects held in memory. Examining the pattern of responses in each dimension revealed a gaussian distribution of error centered on the target value that increased in width under higher memory loads. For large arrays, a subset of responses were not centered on the target but instead predominantly corresponded to mistakenly reproducing one of the other features held in memory. These misreporting responses again occurred independently in each feature dimension, consistent with 'misbinding' due to errors in maintaining the binding information that assigns features to objects. The results support a shared-resource model of working memory, in which increasing memory load incrementally degrades storage of visual information, reducing the fidelity with which both object features and feature bindings are maintained.

Acquisition vs. memorization trade-offs are modulated by walking distance and pattern complexity in a large-scale copying paradigm

In a “block-copying paradigm”, subjects were required to copy a configuration of colored blocks from a model area to a distant work area, using additional blocks provided at an equally distant resource area. Experimental conditions varied between the inter-area separation (walking distance) and the complexity of the block patterns to be copied. Two major behavioral strategies were identified: in the memory-intensive strategy, subjects memorize large parts of the pattern and rebuild them without intermediate visits at the model area. In the acquisition-intensive strategy, subjects memorize one block at a time and return to the model after having placed this block. Results show that the frequency of the memory-intensive strategy is increased for larger inter-area separations (larger walking distances) and for simpler block patterns. This strategy-shift can be interpreted as the result of an optimization process or trade-off, minimizing combined, condition-dependent costs of the two strategies. Combined costs correlate with overall response time. We present evidence that for the memory-intensive strategy, costs correlate with model visit duration, while for the acquisition-intensive strategy, costs correlate with inter-area transition (i.e., walking) times.

Short-term memory scanning viewed as exemplar-based categorization

Exemplar-similarity models such as the exemplar-based random walk (EBRW) model (Nosofsky & Palmeri, 1997b) were designed to provide a formal account of multidimensional classification choice probabilities and response times (RTs). At the same time, a recurring theme has been to use exemplar models to account for old-new item recognition and to explain relations between classification and recognition. However, a major gap in research is that the models have not been tested on their ability to provide a theoretical account of RTs and other aspects of performance in the classic Sternberg (1966) short-term memory-scanning paradigm, perhaps the most venerable of all recognition-RT tasks. The present research fills that gap by demonstrating that the EBRW model accounts in natural fashion for a wide variety of phenomena involving diverse forms of short-term memory scanning. The upshot is that similar cognitive operating principles may underlie the domains of multidimensional classification and short-term old-new recognition.

Markers of preparatory attention predict visual short-term memory performance

Visual short-term memory (VSTM) is limited in capacity. Therefore, it is important to encode only visual information that is most likely to be relevant to behaviour. Here we asked which aspects of selective biasing of VSTM encoding predict subsequent memory-based performance. We measured EEG during a selective VSTM encoding task, in which we varied parametrically the memory load and the precision of recall required to compare a remembered item to a subsequent probe item. On half the trials, a spatial cue indicated that participants only needed to encode items from one hemifield. We observed a typical sequence of markers of anticipatory spatial attention: early attention directing negativity (EDAN), anterior attention directing negativity (ADAN), late directing attention positivity (LDAP); as well as of VSTM maintenance: contralateral delay activity (CDA). We found that individual differences in preparatory brain activity (EDAN/ADAN) predicted cue-related changes in recall accuracy, indexed by memory-probe discrimination sensitivity (d′). Importantly, our parametric manipulation of memory-probe similarity also allowed us to model the behavioural data for each participant, providing estimates for the quality of the memory representation and the probability that an item could be retrieved. We found that selective encoding primarily increased the probability of accurate memory recall; that ERP markers of preparatory attention predicted the cue-related changes in recall probability.

Maintaining binding in working memory: comparing the effects of intentional goals and incidental affordances

Much research on memory for binding depends on incidental measures. However, if encoding associations benefits from conscious attention, then incidental measures of binding memory might not yield a sufficient understanding of how binding is accomplished. Memory for letters and spatial locations was compared in three within-participants tasks, one in which binding was not afforded by stimulus presentation, one in which incidental binding was possible, and one in which binding was explicitly to be remembered. Some evidence for incidental binding was observed, but unique benefits of explicit binding instructions included preserved discrimination as set size increased and drastic reduction in false alarms to lures that included a new spatial location and an old letter. This suggests that substantial cognitive benefits, including enhanced memory for features themselves, might occur through intentional binding, and that incidental measures of binding might not reflect these advantages.

Loss of visual working memory within seconds: the combined use of refreshable and non-refreshable features

We reexamine the role of time in the loss of information from working memory, the limited information accessible for cognitive tasks. The controversial issue of whether working memory deteriorates over time was investigated using arrays of unconventional visual characters. Each array was followed by a postperceptual mask, a variable retention interval (RI), and a recognition probe character. Dramatic forgetting across an unfilled RI of up to 6 s was observed. Adding a distracting task during the RI (repetition, subtraction, or parity judgment using spoken digits) lowered the level of recall but not increasingly so across RIs. Also, arrays of English letters were not forgotten during the RI unless distracting stimuli were included, in contrast to the finding for unconventional characters. The results suggest that unconventional visual items include some features inevitably lost over time. Attention-related processing, however, assists in the retention of other features and of English letters. We identify important constraints for working memory theories and propose that an equilibrium between forgetting and reactivation holds but only for elements that are not inevitably lost over time.