How high is visual short-term memory capacity for object layout?

Adaptive allocation of human visual working memory capacity during statistical and categorical learning

Human brains are finite, and thus have bounded capacity. An efficient strategy for a capacity-limited agent is to continuously adapt by dynamically reallocating capacity in a task-dependent manner. Here we study this strategy in the context of visual working memory (VWM). People use their VWM stores to remember visual information over seconds or minutes. However, their memory performances are often error-prone, presumably due to VWM capacity limits. We hypothesize that people attempt to be flexible and robust by strategically reallocating their limited VWM capacity based on two factors: (a) the statistical regularities (e.g., stimulus feature means and variances) of the to-be-remembered items, and (b) the requirements of the task that they are attempting to perform. The latter specifies, for example, which types of errors are costly versus irrelevant for task performance. These hypotheses are formalized within a normative computational modeling framework based on rate-distortion theory, an extension of conventional Bayesian approaches that uses information theory to study rate-limited (or capacity-limited) processes. Using images of plants that are naturalistic and precisely controlled, we carried out two sets of experiments. Experiment 1 found that when a stimulus dimension (the widths of plants' leaves) was assigned a distribution, subjects adapted their VWM performances based on this distribution. Experiment 2 found that when one stimulus dimension (e.g., leaf width) was relevant for distinguishing plant categories but another dimension (leaf angle) was irrelevant, subjects' responses in a memory task became relatively more sensitive to the relevant stimulus dimension. Together, these results illustrate the task-dependent robustness of VWM, thereby highlighting the dependence of memory on learning.

Scene Buildup From Latent Memory Representations Across Eye Movements

An unresolved problem in eye movement research is how a representation is constructed on-line from several consecutive fixations of a scene. Such a scene representation is generally understood to be sparse; yet, for meeting behavioral goals a certain level of detail is needed. We propose that this is achieved through the buildup of latent representations acquired at fixation. Latent representations are retained in an activity-silent manner, require minimal energy expenditure for their maintenance, and thus allow a larger storage capacity than traditional, activation based, visual working memory. The latent representations accumulate and interact in working memory to form to the scene representation. The result is rich in detail while sparse in the sense that it is restricted to the task-relevant aspects of the scene sampled through fixations. Relevant information can quickly and flexibly be retrieved by dynamical attentional prioritization. Latent representations are observable as transient functional connectivity patterns, which emerge due to short-term changes in synaptic weights. We discuss how observing latent representations could benefit from recent methodological developments in EEG-eye movement co-registration.

Visual memory performance for color depends on spatiotemporal context

Performance on visual short-term memory for features has been known to depend on stimulus complexity, spatial layout, and feature context. However, with few exceptions, memory capacity has been measured for abruptly appearing, single-instance displays. In everyday life, objects often have a spatiotemporal history as they or the observer move around. In three experiments, we investigated the effect of spatiotemporal history on explicit memory for color. Observers saw a memory display emerge from behind a wall, after which it disappeared again. The test display then emerged from either the same side as the memory display or the opposite side. In the first two experiments, memory improved for intermediate set sizes when the test display emerged in the same way as the memory display. A third experiment then showed that the benefit was tied to the original motion trajectory and not to the display object per se. The results indicate that memory for color is embedded in a richer episodic context that includes the spatiotemporal history of the display.

Visual awareness of objects and their colour

At any given moment, our awareness of what we 'see' before us seems to be rather limited. If, for instance, a display containing multiple objects is shown (red or green disks), when one object is suddenly covered at random, observers are often little better than chance in reporting about its colour (Wolfe, Reinecke, & Brawn, Visual Cognition, 14, 749-780, 2006). We tested whether, when object attributes (such as colour) are unknown, observers still retain any knowledge of the presence of that object at a display location. Experiments 1-3 involved a task requiring two-alternative (yes/no) responses about the presence or absence of a colour-defined object at a probed location. On this task, if participants knew about the presence of an object at a location, responses indicated that they also knew about its colour. A fourth experiment presented the same displays but required a three-alternative response. This task did result in a data pattern consistent with participants' knowing more about the locations of objects within a display than about their individual colours. However, this location advantage, while highly significant, was rather small in magnitude. Results are compared with those of Huang (Journal of Vision, 10(10, Art. 24), 1-17, 2010), who also reported an advantage for object locations, but under quite different task conditions.

Hierarchical encoding in visual working memory: ensemble statistics bias memory for individual items

Influential models of visual working memory treat each item to be stored as an independent unit and assume that there are no interactions between items. However, real-world displays have structure that provides higher-order constraints on the items to be remembered. Even in the case of a display of simple colored circles, observers can compute statistics, such as mean circle size, to obtain an overall summary of the display. We examined the influence of such an ensemble statistic on visual working memory. We report evidence that the remembered size of each individual item in a display is biased toward the mean size of the set of items in the same color and the mean size of all items in the display. This suggests that visual working memory is constructive, encoding displays at multiple levels of abstraction and integrating across these levels, rather than maintaining a veridical representation of each item independently.

Color Relations Increase the Capacity of Visual Short-Term Memory

Do color relations such as similarity or harmony influence the ease with which colored patterns can be perceived and held in mind? We tested the influence of a relation supported in research on color harmony—similarity of hue—on the capacity of visual short-term memory (VSTM) for colors in patterns. Palettes of 4 similar-hue colors were rated as more pleasant (harmonious) than dissimilar-color palettes. The palettes were used in a VSTM color task. Patterns of 9 to 15 colored squares were presented, and accuracy of color change detection was measured. Memory performance was higher overall for similar-color palettes than for dissimilar-color palettes (experiments 1 and 3). Is this due to color similarity per se, or due to the harmony between colors in similar palettes? A final experiment provided strong support for the importance of color similarity as opposed to harmony. Overall, the advantages for color similarity, in terms of number of color squares held in memory (memory capacity) were 26% to 45% over dissimilar colors. The results indicate that color relations can have a strong impact on the capacity for perceiving and retaining color patterns.