Understanding the object benefit in visual short-term memory: the roles of feature proximity and connectedness

Similarity in motion binds and bends judgments of aspect ratio

It is well known that objects become grouped in perceptual organization when they share some visual feature, like a common direction of motion. Less well known is that grouping can change how people perceive a set of objects. For example, when a pair of shapes consistently share a common region of space, their aspect ratios tend to be perceived as more similar (are attracted toward each other). Conversely, when shapes are assigned to different regions in space their aspect ratios repel from each other. Here we examine whether the visual system produce both attractive and repulsive distortions when the state of grouping between a pair of shapes changes on a moment-to-moment basis. Observers viewed a pair of ellipses that differed in terms of how flat or tall they were and reported the aspect ratio of one ellipse from the pair. Each ellipse was defined by a cloud of coherently-moving dots, and the dots within the two ellipses had either the same or different directions of motion, varying from trial-to-trial. We found that the cued ellipse's aspect ratio was reported to be repelled from the aspect ratio of the uncued ellipse when the shapes had different directions of motion compared to when they had the same direction of motion. These results suggest that the visual system can adaptively alter visual experience based on grouping, in particular, repelling the appearance of objects when they do not appear to go together, and it can do so quickly and flexibly.

Memory matching features bias the ensemble perception of facial identity

Introduction

Humans have the ability to efficiently extract summary statistics (i.e., mean) from a group of similar objects, referred to as ensemble coding. Recent studies have demonstrated that ensemble perception of simple objects is modulated by the visual working memory (VWM) task through matching features in VWM. However, few studies have examined the extending scope of such a matching feature effect and the influence of the organization mode (i.e., the way of combining memory matching features with ensemble properties) on this effect. Two experiments were done to explore these questions.

Methods

We used a dual-task paradigm for both experiments, which included a VWM task and a mean estimation task. Participants were required to adjust a test face to the mean identity face and report whether the irregular objects in a memory probe were identical or different to the studied objects. In Experiment 1, using identity faces as ensemble stimuli, we compared participants’ performances in trials where a subset color matched that of the studied objects to those of trials without color-matching subsets. In Experiment 2, we combined memory matching colors with ensemble properties in common region cues and compared the effect with that of Experiment 1.

Results

Results of Experiments 1 and 2 showed an effect of the VWM task on high-level ensemble perception that was similar to previous studies using a low-level averaging task. However, the combined analysis of Experiments 1 and 2 revealed that memory matching features had less influence on mean estimations when matching features and ensemble properties combined in the common region than when combined as parts of a complete unit.

Conclusion

These findings suggest that the impact of memory matching features is not limited by the level of stimulus feature, but can be impacted by the organization between matching features and ensemble target properties.

Task Demands Differentially Affect Processing of Intrinsic and Extrinsic Object Features in Working Memory

Some argue that visual working memory operates on integrated object representations. Here, we contend that obligatory feature integration occurs with intrinsic but not extrinsic object features. Working memory for shapes and colors was assessed using a change-detection task with a central test probe, while recording event-related potentials (ERPs). Color was either an intrinsic surface feature of a shape or connected to the shape via a proximal but spatially disjunct extrinsic frame. There were two types of test: The direct test required memory for shape and color; the indirect test required only shape memory. Study-test changes of color were therefore either task-relevant or task-irrelevant. We assessed performance costs and event-related potential (ERP) effects arising from color changes. In the direct test, performance was poorer for extrinsic than intrinsic stimuli; task-relevant color changes elicited enhanced frontal negativity (N2, FN400) for both intrinsic and extrinsic stimuli. In the indirect test, performance costs and ERP effects associated with irrelevant color change were larger for intrinsic than extrinsic stimuli. This suggests intrinsic information is more readily integrated into the working-memory representation and evaluated against the test probe. Findings imply that feature integration is not obligatory under all conditions but influenced by stimulus-driven and task-related focus of attention.

Object-based visual working memory: an object benefit for equidistant memory items presented within simple contours

Previous research has shown that more information can be stored in visual working memory (VWM) when multiple items belong to the same object. Here, in four experiments, we investigated the object effect on memory for spatially equidistant features by manipulating simple, task-irrelevant contours that combined these features. In Experiments 1, 3, and, 4, three grating orientations, and in Experiment 2, one color and two orientations, were presented simultaneously to be memorized. Mixture modeling was applied to estimate both the precision and the guess rates of recall errors. Overall results showed that two target features were remembered more accurately when both were part of the same object. Further analysis showed that the probability of recall increased in particular when both features were extracted from the same object. In Experiment 2, we found that the object effect was greater for features from orthogonal dimensions, but this came at the cost of lower memory precision. In Experiment 3, when we kept the locations of the features perfectly consistent over trials so that the participants could attend to these locations rather than the contour, we still found object benefits. Finally, in Experiment 4 when we manipulated the temporal order of the object and the memory features presentations, it was confirmed that the object benefit is unlikely to stem from the strategical usage of object information. These results suggested that the object benefit arises automatically, likely at an early perceptual level.

The subconscious impact of line orientations in background images on memory of Chinese written characters

Oriented lines impact human cognition subconsciously. This study aimed to determine whether line orientations in the background of Chinese written characters influenced individual’s memory and emotion. Five pictures with Chinese characters as experimental material, in which four had equidistant parallel lines (0°, -45°, 90°, and +45°) as background and the other one had a blank background, were presented on a personal computer screen, for 15 seconds each, to 94 participants. The participants were then given 45 seconds to write down what they had just memorized. Participants’ emotion was identified by their Heart Rate Variability (HRV) simultaneously during the viewing process. The results showed that vertical (90°) and 45° leftward leaning lines (-45°) did not weaken users’ memory, and no significant difference in memory was found between these two states and the blank background, while horizontal (0°) and 45° rightward leaning lines (+45°) weakened the memory effect significantly. Overall, memory decreased in the condition of horizontally lined background while no influence in vertically lined background condition; and it showed asymmetry under leftward and rightward leaning line conditions: memory decreased in rightward leaning lined background while no influence in leftward lined background. Moreover, the results of emotion and memory showed negative similar trend. These findings provide practical suggestions for visual information design.

Visual working memory for connected 3D objects: effects of stimulus complexity, dimensionality and connectivity

Visual working memory (VWM) is typically measured using arrays of two-dimensional isolated stimuli with simple visual identities (e.g., color or shape), and these studies typically find strong capacity limits. Science, technology, engineering and mathematics (STEM) experts are tasked with reasoning with representations of three-dimensional (3D) connected objects, raising questions about whether those stimuli would be subject to the same limits. Here, we use a color change detection task to examine working memory capacity for 3D objects made up of differently colored cubes. Experiment 1a shows that increasing the number of parts of an object leads to less sensitivity to color changes, while change-irrelevant structural dimensionality (the number of dimensions into which parts of the structure extend) does not. Experiment 1b shows that sensitivity to color changes decreases similarly with increased complexity for multipart 3D connected objects and disconnected 2D squares, while sensitivity is slightly higher with 3D objects. Experiments 2a and 2b find that when other stimulus characteristics, such as size and visual angle, are controlled, change-irrelevant dimensionality and connectivity have no effect on performance. These results suggest that detecting color changes on 3D connected objects and on displays of isolated 2D stimuli are subject to similar set size effects and are not affected by dimensionality and connectivity when these properties are change-irrelevant, ruling out one possible explanation for scientists’ advantages in storing and manipulating representations of complex 3D objects.

Global-local consistency benefits memory-guided tracking of a moving target

INTRODUCTION

Previous findings have demonstrated that several Gestalt principles do facilitate VSTM performance in change detection tasks. However, few studies have investigated the role of and time-course of global-local consistency in motion perception.

METHODS

Participants were required to track a moving target surrounded by three different backgrounds: blank, inconsistent, or consistent. Global-local objects were be bound to move together (covariation). During the PMT, participants had to follow the moving target with their eyes and react as fast as possible when the target had just vanished behind the obstruction or would arrive at a predetermined point of interception. Variable error (VE) and constant error (CE) of estimated time-to-contact (TTC) and gain of smooth pursuit eye movements were calculated in various conditions and analyzed qualitatively.

RESULTS

Experiment 1 established the basic finding that VSTM performance could benefit from global-local consistency. Experiment 2 extended this finding by eye-tracking device. Both in visible phase and in occluded phase, CEs were smaller for the target in a consistent background than for the target in an inconsistent background and for the target in a blank background, with both differences significant (ps < .05). However, the difference in VE among three conditions was not significant. At early stage (100-250 ms), later stage (2750-3000 ms), and termination stage (5750-6000 ms) of smooth pursuit, the velocity gains were higher in the trials with consistent backgrounds than in the trials with inconsistent backgrounds and blank backgrounds (ps < .001). With the exception of 100-250 ms phase, the means did not differ between the inconsistent background and the blank background trials (ps > .1).

CONCLUSIONS

Global-local consistency could be activated within the first few hundred milliseconds to prioritize the deployment of attention and eye movement to component target. Meanwhile, it also removes ambiguity from motion tracking and TTC estimation under some unpredictable conditions, leading to the consistency advantage during smooth-pursuit termination phase. Global-local consistency may act as an important information source to TTC estimation and oculomotor response in PMT.

Automatically binding relevant and irrelevant features in visual working memory

It is generally assumed that, to save storage space, features are stored as integrated objects in visual working memory (VWM). Although such an object-based account does not always hold because features can be processed in parallel, a previous study has shown that different features can be automatically bound with their locations (task-irrelevant feature) into an integrated unit, resulting in improved memory performance. The present study was designed to further explore this phenomenon by investigating whether other features, which are not spatial in origin, can act as the binding cue to form such automatic binding. To test this, we used three different features as binding cues (i.e., colour, spatial frequency, and shape) over multiple separate experiments. The results consistently showed that when two features shared the same binding cue, memory performance was better relative to when each of those features had their own binding cue. We conclude that any task-irrelevant feature can act as a binding cue to automatically bind with task-relevant features even across different objects, resulting in memory enhancement.

A compressibility account of the color-sharing bonus in working memory

It has been established that objects sharing color in a visual display can boost working memory. The capacity to encode singletons particularly benefits from the repetition of colors encoded as perceptual groups. We manipulated the algorithmic complexity of visual displays to test whether compressibility of information could account for the color-sharing bonus. This study used a free recall working memory task in which the participants were shown displays of 2 to 8 color items. We examined the influence of set size, complexity, number of same-color clusters and amount of color redundancy. The results showed that the probability of correct recall of the pattern and the proportion of similarity between the pattern and the response decreased with an increase of each manipulated variable, except for color redundancy in terms of probability of correct recall. The model performance of complexity did not differ from that of clusters, but complexity was found more accurate than either set size or color redundancy. The results also showed that similar items were more often recalled adjacently, and complexity correlated strongly with the number of extra color repetitions in the response, suggesting that more complex patterns encouraged the use of information compression. Moreover, color repetitions were more often recalled first and the probability of correct recall for singletons and sub-patterns could be predicted by the compressibility measure. We discuss the potential advantage of using compressibility measures to capture the effects of regularities in visual patterns, in particular to refine analysis of the color-sharing bonus.

Time-dependent discrimination advantages for harmonic sounds suggest efficient coding for memory

Perceptual systems have finite memory resources and must store incoming signals in compressed formats. To explore whether representations of a sound's pitch might derive from this need for compression, we compared discrimination of harmonic and inharmonic sounds across delays. In contrast to inharmonic spectra, harmonic spectra can be summarized, and thus compressed, using their fundamental frequency (f0). Participants heard two sounds and judged which was higher. Despite being comparable for sounds presented back-to-back, discrimination was better for harmonic than inharmonic stimuli when sounds were separated in time, implicating memory representations unique to harmonic sounds. Patterns of individual differences (correlations between thresholds in different conditions) indicated that listeners use different representations depending on the time delay between sounds, directly comparing the spectra of temporally adjacent sounds, but transitioning to comparing f0s across delays. The need to store sound in memory appears to determine reliance on f0-based pitch and may explain its importance in music, in which listeners must extract relationships between notes separated in time.

Charting the Diversity of Strategic Processes in Visuospatial Short-Term Memory

Despite the abundant literature on visuospatial short-term memory, researchers have devoted little attention to strategic processes: What procedures do subjects implement to memorize visuospatial material? Evidence for various strategies exists, but it is spread across a variety of fields. This integrative review of the literature brings together scattered evidence to provide an overview of strategic processes in visuospatial memory tasks. The diversity of strategies and their proposed operating mechanisms are reviewed and discussed. The evidence leads to proposing seven broad strategic processes used in visuospatial short-term memory, each with multiple variants. Strategies can vary across individuals, but the same subjects also appear to use multiple strategies depending on the perceptual features of to-be-remembered displays. These results point to a view of visuospatial strategies as a functional library of facilitatory processes on which subjects can draw to support visuospatial short-term memory performance. Implications are discussed for the difference between visual and spatial tasks, for the appropriate measurement of strategic behaviors, and for the interpretation of performance in visuospatial memory tasks.

The transition from feature to object: Storage unit in visual working memory depends on task difficulty

Visual working memory (VWM) is a cognitive memory buffer for temporarily processing and storing visual information. Previous studies suggest that its capacity is severely limited, and there is an ongoing debate on whether the storage capacity is object-based or feature-based in VWM. In this study, a change-detection task was employed to investigate whether and how task difficulty can affect VWM, specifically, its capacity and the unit of storage. Task difficulty was manipulated through the set size of memory items, memory fidelity required by the resolution of representation and the type of feature tested. We examined two types of stimuli: the single-feature type, where each memory item was composed of a single feature (color or shape), and the conjunctive-feature type, where each item was composed of a conjunction of two features (colored shape). Experiment 1 replicated the previous findings that memory capacity for colors was larger than shapes, and decreased with the resolution demand regardless of the type of stimuli. In Experiment 2, we analyzed and compared the results from single-feature items and conjunctive-feature items in the low- and high-resolution conditions while controlling for the number of to-be-remembered features. By directly matching the estimated capacity based on an object-unit and a feature-unit with the theoretical prediction, the results showed that the unit storage in VWM tended to be feature-based with low task difficulty, and to be object-based with high task difficulty. This suggests that VWM is dynamic and flexible, dependent on the load of the current task.

Working memory for stereoscopic depth is limited and imprecise-evidence from a change detection task

Most studies on visual working memory (VWM) and spatial working memory (SWM) have employed visual stimuli presented at the fronto-parallel plane and few have involved depth perception. VWM is often considered as a memory buffer for temporarily holding and manipulating visual information that relates to visual features of an object, and SWM for holding and manipulating spatial information that concerns the spatial location of an object. Although previous research has investigated the effect of stereoscopic depth on VWM, the question of how depth positions are stored in working memory has not been systematically investigated, leaving gaps in the existing literature on working memory. Here, we explore working memory for depth by using a change detection task. The memory items were presented at various stereoscopic depth planes perpendicular to the line of sight, with one item per depth plane. Participants were asked to make judgments on whether the depth position of the target (one of the memory items) had changed. The results showed a conservative response bias that observers tended to make 'no change' responses when detecting changes in depth. In addition, we found that similar to VWM, the change detection accuracy degraded with the number of memory items presented, but the accuracy was much lower than that reported for VWM, suggesting that the storage for depth information is severely limited and less precise than that for visual information. The detection sensitivity was higher for the nearest and farthest depths and was better when the probe was presented along with the other items originally in the memory array, indicating that how well the to-be-stored depth can be stored in working memory depends on its relation with the other depth positions.

The influence of object structure on visual short-term memory for multipart objects

Numerous studies have shown that more visual features can be stored in visual short-term memory (VSTM) when they are encoded from fewer objects (Luck & Vogel, 1997, Nature, 390, 279-281; Olson & Jiang, 2002, Perception & Psychophysics, 64[7], 1055-1067). This finding has been consistent for simple objects with one surface and one boundary contour, but very few experiments have shown a clear performance benefit when features are organized as multipart objects versus spatially dispersed single-feature objects. Some researchers have suggested multipart object integration is not mandatory because of the potential ambiguity of the display (Balaban & Luria, 2015, Cortex, 26(5), 2093-2104; Luria & Vogel, 2014, Journal of Cognitive Neuroscience, 26[8], 1819-1828). For example, a white bar across the middle of a red circle could be interpreted as two objects, a white bar occluding a red circle, or as a single two-colored object. We explore whether an object benefit can be found by disambiguating the figure-ground organization of multipart objects using a luminance gradient and linear perspective to create the appearance of a unified surface. Also, we investigated memory for objects with a visual feature indicated by a hole, rather than an additional surface on the object. Results indicate the organization of multipart objects can influence VSTM performance, but the effect is driven by how the specific organization allows for use of global ensemble statistics of the memory array rather than a memory benefit for local object representations.

Composite Face Effect Predicts Configural Encoding in Visual Short-Term Memory

In natural vision, visual scenes consist of individual items (e.g., trees) and global properties of items as a whole (e.g., forest). These different levels of representations can all contribute to perception, natural scene understanding, sensory memory, working memory, and long-term memory. Despite these various hierarchical representations across perception and cognition, the nature of the global representations has received considerably less attention in empirical research on working memory than item representations. The present study aimed to understand the perceptual root of the configural information retained in Visual Short-term Memory (VSTM). Specifically, we assessed whether configural VSTM was related to holistic face processing across participants using an individual differences approach. Configural versus item encoding in VSTM was assessed using Xie and Zhang's (2017) dual-trace Signal Detection Theory model in a change detection task for orientation. Configural face processing was assessed using Le Grand composite face effect (CFE). In addition, overall face recognition was assessed using Glasgow Face Matching Test (GFMT). Across participants, holistic face encoding, but not face recognition accuracy, predicted configural information, but not item information, retained in VSTM. Together, these findings suggest that configural encoding in VSTM may have a perceptual root.

Visual working memory representation as a topological defined perceptual object

The question of what the basic unit is of visual working memory remains one of the most fundamental and controversial issues. In the current study, we proposed a unique perspective based on early topological perception to describe the nature of representation in visual working memory. In a series of updating change-detection tasks, the repetition-benefit effect on color memory was not affected when items in the second memory array underwent massive changes of nontopological features from the first memory array. However, when the topological properties of an item changed, the repetition-benefit effect was destroyed, suggesting that the item was perceived as a new object impairing the original memory. Hence, our results suggest that a perceptual object defined by its topological invariance might be a unique perspective from which to describe representations of visual working memory.

Object Vision in a Structured World

In natural vision, objects appear at typical locations, both with respect to visual space (e.g., an airplane in the upper part of a scene) and other objects (e.g., a lamp above a table). Recent studies have shown that object vision is strongly adapted to such positional regularities. In this review we synthesize these developments, highlighting that adaptations to positional regularities facilitate object detection and recognition, and sharpen the representations of objects in visual cortex. These effects are pervasive across various types of high-level content. We posit that adaptations to real-world structure collectively support optimal usage of limited cortical processing resources. Taking positional regularities into account will thus be essential for understanding efficient object vision in the real world.

Saturation and brightness modulate the effect of depth on visual working memory

Although previous studies show inconsistent results regarding the effect of depth perception on visual working memory (VWM), a recent finding shows that perceptually closer-in-depth items are better remembered than farther items when combining the congruent disparity and relative size cues. In this study, we employed a similar change detection paradigm to investigate the effects of saturation and brightness, alone or in combination with binocular disparity, on VWM. By varying the appearance of the memory items, we aimed to manipulate the visual salience as well as to simulate the aerial perspective cue that induces depth perception. We found that the change detection accuracy was significantly improved for brighter and more saturated items, but not for items solely with higher saturation. Additionally, combining saturation with the congruent disparity cue significantly improved memory performance for perceptually closer items over farther items. Conflicting the disparity cue with saturation eliminated the memory benefit for the closer items. These results indicate that saturation and brightness could modulate the effect of depth on VWM, and both visual salience and depth perception affect VWM possibly through a common underlying mechanism of setting priority for attentional selection.

Working memory capacity is enhanced by distributed prefrontal activation and invariant temporal dynamics

The amount of information that can be stored in working memory is limited but may be improved with practice. The basis of improved efficiency at the level of neural activity is unknown. To investigate this question, we trained monkeys to perform a working memory task that required memory for multiple stimuli. Performance decreased as a function of number of stimuli to be remembered, but improved as the animals practiced the task. Neuronal recordings acquired during this training revealed two hitherto unknown mechanisms of working memory capacity improvement. First, more prefrontal neurons became active as working memory improved, but their baseline activity decreased. Second, improved working memory capacity was characterized by less variable temporal dynamics, resulting in a more consistent firing rate at each time point during the course of a trial. Our results reveal that improved performance of working memory tasks is achieved through more distributed activation and invariant neuronal dynamics.

Task-dependent effects of voluntary space-based and involuntary feature-based attention on visual working memory

Previous research has shown that visual working memory (VWM) can be modulated by space-based or feature-based attentional selection. However, it remains unclear how the two modes of attention operate jointly to affect VWM, and in particular, if involuntary feature-based attention plays a role in VWM. In this study, a pre-cued change detection paradigm was employed to investigate the concurrent effects of space- and feature-based attention on VWM. Space-based attention was manipulated by informative spatial cueing and by varying the proximity between the test item and the cued (fixated) memory item, while feature-based attention was induced in an involuntary manner by having the test item to share the same color or shape with the cued item on a fraction of trials. The results showed that: (1) the memory performance for the cued items was always better than the uncued items, suggesting a beneficial effect of voluntary spatial attention; (2) with a brief duration of the memory array (250 ms), cue-test proximity benefited VWM in the shape judgment task but not in the color judgment task, whereas with a longer duration (1200 ms), no proximity effect was found for either task; (3) VWM was improved for the same-colored items regardless of the task and duration; (4) VWM was improved for the same-shaped items only in the shape judgment task with the longer duration of the memory array. A discrimination task further showed that the proximity effect associated with VWM reflects a perceptual bottleneck in memory encoding for shape but not for color with a brief display. Our results suggest that involuntary feature-based attention could be triggered by spatial cueing to modulate VWM; involuntary color-based attention facilitates VWM independently of task, whereas shape-based facilitation is task-dependent, i.e., confined only to the shape judgment task, presumably reflecting different attention-guiding potencies of the two features.

Depth benefits now loading: Visual working memory capacity and benefits in 3-D

The present studies explored how performance in multidimensional displays varies as a function of visual working memory load, item distribution across depths, and individual capacity differences. In Experiment 1, the benefit of depth information (one depth vs. two depths) was examined across seven set sizes within a change-detection paradigm. Multiple depth planes engendered performance benefits with five items, but elicited performance decrements with three items. These effects were associated with working memory capacity, such that benefits were only observed when the working memory load exceeded an individual's max capacity. Experiment 2 evaluated how the distribution of items in depth aids working memory performance. Equal distribution of items across depths produced higher accuracy compared with when the target was isolated in depth. Lastly, Experiment 3 explored how differences in working memory capacity affect an individual's ability to use depth information to improve their performance. The results indicate that both low-capacity and high-capacity individuals can benefit from depth information, but this may vary as a function of working memory load. Overall, the results indicate that multidimensional displays can improve performance with sufficient working memory load, possibly through some sort of depth tag.

Evidence for the beneficial effect of perceptual grouping on visual working memory: an empirical study on illusory contour and a meta-analytic study

The capacity of visual working memory (VWM) is found to be extremely limited. Past research shows that VWM can be facilitated by Gestalt principles of grouping, however, it remains controversial whether factors like the type of Gestalt principles, the characteristics of stimuli and the nature of experimental design could affect the beneficial effect of grouping. In particular, studies have shown that perceptual grouping could improve memory performance for a feature that is relevant for grouping, but it is unclear whether the same improvement exists for a feature that is irrelevant for grouping. In this article, an empirical study and a meta-analytic study were conducted to investigate the effect of perceptual grouping on VWM. In the empirical study, we examined the grouping effect by employing a Kanizsa illusion in which memory items were grouped by illusory contour. We found that the memory performance was improved for the grouped items even though the tested feature was grouping irrelevant, and the improvement was not significantly different from the effect of grouping by physical connectedness or by solid occlusion. In the meta-analytic study, we systematically and quantitatively examined the effect of perceptual grouping on VWM by pulling the results from all eligible studies, and found that the beneficial grouping effect was robust but the magnitude of the effect can be affected by several moderators. Factors like the types of grouping methods, the duration and the layout of the memory display, and the characteristics of the tested feature moderated the grouping effect, whereas whether employing a cue or a verbal suppression task did not. Our study suggests that the underlying mechanism of the grouping benefit may be distinct with regard to grouping relevancy of the to-be-stored feature. The grouping effect on VWM may be independent of attention for a grouping relevant feature, but may rely on attentional prioritization for a grouping irrelevant feature.

Examining the neural correlates of active and passive forms of verbal-spatial binding in working memory

We designed an fMRI study to pinpoint the neural correlates of active and passive binding in working memory. Participants were instructed to memorize three words and three spatial locations. In the passive binding condition, words and spatial locations were directly presented as bound. Conversely, in the active binding condition, words and spatial locations were presented as separated, and participants were directed to intentionally create associations between them. Our results showed that participants performed better on passive binding relative to active binding. FMRI analysis revealed that both binding conditions induced greater activity within the hippocampus. Additionally, our analyses divulged regions specifically engaged in passive and active binding. Altogether, these data allow us to propose the hippocampus as a central candidate for working memory binding. When needed, a frontal-parietal network can contribute to the rearrangement of information. These findings may inform theories of working memory binding.

Interactions between visual working memory representations

We investigated whether the representations of different objects are maintained independently in working memory or interact with each other. Observers were shown two sequentially presented orientations and required to reproduce each orientation after a delay. The sequential presentation minimized perceptual interactions so that we could isolate interactions between memory representations per se. We found that similar orientations were repelled from each other whereas dissimilar orientations were attracted to each other. In addition, when one of the items was given greater attentional priority by means of a cue, the representation of the high-priority item was not influenced very much by the orientation of the low-priority item, but the representation of the low-priority item was strongly influenced by the orientation of the high-priority item. This indicates that attention modulates the interactions between working memory representations. In addition, errors in the reported orientations of the two objects were positively correlated under some conditions, suggesting that representations of distinct objects may become grouped together in memory. Together, these results demonstrate that working-memory representations are not independent but instead interact with each other in a manner that depends on attentional priority.

Lateralization of Executive Function: Working Memory Advantage for Same Hemifield Stimuli in the Monkey

Working memory capacity, the amount of information that may be maintained in mind over a period of seconds, is extremely limited, to a handful of items. Some evidence exists that the number of visual items that may be maintained in working memory is independent for the two hemifields. To test this idea, we trained monkeys to perform visual working memory tasks that required maintenance in memory of the locations and/or shapes of 3–5 visual stimuli. We then tested whether systematic performance differences were present for stimuli concentrated in the same hemifield, vs. distributed across hemifields. We found little evidence to support the expectation that working memory capacity is independent in the two hemifields. Instead, when an advantage of stimulus arrangement was present, it involved multiple stimuli presented in the same hemifield. This conclusion was consistent across variations of the task, performance levels, and apparent strategies adopted by individual subjects. This result suggests that factors such as grouping that favor processing of stimuli in relative proximity may counteract the benefits of independent processing in the two hemispheres. Our results reveal an important property of working memory and place constraints on models of working memory capacity.

The development of real-time stability supports visual working memory performance: Young children's feature binding can be improved through perceptual structure

Working memory is a basic cognitive process that predicts higher-level skills. A central question in theories of working memory development is the generality of the mechanisms proposed to explain improvements in performance. Prior theories have been closely tied to particular tasks and/or age groups, limiting their generalizability. The cognitive dynamics theory of visual working memory development has been proposed to overcome this limitation. From this perspective, developmental improvements arise through the coordination of cognitive processes to meet demands of different behavioral tasks. This notion is described as real-time stability, and can be probed through experiments that assess how changing task demands impact children's performance. The current studies test this account by probing visual working memory for colors and shapes in a change detection task that compares detection of changes to new features versus swaps in color-shape binding. In Experiment 1, 3- to 4-year-old children showed impairments specific to binding swaps, as predicted by decreased real-time stability early in development; 5- to 6-year-old children showed a slight advantage on binding swaps, but 7- to 8-year-old children and adults showed no difference across trial types. Experiment 2 tested the proposed explanation of young children's binding impairment through added perceptual structure, which supported the stability and precision of feature localization in memory-a process key to detecting binding swaps. This additional structure improved young children's binding swap detection, but not new-feature detection or adults' performance. These results provide further evidence for the cognitive dynamics and real-time stability explanation of visual working memory development. (PsycINFO Database Record

Evidence for the effect of depth on visual working memory

Visual working memory (VWM) is a cognitive memory buffer for temporarily holding, processing, and manipulating visual information. Previous studies have demonstrated mixed results of the effect of depth perception on VWM, with some showing a beneficial effect while others not. In this study, we employed an adapted change detection paradigm to investigate the effects of two depth cues, binocular disparity and relative size. The memory array consisted of a set of pseudo-randomly positioned colored items, and the task was to judge whether the test item was changed compared to the memory item after a retention interval. We found that presenting the items in stereoscopic depth alone hardly affected VWM performance. When combining the two coherent depth cues, a significant larger VWM capacity of the perceptually closer-in-depth items was observed than that of the farther items, but the capacity for the two-depth-planes condition was not significantly different from that for the one-plane condition. Conflicting the two depth cues resulted in cancelling the beneficial effect of presenting items at a closer depth plane. The results indicate that depth perception could affect VWM, and the visual system may have an advantage in maintaining closer-in-depth objects in working memory.

The Whole Warps the Sum of Its Parts

The efficiency of averaging properties of sets without encoding redundant details is analogous to gestalt proposals that perception is parsimoniously organized as a function of recurrent order in the world. This similarity suggests that grouping and averaging are part of a broader set of strategies allowing the visual system to circumvent capacity limitations. To examine how gestalt grouping affects the manner in which information is averaged and remembered, I compared the error in observers' adjustments of remembered sizes of individual circles in two different mean-size sets defined by similarity, proximity, connectedness, or a common region. Overall, errors were more similar within the same gestalt-defined groups than between different gestalt-defined groups, such that the remembered sizes of individual circles were biased toward the mean size of their respective gestalt-defined groups. These results imply that gestalt grouping facilitates perceptual averaging to minimize the error with which individual items are encoded, thereby optimizing the efficiency of visual short-term memory.

Contralateral delay activity tracks the influence of Gestalt grouping principles on active visual working memory representations

Recent studies have demonstrated that factors influencing perception, such as Gestalt grouping cues, can influence the storage of information in visual working memory (VWM). In some cases, stationary cues, such as stimulus similarity, lead to superior VWM performance. However, the neural correlates underlying these benefits to VWM performance remain unclear. One neural index, the contralateral delay activity (CDA), is an event-related potential that shows increased amplitude according to the number of items held in VWM and asymptotes at an individual's VWM capacity limit. Here, we applied the CDA to determine whether previously reported behavioral benefits supplied by similarity, proximity, and uniform connectedness were reflected as a neural savings such that the CDA amplitude was reduced when these cues were present. We implemented VWM change-detection tasks with arrays including similarity and proximity (Experiment 1); uniform connectedness (Experiments 2a and 2b); and similarity/proximity and uniform connectedness (Experiment 3). The results indicated that when there was a behavioral benefit to VWM, this was echoed by a reduction in CDA amplitude, which suggests more efficient processing. However, not all perceptual grouping cues provided a VWM benefit in the same measure (e.g., accuracy) or of the same magnitude. We also found unexpected interactions between cues. We observed a mixed bag of effects, suggesting that these powerful perceptual grouping benefits are not as predictable in VWM. The current findings indicate that when grouping cues produce behavioral benefits, there is a parallel reduction in the neural resources required to maintain grouped items within VWM.

The effects of direction similarity in visual working memory: Behavioural and event-related potential studies

Object similarity can improve visual working memory (VWM) performance in the change-detection task, but impair the recognition performance when it occurs at retrieval of VWM in the recognition task. The effect of direction similarity is an issue that has not been well resolved. Furthermore, electrophysiological evidence in support of the mechanisms that underlie the effects of similarity is still scarce. In the current study, we conducted three behavioural experiments to examine the effects of direction similarity on memory performance with regard to both the encoding and retrieval phases of VWM and one event-related potential (ERP) experiment to explore the neural signatures of direction similarity in VWM. Our behavioural studies indicated that direction similarity improved performance when it occurred at the encoding phase but impaired performance when it occurred at the retrieval phase. Moreover, the ERP experiment showed that the amplitude of the contralateral delay activity (CDA) increased with the increasing set size for similar but not dissimilar directions. In addition, the CDA amplitude for similar directions was lower than that for dissimilar directions at set size 2. Taken together, these findings suggest that direction similarity at encoding has a positive effect on VWM performance and at retrieval has a negative effect. Given that VWM capacity depends on information load and the number of objects, the positive effect of similarity may be attributed to reduced information load of memory objects.

Visual working memory organization is subject to top-down control

The limited capacity of visual working memory (VWM) can be maximized by combining multiple features into a single representation through grouping principles such as connection, proximity, and similarity. In this study, we sought to understand how VWM organizes information by investigating how connection and similarity cues are used either alone or in the presence of another grouping cue. Furthermore, we examined whether the use of one cue over another is within volitional control. Participants remembered displays of objects that contained no grouping cues, connection cues only, similarity cues only, or both connection and similarity cues. We found that it is possible to use either connection or similarity cues, although connection cues tend to dominate if the cues are in conflict with one another. However, it is possible to flexibly use either similarity or connection cues if both are present, depending on the task goals.

Organization principles in visual working memory: Evidence from sequential stimulus display

Although the mechanisms of visual working memory (VWM) have been studied extensively in recent years, the active property of VWM has received less attention. In the current study, we examined how VWM integrates sequentially presented stimuli by focusing on the role of Gestalt principles, which are important organizing principles in perceptual integration. We manipulated the level of Gestalt cues among three or four sequentially presented objects that were memorized. The Gestalt principle could not emerge unless all the objects appeared together. We distinguished two hypotheses: a perception-alike hypothesis and an encoding-specificity hypothesis. The former predicts that the Gestalt cue will play a role in information integration within VWM; the latter predicts that the Gestalt cue will not operate within VWM. In four experiments, we demonstrated that collinearity (Experiment 1) and closure (Experiment 2) cues significantly improved VWM performance, and this facilitation was not affected by the testing manner (Experiment 3) or by adding extra colors to the memorized objects (Experiment 4). Finally, we re-established the Gestalt cue benefit with similarity cues (Experiment 5). These findings together suggest that VWM realizes and uses potential Gestalt principles within the stored representations, supporting a perception-alike hypothesis.

Mean size as a unit of visual working memory

Visual environments often contain multiple elements, some of which are similar to one another or spatially grouped together. In the current study we investigated how one can use perceptual groups in representing ensemble features of the groups. In experiment 1 we found that participants' performance improved when items were easily segmented by a grouping cue based on proximity, suggesting that spatial grouping facilitates extracting and remembering ensemble representations from visual arrays consisting of multiple elements. In experiment 2 we found that spatial grouping improved performance only when the grouped subsets were tested for the memory task, whereas it impaired performance when other subsets that were not grouped were tested, suggesting that the benefit from grouping may come from better extraction for storage, rather than later decision processes such as accessibility. Taken together, our results suggest that perceptual grouping of multiple items by proximity facilitates extraction of ensemble statistics from groups of items, enhancing visual memory of the ensembles in a visual array.

The Gestalt principle of similarity benefits visual working memory

Visual working memory (VWM) is essential for many cognitive processes, yet it is notably limited in capacity. Visual perception processing is facilitated by Gestalt principles of grouping, such as connectedness, similarity, and proximity. This introduces the question, do these perceptual benefits extend to VWM? If so, can this be an approach to enhance VWM function by optimizing the processing of information? Previous findings have demonstrated that several Gestalt principles (connectedness, common region, and spatial proximity) do facilitate VWM performance in change detection tasks (Jiang, Olson, & Chun, 2000; Woodman, Vecera, & Luck, 2003; Xu, 2002, 2006; Xu & Chun, 2007). However, one prevalent Gestalt principle, similarity, has not been examined with regard to facilitating VWM. Here, we investigated whether grouping by similarity benefits VWM. Experiment 1 established the basic finding that VWM performance could benefit from grouping. Experiment 2 replicated and extended this finding by showing that similarity was only effective when the similar stimuli were proximal. In short, the VWM performance benefit derived from similarity was constrained by spatial proximity, such that similar items need to be near each other. Thus, the Gestalt principle of similarity benefits visual perception, but it can provide benefits to VWM as well.

Hits and misses: leveraging tDCS to advance cognitive research

The popularity of non-invasive brain stimulation techniques in basic, commercial, and applied settings grew tremendously over the last decade. Here, we focus on one popular neurostimulation method: transcranial direct current stimulation (tDCS). Many assumptions regarding the outcomes of tDCS are based on the results of stimulating motor cortex. For instance, the primary motor cortex is predictably suppressed by cathodal tDCS or made more excitable by anodal tDCS. However, wide-ranging studies testing cognition provide more complex and sometimes paradoxical results that challenge this heuristic. Here, we first summarize successful efforts in applying tDCS to cognitive questions, with a focus on working memory (WM). These recent findings indicate that tDCS can result in cognitive task improvement or impairment regardless of stimulation site or direction of current flow. We then report WM and response inhibition studies that failed to replicate and/or extend previously reported effects. From these opposing outcomes, we present a series of factors to consider that are intended to facilitate future use of tDCS when applied to cognitive questions. In short, common pitfalls include testing too few participants, using insufficiently challenging tasks, using heterogeneous participant populations, and including poorly motivated participants. Furthermore, the poorly understood underlying mechanism for long-lasting tDCS effects make it likely that other important factors predict responses. In conclusion, we argue that although tDCS can be used experimentally to understand brain function its greatest potential may be in applied or translational research.

Object grouping based on real-world regularities facilitates perception by reducing competitive interactions in visual cortex

In virtually every real-life situation humans are confronted with complex and cluttered visual environments that contain a multitude of objects. Because of the limited capacity of the visual system, objects compete for neural representation and cognitive processing resources. Previous work has shown that such attentional competition is partly object based, such that competition among elements is reduced when these elements perceptually group into an object based on low-level cues. Here, using functional MRI (fMRI) and behavioral measures, we show that the attentional benefit of grouping extends to higher-level grouping based on the relative position of objects as experienced in the real world. An fMRI study designed to measure competitive interactions among objects in human visual cortex revealed reduced neural competition between objects when these were presented in commonly experienced configurations, such as a lamp above a table, relative to the same objects presented in other configurations. In behavioral visual search studies, we then related this reduced neural competition to improved target detection when distracter objects were shown in regular configurations. Control studies showed that low-level grouping could not account for these results. We interpret these findings as reflecting the grouping of objects based on higher-level spatial-relational knowledge acquired through a lifetime of seeing objects in specific configurations. This interobject grouping effectively reduces the number of objects that compete for representation and thereby contributes to the efficiency of real-world perception.

Local and global level-priming occurs for hierarchical stimuli composed of outlined, but not filled-in, elements

When attention is directed to the local or global level of a hierarchical stimulus, attending to that same scale of information is subsequently facilitated. This effect is called level-priming, and in its pure form, it has been dissociated from stimulus- or response-repetition priming. In previous studies, pure level-priming has been demonstrated using hierarchical stimuli composed of alphanumeric forms consisting of lines. Here, we test whether pure level-priming extends to hierarchical configurations of generic geometric forms composed of elements that can be depicted either outlined or filled-in. Interestingly, whereas hierarchical stimuli composed of outlined elements benefited from pure level-priming, for both local and global targets, those composed of filled-in elements did not. The results are not readily attributable to differences in spatial frequency content, suggesting that forms composed of outlined and filled-in elements are treated differently by attention and/or priming mechanisms. Because our results present a surprising limit on attentional persistence to scale, we propose that other findings in the attention and priming literature be evaluated for their generalizability across a broad range of stimulus classes, including outlined and filled-in depictions.

Building blocks of visual working memory: objects or Boolean maps?

The nature of the building blocks of information in visual working memory (VWM) is a fundamental issue that has not been well resolved. Most researchers take objects as the building blocks, although this perspective has received criticism. The objects could be physically separated ones (strict object hypothesis) or hierarchical objects created from separated individuals (broad object hypothesis). Meanwhile, a newly proposed Boolean map theory for visual attention suggests that Boolean maps may be the building blocks of VWM (Boolean map hypothesis); this perspective could explain many critical findings of VWM. However, no previous study has examined these hypotheses. We explored this issue by focusing on a critical point on which they make distinct predictions. We asked participants to remember two distinct objects (2-object), three distinct objects (3-object), or three objects with repeated information (mixed-3-object, e.g., one red bar and two green bars, green bars could be represented as one hierarchical object) and adopted contralateral delay activity (CDA) to tap into the maintenance phase of VWM. The mixed-3-object condition could generate two Boolean maps, three objects, or three objects most of the time (hierarchical objects are created in certain trials, retaining two objects). Simple orientations (Experiment 1) and colors (Experiments 2 and 3) were used as stimuli. Although the CDA of the mixed-3-object condition was slightly lower than that of the 3-object condition, no significant difference was revealed between them. Both conditions displayed significantly higher CDAs than the 2-object condition. These findings support the broad object hypothesis. We further suggest that Boolean maps might be the unit for retrieval/comparison in VWM.

Selection and storage of perceptual groups is constrained by a discrete resource in working memory

Perceptual grouping can lead observers to perceive a multielement scene as a smaller number of hierarchical units. Past work has shown that grouping enables more elements to be stored in visual working memory (WM). Although this may appear to contradict so-called discrete resource models that argue for fixed item limits in WM storage, it is also possible that grouping reduces the effective number of "items" in the display. To test this hypothesis, we examined how mnemonic resolution declined as the number of items to be stored increased. Discrete resource models predict that precision will reach a stable plateau at relatively early set sizes, because no further items can be stored once putative item limits are exceeded. Thus, we examined whether the precision by set size function was bilinear when storage was enhanced via perceptual grouping. In line with the hypothesis that each perceptual group counted as a single "item," precision still reached a clear plateau at a set size determined by the number of stored groups. Moreover, the maximum number of elements stored was doubled, and electrophysiological measures showed that selection and storage-related neural responses were the same for a single element and a multielement perceptual group. Thus, perceptual grouping allows more elements to be held in working memory while storage is still constrained by a discrete item limit.

Feature binding and attention in working memory: a resolution of previous contradictory findings

We aimed to resolve an apparent contradiction between previous experiments from different laboratories, using dual-task methodology to compare effects of a concurrent executive load on immediate recognition memory for colours or shapes of items or their colour-shape combinations. Results of two experiments confirmed previous evidence that an irrelevant attentional load interferes equally with memory for features and memory for feature bindings. Detailed analyses suggested that previous contradictory evidence arose from limitations in the way recognition memory was measured. The present findings are inconsistent with an earlier suggestion that feature binding takes place within a multimodal episodic buffer Baddeley, ( 2000 ) and support a subsequent account in which binding takes place automatically prior to information entering the episodic buffer Baddeley, Allen, & Hitch, ( 2011 ). Methodologically, the results suggest that different measures of recognition memory performance (A', d', corrected recognition) give a converging picture of main effects, but are less consistent in detecting interactions. We suggest that this limitation on the reliability of measuring recognition should be taken into account in future research so as to avoid problems of replication that turn out to be more apparent than real.

Binding targets' responses to distractors' locations: distractor response bindings in a location-priming task

Responses to target stimuli can be encoded together with distracting objects accompanying these targets into a single stimulus-response episode or a single event file. Repeating any object of such an episode can trigger the response encoded in this episode. Hence, repeating a distractor may retrieve the response given to the target that was accompanied by this distractor. In the present experiments, we analyzed whether the binding of target responses to the distractor can be generalized even to the location of a distractor. In two experiments, we used a location-based prime-probe task and found that repeating the location of a distractor triggered the response to the target that had previously been accompanied by a distractor in the repeated location, even if the identity of the distractor changed from the prime to the probe.

Shape and color conjunction stimuli are represented as bound objects in visual working memory

The integrated object view of visual working memory (WM) argues that objects (rather than features) are the building block of visual WM, so that adding an extra feature to an object does not result in any extra cost to WM capacity. Alternative views have shown that complex objects consume additional WM storage capacity so that it may not be represented as bound objects. Additionally, it was argued that two features from the same dimension (i.e., color-color) do not form an integrated object in visual WM. This led some to argue for a "weak" object view of visual WM. We used the contralateral delay activity (the CDA) as an electrophysiological marker of WM capacity, to test those alternative hypotheses to the integrated object account. In two experiments we presented complex stimuli and color-color conjunction stimuli, and compared performance in displays that had one object but varying degrees of feature complexity. The results supported the integrated object account by showing that the CDA amplitude corresponded to the number of objects regardless of the number of features within each object, even for complex objects or color-color conjunction stimuli.

What are the units of storage in visual working memory?

An influential theory suggests that integrated objects, rather than individual features, are the fundamental units that limit our capacity to temporarily store visual information (S. J. Luck & E. K. Vogel, 1997). Using a paradigm that independently estimates the number and precision of items stored in working memory (W. Zhang & S. J. Luck, 2008), here we show that the storage of features is not cost-free. The precision and number of objects held in working memory was estimated when observers had to remember either the color, the orientation, or both the color and orientation of simple objects. We found that while the quantity of stored objects was largely unaffected by increasing the number of features, the precision of these representations dramatically decreased. Moreover, this selective deterioration in object precision depended on the multiple features being contained within the same objects. Such fidelity costs were even observed with change detection paradigms when those paradigms placed demands on the precision of the stored visual representations. Taken together, these findings not only demonstrate that the maintenance of integrated features is costly; they also suggest that objects and features affect visual working memory capacity differently.

Distinctive neural mechanisms supporting visual object individuation and identification

Many everyday activities, such as driving on a busy street, require the encoding of distinctive visual objects from crowded scenes. Given resource limitations of our visual system, one solution to this difficult and challenging task is to first select individual objects from a crowded scene (object individuation) and then encode their details (object identification). Using functional magnetic resonance imaging, two distinctive brain mechanisms were recently identified that support these two stages of visual object processing. While the inferior intraparietal sulcus (IPS) selects a fixed number of about four objects via their spatial locations, the superior IPS and the lateral occipital complex (LOC) encode the features of a subset of the selected objects in great detail (object shapes in this case). Thus, the inferior IPS individuates visual objects from a crowded display and the superior IPS and higher visual areas participate in subsequent object identification. Consistent with the prediction of this theory, even when only object shape identity but not its location is task relevant, this study shows that object individuation in the inferior IPS treats four identical objects similarly as four objects that are all different, whereas object shape identification in the superior IPS and the LOC treat four identical objects as a single unique object. These results provide independent confirmation supporting the dissociation between visual object individuation and identification in the brain.