Perceptual grouping in change detection

Attentional prioritisation and facilitation for similar stimuli in visual working memory

Visual working memory (VWM) allows for the brief retention of approximately three to four items. Interestingly, when these items are similar to each other in a feature domain, memory recall performance is elevated compared to when they are dissimilar. This similarity benefit is currently not accounted for by models of VWM. Previous research has suggested that this similarity benefit may arise from selective attentional prioritisation in the maintenance phase. However, the similarity effect has not been contrasted under circumstances where dissimilar item types can adequately compete for memory resources. In Experiment 1, similarity benefits were seen for all-similar over all-dissimilar displays. This was also seen in mixed displays, change detection performance was higher when one of the two similar items changed, compared to when the dissimilar item changed. Surprisingly, the similarity effect was stronger in these mixed displays then when comparing the all-similar and all-dissimilar. Experiment 2 investigated this further by examining how attention was allocated in the memory encoding phase via eye movements. Results revealed that attention prioritised similar over dissimilar items in the mixed displays. Similar items were more likely to receive the first fixation and were fixated more often than dissimilar items. Furthermore, dwell times were elongated for dissimilar items, suggesting that encoding was less efficient. These results suggest that there is an attentional strategy towards prioritising similar items over dissimilar items, and that this strategy's influence can be observed in the memory encoding phase.

Stories in the Mind? The Role of Story-Based Categorizations in Motion Classification

Categorization is fundamental for spatial and motion representation in both the domain of artificial intelligence and human cognition. In this paper, we investigated whether motion categorizations designed in artificial intelligence can inform human cognition. More concretely, we investigated if such categorizations (also known as qualitative representations) can inform the psychological understanding of human perception and memory of motion scenes. To this end, we took two motion categorizations in artificial intelligence, Motion-RCC and Motion-OPRA1 , and conducted four experiments on human perception and memory. Participants viewed simple motion scenes and judged the similarity of transformed scenes with this reference scene. Those transformed scenes differed in none, one, or both Motion-RCC and Motion-OPRA1 categories. Importantly, we applied an equal absolute metric change to those transformed scenes, so that differences in the similarity judgments should be due only to differing categories. In Experiments 1a and 1b, where the reference stimulus and transformed stimuli were visible at the same time (perception), both Motion-OPRA1 and Motion-RCC influenced the similarity judgments, with a stronger influence of Motion-OPRA1 . In Experiments 2a and 2b, where participants first memorized the reference stimulus and viewed the transformed stimuli after a short blank (memory), only Motion-OPRA1 had marked influences on the similarity judgments. Our findings demonstrate a link between human cognition and these motion categorizations developed in artificial intelligence. We argue for a continued and close multidisciplinary approach to investigating the representation of motion scenes.

Disentangling Processing and Storage Accounts of Working Memory Development in Childhood

Researchers have been asking the question of what drives the development of working memory (WM) during childhood for decades. This question is particularly challenging because so many aspects of cognition develop with age that it is difficult to disentangle them and find out which factors are causal or fundamental. In this review, we first prepare to discuss this issue by inquiring whether increases in storage, processing, or both are the fundamental driving factor(s) of the age-related increase in WM capability in childhood. We contend that by experimentally manipulating either factor and observing changes in the other, it is possible to learn about causal roles in WM development. We discuss research on school-aged children that seems to suggest, by means of such an approach, that the growth of storage is causal for some phases or steps in WM tasks, but that the growth of processing is causal for other steps. In our theoretical proposal, storage capacity of the focus of attention determines earlier steps of information processing by constraining the selective encoding of information into WM, whereas processing dependent on the focus of attention determines later steps, like the detection of patterns that can simplify the effective memory load and adoption of a proactive stance of maintenance in dual-task settings. Future directions for research are discussed.

Processing spatial configurations in visuospatial working memory is influenced by shifts of overt visual attention

When memorizing multiple objects, humans process them in relation to each other, proposing a configuration benefit. Shifts in overt visual attention through eye movements might influence the processing of spatial configurations. Whereas some research suggests that overt visual attention aids the processing of spatial representations, other research suggests a snapshot-like processing of spatial configurations, thus likely not relying on eye movements. In the first experiment, we focused on the comparison between an enforced fixation and a free view condition regarding configurational effects. Participants encoded objects' locations and were asked for changes at retrieval. One object was displaced in half of the trials and was either accompanied by a configuration or was displayed alone. In the second experiment, we expanded this idea by enforcing fixation during different task phases, namely encoding, maintenance and retrieval. We investigated if a fixed gaze during one specific phase drives the influence of eye movements when processing spatial configurations. We observed reliable configuration benefits for the free view conditions. Whereas a fixed gaze throughout the whole trial reduced the effect, enforced fixations during the task phases did not break the configuration benefit. Our findings suggest that whereas the processing of spatial configurations in memory is supported by the ability of performing shifts of overt visual attention, configurational processing does not rely on these shifts occurring throughout the task. Our results indicate a reciprocal relationship of visuospatial working memory and eye movements.

Perceptual bias contextualized in visually ambiguous stimuli

The visual appearance of an object is a function of stimulus properties as well as perceptual biases imposed by the observer. The context-specific trade-off between both can be measured accurately in a perceptual judgment task, involving grouping by proximity in ambiguous dot lattices. Such grouping depends lawfully on a stimulus parameter of the dot lattices known as their aspect ratio (AR), whose effect is modulated by a perceptual bias representing the preference for a cardinal orientation. In two experiments, we investigated how preceding context can lead to bias modulation, either in a top-down fashion via visual working memory (VWM) or bottom-up via sensory priming. In Experiment 1, we embedded the perceptual judgment task in a change detection paradigm and studied how the factors of VWM load (complexity of the memory array) and content (congruency in orientation to the ensuing dot lattice) affect the prominence of perceptual bias. A robust vertical orientation bias was observed, which was increased by VWM load and modulated by congruent VWM content. In Experiment 2, dot lattices were preceded by oriented primes. Here, primes regardless of orientation elicited a vertical orientation bias in dot lattices compared to a neutral baseline. Taken together, the two experiments demonstrate that top-down context (VWM load and content) effectively controls orientation bias modulation, while bottom-up context (i.e., priming) merely acts as an undifferentiated trigger to perceptual bias. These findings characterize the temporal context sensitivity of Gestalt perception, shed light on the processes responsible for different perceptual outcomes of ambiguous stimuli, and identify some of the mechanisms controlling perceptual bias.

Change-detection training and its effects on visual processing skills

Previous cognitive training research with the change-detection paradigm found only sparse effects that went beyond improvements in the training task but stressed an increase in fidelity of internal memory representations. Motivated by the demanding visual processing requirements of change-detection training, we extended this work by focusing on whether training on a change-detection task would improve visual processing skills. Fifty participants were randomly assigned to train on a change-detection task or on a control task for seven sessions. Participants’ visual processing skills were assessed before and after the intervention, focusing on visual search, contrast sensitivity, and contour integration. Our results suggest a general improvement in perceptual skills that was primarily driven by a conjunction search task and to a much lesser extent by a complex visual search task and a contrast sensitivity task. The data from the conjunction search task further suggest a causal link between training and improvements of perceptual as opposed to attentional processes. Since the change-detection paradigm is commonly used to assess working memory capacity, future research needs to investigate how much of its variance is explained by memory performance and how much is explained by perceptual processes.

Benefits and pitfalls of data compression in visual working memory

Data compression in memory is a cognitive process allowing participants to cope with complexity to reduce information load. However, previous studies have not yet considered the hypothesis that this process could also lead to over-simplifying information due to haphazard amplification of the compression process itself. For instance, we could expect that the over-regularized features of a visual scene could produce false recognition of patterns, not because of storage capacity limits but because of an errant compression process. To prompt memory compression in our participants, we used multielement visual displays for which the underlying information varied in compressibility. The compressibility of our material could vary depending on the number of common features between the multi-dimensional objects in the displays. We measured both accuracy and response times by probing memory representations with probes that we hypothesized could modify the participants' representations. We confirm that more compressible information facilitates performance, but a more novel finding is that compression can produce both typical memory errors and lengthened response times. Our findings provide clearer evidence of the forms of compression that participants carry out.

Attention effects in working memory that are asymmetric across sensory modalities

A key unanswered question about working memory is the nature of interference between items. At one extreme of existing theories, interference occurs between any two items because of a general capacity limit. At another extreme, interference depends on the similarity between particular features of different items. We examine this question in three experiments by presenting two sets of items on each trial, comprising tones or colors, with three levels of similarity between the two sets: cross-modal, unimodal with different marking features (two different musical instruments or shapes), and unimodal with the same marking feature. Another question is the extent to which the entry of presented items into working memory is obligatory or optional, which we examined by requiring retention of the first, the second, or both sets of stimuli for a recognition test shortly after the presentation of the two sets. The combination of the set similarity and attention manipulations allows us to draw conclusions about the nature of working-memory storage. The findings were not entirely in accord with any pre-existing theory. The effects of feature similarity were present in both modalities but more pronounced for sounds, whereas the detrimental effects of attention to both sets for retention occurred only for visual stimuli. Based on the findings we suggest a new, hybrid conception of working memory storage.

A compressibility account of the color-sharing bonus in working memory

It has been established that objects sharing color in a visual display can boost working memory. The capacity to encode singletons particularly benefits from the repetition of colors encoded as perceptual groups. We manipulated the algorithmic complexity of visual displays to test whether compressibility of information could account for the color-sharing bonus. This study used a free recall working memory task in which the participants were shown displays of 2 to 8 color items. We examined the influence of set size, complexity, number of same-color clusters and amount of color redundancy. The results showed that the probability of correct recall of the pattern and the proportion of similarity between the pattern and the response decreased with an increase of each manipulated variable, except for color redundancy in terms of probability of correct recall. The model performance of complexity did not differ from that of clusters, but complexity was found more accurate than either set size or color redundancy. The results also showed that similar items were more often recalled adjacently, and complexity correlated strongly with the number of extra color repetitions in the response, suggesting that more complex patterns encouraged the use of information compression. Moreover, color repetitions were more often recalled first and the probability of correct recall for singletons and sub-patterns could be predicted by the compressibility measure. We discuss the potential advantage of using compressibility measures to capture the effects of regularities in visual patterns, in particular to refine analysis of the color-sharing bonus.

(Re-)organisation of spatial configurations in visual working memory: The fate of objects rendered relevant or irrelevant by selective attention

People maintain object locations not as independent absolute positions but based on inter-object relations in the form of a spatial configuration in visual working memory (VWM). Initial evidence suggests that spatial configurations can be reorganised during maintenance; however, this mechanism is not well understood. We report three experiments investigating this reorganisation process. First, we found that directing spatial attention with a retro cue triggers a reorganisation of spatial configurations during maintenance (Experiment 1). Second, we investigated the role of contextual objects rendered either relevant or irrelevant through the retro cue by manipulating the locations of the context at retrieval both within a partial display (Experiment 2a) and a whole display (Experiment 2b). Whereas the similar impairment of memory performance by changing the relevant and global context suggests a complete reorganisation of spatial configurations in VWM, this interpretation was challenged by the observation of impaired memory performance with changes of the irrelevant objects in a whole display that retains the relevant locations. Thus, we suggest that reorganisation should be considered to be the formation of a partial configuration based on the objects rendered relevant by the retro cue in addition to the originally encoded global configuration, with both configurations affecting memory performance.

Charting the Diversity of Strategic Processes in Visuospatial Short-Term Memory

Despite the abundant literature on visuospatial short-term memory, researchers have devoted little attention to strategic processes: What procedures do subjects implement to memorize visuospatial material? Evidence for various strategies exists, but it is spread across a variety of fields. This integrative review of the literature brings together scattered evidence to provide an overview of strategic processes in visuospatial memory tasks. The diversity of strategies and their proposed operating mechanisms are reviewed and discussed. The evidence leads to proposing seven broad strategic processes used in visuospatial short-term memory, each with multiple variants. Strategies can vary across individuals, but the same subjects also appear to use multiple strategies depending on the perceptual features of to-be-remembered displays. These results point to a view of visuospatial strategies as a functional library of facilitatory processes on which subjects can draw to support visuospatial short-term memory performance. Implications are discussed for the difference between visual and spatial tasks, for the appropriate measurement of strategic behaviors, and for the interpretation of performance in visuospatial memory tasks.

Working memory for stereoscopic depth is limited and imprecise-evidence from a change detection task

Most studies on visual working memory (VWM) and spatial working memory (SWM) have employed visual stimuli presented at the fronto-parallel plane and few have involved depth perception. VWM is often considered as a memory buffer for temporarily holding and manipulating visual information that relates to visual features of an object, and SWM for holding and manipulating spatial information that concerns the spatial location of an object. Although previous research has investigated the effect of stereoscopic depth on VWM, the question of how depth positions are stored in working memory has not been systematically investigated, leaving gaps in the existing literature on working memory. Here, we explore working memory for depth by using a change detection task. The memory items were presented at various stereoscopic depth planes perpendicular to the line of sight, with one item per depth plane. Participants were asked to make judgments on whether the depth position of the target (one of the memory items) had changed. The results showed a conservative response bias that observers tended to make 'no change' responses when detecting changes in depth. In addition, we found that similar to VWM, the change detection accuracy degraded with the number of memory items presented, but the accuracy was much lower than that reported for VWM, suggesting that the storage for depth information is severely limited and less precise than that for visual information. The detection sensitivity was higher for the nearest and farthest depths and was better when the probe was presented along with the other items originally in the memory array, indicating that how well the to-be-stored depth can be stored in working memory depends on its relation with the other depth positions.

Composite Face Effect Predicts Configural Encoding in Visual Short-Term Memory

In natural vision, visual scenes consist of individual items (e.g., trees) and global properties of items as a whole (e.g., forest). These different levels of representations can all contribute to perception, natural scene understanding, sensory memory, working memory, and long-term memory. Despite these various hierarchical representations across perception and cognition, the nature of the global representations has received considerably less attention in empirical research on working memory than item representations. The present study aimed to understand the perceptual root of the configural information retained in Visual Short-term Memory (VSTM). Specifically, we assessed whether configural VSTM was related to holistic face processing across participants using an individual differences approach. Configural versus item encoding in VSTM was assessed using Xie and Zhang’s (2017) dual-trace Signal Detection Theory model in a change detection task for orientation. Configural face processing was assessed using Le Grand composite face effect (CFE). In addition, overall face recognition was assessed using Glasgow Face Matching Test (GFMT). Across participants, holistic face encoding, but not face recognition accuracy, predicted configural information, but not item information, retained in VSTM. Together, these findings suggest that configural encoding in VSTM may have a perceptual root.

Reorganization of spatial configurations in visual working memory: A matter of set size?

Humans process single objects in relation to other simultaneously maintained objects in visual working memory. This interdependence is called spatial configuration. Humans are able to reorganize global spatial configurations into relevant partial configurations. We conducted three experiments investigating the process underlying reorganization by manipulating memory set size and the presence of configurations at retrieval. Participants performed a location change detection task for a single object probed at retrieval. At the beginning of each trial, participants memorized the locations of all objects (set size: 4, 8, 12, or 16). During maintenance, a valid retro cue highlighted the side containing the object probed at retrieval, thus enabling participants to reorganize the memorized global spatial configuration to the partial cued configuration. At retrieval, the object probed was shown together with either all objects (complete configuration; Experiment 1a), the cued objects only (congruent configuration; all Experiments), the non-cued objects only (incongruent configuration, all Experiments) or alone (no configuration; Experiment 1b). We observed reorganization of spatial configurations as indicated by a superior location change detection performance with a congruent partial configuration than an incongruent partial configuration across all three experiments. We also observed an overall decrease in accuracy with increasing set size. Most importantly, however, we did not find evidence for a reliable impairment of reorganization with increasing set size. We discuss these findings with regard to the memory representation underlying spatial configurations.

Reorganization of spatial configurations in visual working memory

Human beings have to constantly process multiple objects in visual working memory (VWM). Positional relations to other objects known as spatial configurations contribute significantly to the organization of information in VWM. The aim of our study was to clarify whether spatial configurations can be reorganized to a subset of objects during maintenance. Participants were shown an array of objects, and afterwards the objects disappeared. A valid cue was shown either during encoding or maintenance to highlight the side of the following probed object. Afterwards, the objects reappeared and participants were instructed to detect whether or not a particular object changed its location. We manipulated the configurations at retrieval regarding the number of objects, ranging from all objects to a single object. Our first and second experiment investigated reorganization for a number of six and 12 objects, respectively. In the third experiment, we used a retro cue only and manipulated eye movements (free view vs. enforced fixation). While showing that reorganization of spatial configurations during maintenance is possible in principle, we found some boundary conditions. There was no spatial configuration effect when participants had to fixate. Thus, eye movements are required for a configuration effect to occur.

Effects of changing object identity on location working memory

It is widely accepted that features and locations are represented independently in an initial stage of visual processing. But to what degree are they represented separately at a later stage, after objects enter visual working memory (VWM)? In one of her last studies on VWM, Treisman raised an open question about how people represent locations in VWM, suggesting that locations may be remembered independently of what occupies them. Using photographs of real-world objects, we tested the independence of location memory from object identity in a location change detection task. We introduced changes to object identities between the encoding and test arrays, but instructed participants to treat the objects as placeholders. Three experiments showed that location memory was disrupted when the placeholders changed shape or orientation. The disruption was more noticeable for elongated than for round placeholders and was comparable between real-world objects and rectangles of similar aspect ratio. These findings suggest that location representation is sensitive to the placeholders' geometric properties. Though they contradict the idea that objects are just placeholders in location working memory (WM), the findings support Treisman's proposal that the items in VWM are bound to the global configuration of the memory array.

Saturation and brightness modulate the effect of depth on visual working memory

Although previous studies show inconsistent results regarding the effect of depth perception on visual working memory (VWM), a recent finding shows that perceptually closer-in-depth items are better remembered than farther items when combining the congruent disparity and relative size cues. In this study, we employed a similar change detection paradigm to investigate the effects of saturation and brightness, alone or in combination with binocular disparity, on VWM. By varying the appearance of the memory items, we aimed to manipulate the visual salience as well as to simulate the aerial perspective cue that induces depth perception. We found that the change detection accuracy was significantly improved for brighter and more saturated items, but not for items solely with higher saturation. Additionally, combining saturation with the congruent disparity cue significantly improved memory performance for perceptually closer items over farther items. Conflicting the disparity cue with saturation eliminated the memory benefit for the closer items. These results indicate that saturation and brightness could modulate the effect of depth on VWM, and both visual salience and depth perception affect VWM possibly through a common underlying mechanism of setting priority for attentional selection.

Visual short-term memory through the lifespan: Preserved benefits of context and metacognition

Visual short-term memory (VSTM) ability falls throughout the life span in healthy adults. Using a continuous report task, in a large, population-based sample, we first confirmed that this decline affects the quality and quantity of reported memories as well as knowledge of which item went where. Visual and sensorimotor precision also worsened with advancing age, but this did not account for the reduced memory performance. We then considered two strategies that older individuals might be able to adopt, to offset these memory declines: the use of contextual encoding, and metacognitive monitoring of performance. Context and metacognitive awareness were both associated with significantly better performance, however these effects did not interact with age in our sample. This suggests that older adults retain their capacity to boost memory performance through attention to external context and monitoring of their performance. Strategies that focus on taking advantage of these preserved abilities may therefore help to maintain VSTM performance with advancing age. The article reports on analysis of the Cambridge Centre for Ageing and Neuroscience (Cam-CAN) data. (PsycINFO Database Record

Task-dependent effects of voluntary space-based and involuntary feature-based attention on visual working memory

Previous research has shown that visual working memory (VWM) can be modulated by space-based or feature-based attentional selection. However, it remains unclear how the two modes of attention operate jointly to affect VWM, and in particular, if involuntary feature-based attention plays a role in VWM. In this study, a pre-cued change detection paradigm was employed to investigate the concurrent effects of space- and feature-based attention on VWM. Space-based attention was manipulated by informative spatial cueing and by varying the proximity between the test item and the cued (fixated) memory item, while feature-based attention was induced in an involuntary manner by having the test item to share the same color or shape with the cued item on a fraction of trials. The results showed that: (1) the memory performance for the cued items was always better than the uncued items, suggesting a beneficial effect of voluntary spatial attention; (2) with a brief duration of the memory array (250 ms), cue-test proximity benefited VWM in the shape judgment task but not in the color judgment task, whereas with a longer duration (1200 ms), no proximity effect was found for either task; (3) VWM was improved for the same-colored items regardless of the task and duration; (4) VWM was improved for the same-shaped items only in the shape judgment task with the longer duration of the memory array. A discrimination task further showed that the proximity effect associated with VWM reflects a perceptual bottleneck in memory encoding for shape but not for color with a brief display. Our results suggest that involuntary feature-based attention could be triggered by spatial cueing to modulate VWM; involuntary color-based attention facilitates VWM independently of task, whereas shape-based facilitation is task-dependent, i.e., confined only to the shape judgment task, presumably reflecting different attention-guiding potencies of the two features.

Evidence for the beneficial effect of perceptual grouping on visual working memory: an empirical study on illusory contour and a meta-analytic study

The capacity of visual working memory (VWM) is found to be extremely limited. Past research shows that VWM can be facilitated by Gestalt principles of grouping, however, it remains controversial whether factors like the type of Gestalt principles, the characteristics of stimuli and the nature of experimental design could affect the beneficial effect of grouping. In particular, studies have shown that perceptual grouping could improve memory performance for a feature that is relevant for grouping, but it is unclear whether the same improvement exists for a feature that is irrelevant for grouping. In this article, an empirical study and a meta-analytic study were conducted to investigate the effect of perceptual grouping on VWM. In the empirical study, we examined the grouping effect by employing a Kanizsa illusion in which memory items were grouped by illusory contour. We found that the memory performance was improved for the grouped items even though the tested feature was grouping irrelevant, and the improvement was not significantly different from the effect of grouping by physical connectedness or by solid occlusion. In the meta-analytic study, we systematically and quantitatively examined the effect of perceptual grouping on VWM by pulling the results from all eligible studies, and found that the beneficial grouping effect was robust but the magnitude of the effect can be affected by several moderators. Factors like the types of grouping methods, the duration and the layout of the memory display, and the characteristics of the tested feature moderated the grouping effect, whereas whether employing a cue or a verbal suppression task did not. Our study suggests that the underlying mechanism of the grouping benefit may be distinct with regard to grouping relevancy of the to-be-stored feature. The grouping effect on VWM may be independent of attention for a grouping relevant feature, but may rely on attentional prioritization for a grouping irrelevant feature.

Relational encoding of objects in working memory: Change detection performance is better for violations in group relations

Research has indicated that working memory is based on forming relations between individual elements. In this study, we considered the congruency of object clusters during a change detection task. We demonstrate that changes which violate the relational encoding of a probe display (single-object changes where one object shifts independently from its corresponding group) are more easily detected than changes that maintain group structure (cluster changes where all objects in the group shift in location together)–despite cluster changes involving more objects moving overall. We explore this effect across interactions with direction of single-object movement (distancing from the cluster vs. uniting with the cluster) and trial order, demonstrating that naïve participants improve at a faster rate on single-object changes than cluster changes. It is concluded that storage in working memory functions by building relational bindings between objects and their place within the chunk, rather than by binding objects to their spatial location.

Change perception and change interference within and across feature dimensions

The ability to perceive a change in a visual object is reduced when that change is presented in competition with other changes which are task-irrelevant. We performed two experiments which investigate the basis of this change interference effect. We tested whether change interference occurs as a consequence of some form of attentional capture, or whether the interference occurs at a stage prior to attentional selection of the task-relevant change. A modified probe-detection task was used to explore this issue. Observers were required to report the presence/absence of a specified change-type (colour, shape) in the probe, in a context in which - on certain trials - irrelevant changes occur in non-probe items. There were two key variables in these experiments: the attentional state of the observer, and the dimensional congruence of changes in the probe and non-probe items. Change interference was strongest when the irrelevant changes were the same as those on the report dimension. However the interference pattern persisted even when observers did not know the report dimension at the time the changes occurred. These results seem to rule out attention as a factor. Our results fit best with an interpretation in which change interference produces feature-specific sensory noise which degrades the signal quality of the target change.

A configural effect in visual short-term memory for features from different parts of an object

Previous studies have shown that change detection performance is improved when the visual display holds features (e.g., a colour and an orientation) that are grouped into different parts of the same object compared to when they are all spatially separated (Xu, 2002a, 2002b). These findings indicate that visual short-term memory (VSTM) encoding can be “object based”. Recently, however, it has been demonstrated that changing the orientation of an item could affect the spatial configuration of the display (Jiang, Chun, & Olson, 2004), which may have an important influence on change detection. The perceptual grouping of features into an object obviously reduces the amount of distinct spatial relations in a display and hence the complexity of the spatial configuration. In the present study, we ask whether the object-based encoding benefit observed in previous studies may reflect the use of configural coding rather than the outcome of a true object-based effect. The results show that when configural cues are removed, the object-based encoding benefit remains for features (i.e., colour and orientation) from different parts of an object, but is significantly reduced. These findings support the view that memory for features from different parts of an object can benefit from object-based encoding, but the use of configural coding significantly helps enlarge this effect.

Development of the ability to combine visual and acoustic information in working memory

Presentation of two kinds of materials in working memory (visual and acoustic), with the requirement to attend to one or both modalities, poses an interesting case for working memory development because competing predictions can be formulated. In two experiments, we assessed such predictions with children 7-13 years old and adults. With development, the ability to hold more information in the focus of attention could lead to an increase in the size of the trade-off between modalities; if attention can hold A items during unimodal-attention trials, then on average attention should hold A/2 of those same items during bimodal-attention trials. If A increases with age, so would the dual-task cost, A/2. The results clearly ruled out that possibility. It was the modality- or code-specific components of working memory that improved with age and not the central component. We discuss various mechanisms that could have produced these results, including alternative attention-based mechanisms. The findings point to a rich field for continued research.

The role of alpha oscillations in deriving and maintaining spatial relations in working memory

Previous research has demonstrated distinct neural correlates for maintenance of abstract, relational versus concrete, sensory information in working memory (WM). Storage of spatial relations in WM results in suppression of posterior sensory regions, which suggests that sensory information is task-irrelevant when relational representations are maintained in WM. However, the neural mechanisms by which abstract representations are derived from sensory information remain unclear. Here, using electroencephalography, we investigated the role of alpha oscillations in deriving spatial relations from a sensory stimulus and maintaining them in WM. Participants encoded two locations into WM, then after an initial maintenance period, a cue indicated whether to convert the spatial information to another sensory representation or to a relational representation. Results revealed that alpha power increased over posterior electrodes when sensory information was converted to a relational representation, but not when the information was converted to another sensory representation. Further, alpha phase synchrony between posterior and frontal regions increased for relational compared to sensory trials during the maintenance period. These results demonstrate that maintaining spatial relations and locations in WM rely on distinct neural oscillatory patterns.

Evidence for the effect of depth on visual working memory

Visual working memory (VWM) is a cognitive memory buffer for temporarily holding, processing, and manipulating visual information. Previous studies have demonstrated mixed results of the effect of depth perception on VWM, with some showing a beneficial effect while others not. In this study, we employed an adapted change detection paradigm to investigate the effects of two depth cues, binocular disparity and relative size. The memory array consisted of a set of pseudo-randomly positioned colored items, and the task was to judge whether the test item was changed compared to the memory item after a retention interval. We found that presenting the items in stereoscopic depth alone hardly affected VWM performance. When combining the two coherent depth cues, a significant larger VWM capacity of the perceptually closer-in-depth items was observed than that of the farther items, but the capacity for the two-depth-planes condition was not significantly different from that for the one-plane condition. Conflicting the two depth cues resulted in cancelling the beneficial effect of presenting items at a closer depth plane. The results indicate that depth perception could affect VWM, and the visual system may have an advantage in maintaining closer-in-depth objects in working memory.

Dissociating models of visual working memory by reaction-time distribution analysis

There have been heated debates on whether visual working memory (VWM) represents information in discrete-slots or a reservoir of flexible-resources. However, one key aspect of the models has gone unnoticed, the speed of processing when stored information in memory is assessed for accuracy. The present study evaluated contrasting predictions from the two models regarding the change detection decision times spent on the assessment of stored information by estimating the ex-Gaussian parameters from change detection RT distributions across different set sizes (2, 4, 6, or 8). The estimation showed that the Gaussian components μ and σ became larger as the set size increased from 2 to 4, but stayed constant as it reached 6 and 8, with an exponential component τ increasing at above-capacity set sizes. Moreover, we found that an individual's capacity limit correlates with the memory set size where the Gaussian μ reaches a plateau. These results indicate that the decision time for assessing in-memory items is constant regardless of memory set sizes whereas the time for the remaining not-in-memory items increases as the set size exceeds VWM storage capacity. The findings suggest that the discrete-slot model explains the observed RT distributions better than the flexible-resource model.

The Whole Warps the Sum of Its Parts

The efficiency of averaging properties of sets without encoding redundant details is analogous to gestalt proposals that perception is parsimoniously organized as a function of recurrent order in the world. This similarity suggests that grouping and averaging are part of a broader set of strategies allowing the visual system to circumvent capacity limitations. To examine how gestalt grouping affects the manner in which information is averaged and remembered, I compared the error in observers' adjustments of remembered sizes of individual circles in two different mean-size sets defined by similarity, proximity, connectedness, or a common region. Overall, errors were more similar within the same gestalt-defined groups than between different gestalt-defined groups, such that the remembered sizes of individual circles were biased toward the mean size of their respective gestalt-defined groups. These results imply that gestalt grouping facilitates perceptual averaging to minimize the error with which individual items are encoded, thereby optimizing the efficiency of visual short-term memory.

Contralateral delay activity tracks the influence of Gestalt grouping principles on active visual working memory representations

Recent studies have demonstrated that factors influencing perception, such as Gestalt grouping cues, can influence the storage of information in visual working memory (VWM). In some cases, stationary cues, such as stimulus similarity, lead to superior VWM performance. However, the neural correlates underlying these benefits to VWM performance remain unclear. One neural index, the contralateral delay activity (CDA), is an event-related potential that shows increased amplitude according to the number of items held in VWM and asymptotes at an individual's VWM capacity limit. Here, we applied the CDA to determine whether previously reported behavioral benefits supplied by similarity, proximity, and uniform connectedness were reflected as a neural savings such that the CDA amplitude was reduced when these cues were present. We implemented VWM change-detection tasks with arrays including similarity and proximity (Experiment 1); uniform connectedness (Experiments 2a and 2b); and similarity/proximity and uniform connectedness (Experiment 3). The results indicated that when there was a behavioral benefit to VWM, this was echoed by a reduction in CDA amplitude, which suggests more efficient processing. However, not all perceptual grouping cues provided a VWM benefit in the same measure (e.g., accuracy) or of the same magnitude. We also found unexpected interactions between cues. We observed a mixed bag of effects, suggesting that these powerful perceptual grouping benefits are not as predictable in VWM. The current findings indicate that when grouping cues produce behavioral benefits, there is a parallel reduction in the neural resources required to maintain grouped items within VWM.

Glucose improves object-location binding in visual-spatial working memory

RATIONALE

There is evidence that glucose temporarily enhances cognition and that processes dependent on the hippocampus may be particularly sensitive. As the hippocampus plays a key role in binding processes, we examined the influence of glucose on memory for object-location bindings.

OBJECTIVE

This study aims to study how glucose modifies performance on an object-location memory task, a task that draws heavily on hippocampal function.

METHODS

Thirty-one participants received 30 g glucose or placebo in a single 1-h session. After seeing between 3 and 10 objects (words or shapes) at different locations in a 9 × 9 matrix, participants attempted to immediately reproduce the display on a blank 9 × 9 matrix. Blood glucose was measured before drink ingestion, mid-way through the session, and at the end of the session.

RESULTS

Glucose significantly improves object-location binding (d = 1.08) and location memory (d = 0.83), but not object memory (d = 0.51). Increasing working memory load impairs object memory and object-location binding, and word-location binding is more successful than shape-location binding, but the glucose improvement is robust across all difficulty manipulations. Within the glucose group, higher levels of circulating glucose are correlated with better binding memory and remembering the locations of successfully recalled objects.

CONCLUSIONS

The glucose improvements identified are consistent with a facilitative impact on hippocampal function. The findings are discussed in the context of the relationship between cognitive processes, hippocampal function, and the implications for glucose's mode of action.

Organization principles in visual working memory: Evidence from sequential stimulus display

Although the mechanisms of visual working memory (VWM) have been studied extensively in recent years, the active property of VWM has received less attention. In the current study, we examined how VWM integrates sequentially presented stimuli by focusing on the role of Gestalt principles, which are important organizing principles in perceptual integration. We manipulated the level of Gestalt cues among three or four sequentially presented objects that were memorized. The Gestalt principle could not emerge unless all the objects appeared together. We distinguished two hypotheses: a perception-alike hypothesis and an encoding-specificity hypothesis. The former predicts that the Gestalt cue will play a role in information integration within VWM; the latter predicts that the Gestalt cue will not operate within VWM. In four experiments, we demonstrated that collinearity (Experiment 1) and closure (Experiment 2) cues significantly improved VWM performance, and this facilitation was not affected by the testing manner (Experiment 3) or by adding extra colors to the memorized objects (Experiment 4). Finally, we re-established the Gestalt cue benefit with similarity cues (Experiment 5). These findings together suggest that VWM realizes and uses potential Gestalt principles within the stored representations, supporting a perception-alike hypothesis.

Detection of the number of changes in a display in working memory

Here we examine a new task to assess working memory for visual arrays in which the participant must judge how many items changed from a studied array to a test array. As a clue to processing, on some trials in the first 2 experiments, participants carried out a metamemory judgment in which they were to decide how many items were in working memory. Trial-to-trial fluctuations in these working memory storage judgments correlated with performance fluctuations within an individual, indicating a need to include trial-to-trial variation within capacity models (through either capacity fluctuation or some other attention parameter). Mathematical modeling of the results achieved a good fit to a complex pattern of results, suggesting that working memory capacity limits can apply even to judgments that involve an entire array rather than just a single item that may have changed, thus providing the expected conscious access to at least some of the contents of working memory.

Independent sources of anisotropy in visual orientation representation: a visual and a cognitive oblique effect

The representation of visual orientation is more accurate for cardinal orientations compared to oblique, and this anisotropy has been hypothesized to reflect a low-level visual process (visual, "class 1" oblique effect). The reproduction of directional and orientation information also leads to a mean error away from cardinal orientations or directions. This anisotropy has been hypothesized to reflect a high-level cognitive process of space categorization (cognitive, "class 2," oblique effect). This space categorization process would be more prominent when the visual representation of orientation degrades such as in the case of working memory with increasing cognitive load, leading to increasing magnitude of the "class 2" oblique effect, while the "class 1" oblique effect would remain unchanged. Two experiments were performed in which an array of orientation stimuli (1-4 items) was presented and then subjects had to realign a probe stimulus within the previously presented array. In the first experiment, the delay between stimulus presentation and probe varied, while in the second experiment, the stimulus presentation time varied. The variable error was larger for oblique compared to cardinal orientations in both experiments reproducing the visual "class 1" oblique effect. The mean error also reproduced the tendency away from cardinal and toward the oblique orientations in both experiments (cognitive "class 2" oblique effect). The accuracy or the reproduced orientation degraded (increasing variable error) and the cognitive "class 2" oblique effect increased with increasing memory load (number of items) in both experiments and presentation time in the second experiment. In contrast, the visual "class 1" oblique effect was not significantly modulated by any one of these experimental factors. These results confirmed the theoretical predictions for the two anisotropies in visual orientation reproduction and provided support for models proposing the categorization of orientation in visual working memory.

Emotional Arousal and Memory Binding: An Object-Based Framework

Binding various features of an event together and maintaining these connections in memory is an essential component of episodic memories. Previous theories make contradictory predictions about the effects of emotional arousal on memory binding. In this article, I review evidence for both arousal-impaired and arousal-enhanced memory binding and explain these contradictory findings using an object-based framework. According to this framework, emotionally arousing objects attract attention that enhances binding of their constituent features. In contrast, the emotional arousal associated with one object either impairs or has no effect on the associations between that object and other distinct objects or background contextual information. After initial encoding, the attention-grabbing nature of emotionally arousing objects can lead to interference in working memory, making it more difficult to maintain other bound representations. These contrasting effects of arousal on memory binding should help predict which aspects of emotional memories are likely to be accurate and which aspects are likely to be misremembered.

Visual memory performance for color depends on spatiotemporal context

Performance on visual short-term memory for features has been known to depend on stimulus complexity, spatial layout, and feature context. However, with few exceptions, memory capacity has been measured for abruptly appearing, single-instance displays. In everyday life, objects often have a spatiotemporal history as they or the observer move around. In three experiments, we investigated the effect of spatiotemporal history on explicit memory for color. Observers saw a memory display emerge from behind a wall, after which it disappeared again. The test display then emerged from either the same side as the memory display or the opposite side. In the first two experiments, memory improved for intermediate set sizes when the test display emerged in the same way as the memory display. A third experiment then showed that the benefit was tied to the original motion trajectory and not to the display object per se. The results indicate that memory for color is embedded in a richer episodic context that includes the spatiotemporal history of the display.

Central and peripheral components of working memory storage

This study reexamines the issue of how much of working memory storage is central, or shared across sensory modalities and verbal and nonverbal codes, and how much is peripheral, or specific to a modality or code. In addition to the exploration of many parameters in 9 new dual-task experiments and reanalysis of some prior evidence, the innovations of the present work compared to previous studies of memory for 2 stimulus sets include (a) use of a principled set of formulas to estimate the number of items in working memory and (b) a model to dissociate central components, which are allocated to very different stimulus sets depending on the instructions, from peripheral components, which are used for only 1 kind of material. We consistently find that the central contribution is smaller than was suggested by Saults and Cowan (2007) and that the peripheral contribution is often much larger when the task does not require the binding of features within an object. Previous capacity estimates are consistent with the sum of central plus peripheral components observed here. We consider the implications of the data as constraints on theories of working memory storage and maintenance.

Remembering complex objects in visual working memory: do capacity limits restrict objects or features?

Visual working memory stores stimuli from our environment as representations that can be accessed by high-level control processes. This study addresses a longstanding debate in the literature about whether storage limits in visual working memory include a limit to the complexity of discrete items. We examined the issue with a number of change-detection experiments that used complex stimuli that possessed multiple features per stimulus item. We manipulated the number of relevant features of the stimulus objects in order to vary feature load. In all of our experiments, we found that increased feature load led to a reduction in change-detection accuracy. However, we found that feature load alone could not account for the results but that a consideration of the number of relevant objects was also required. This study supports capacity limits for both feature and object storage in visual working memory.

The Gestalt principle of similarity benefits visual working memory

Visual working memory (VWM) is essential for many cognitive processes, yet it is notably limited in capacity. Visual perception processing is facilitated by Gestalt principles of grouping, such as connectedness, similarity, and proximity. This introduces the question, do these perceptual benefits extend to VWM? If so, can this be an approach to enhance VWM function by optimizing the processing of information? Previous findings have demonstrated that several Gestalt principles (connectedness, common region, and spatial proximity) do facilitate VWM performance in change detection tasks (Jiang, Olson, & Chun, 2000; Woodman, Vecera, & Luck, 2003; Xu, 2002, 2006; Xu & Chun, 2007). However, one prevalent Gestalt principle, similarity, has not been examined with regard to facilitating VWM. Here, we investigated whether grouping by similarity benefits VWM. Experiment 1 established the basic finding that VWM performance could benefit from grouping. Experiment 2 replicated and extended this finding by showing that similarity was only effective when the similar stimuli were proximal. In short, the VWM performance benefit derived from similarity was constrained by spatial proximity, such that similar items need to be near each other. Thus, the Gestalt principle of similarity benefits visual perception, but it can provide benefits to VWM as well.

Viewpoint-dependent representation of contextual information in visual working memory

Objects are not represented individually in visual working memory (VWM), but in relation to the contextual information provided by other memorized objects. We studied whether the contextual information provided by the spatial configuration of all memorized objects is viewpoint-dependent. We ran two experiments asking participants to detect changes in locations between memory and probe for one object highlighted in the probe image. We manipulated the changes in viewpoint between memory and probe (Exp. 1: 0°, 30°, 60°; Exp. 2: 0°, 60°), as well as the spatial configuration visible in the probe image (Exp. 1: full configuration, partial configuration; Exp. 2: full configuration, no configuration). Location change detection was higher with the full spatial configuration than with the partial configuration or with no spatial configuration at viewpoint changes of 0°, thus replicating previous findings on the nonindependent representations of individual objects in VWM. Most importantly, the effect of spatial configurations decreased with increasing viewpoint changes, suggesting a viewpoint-dependent representation of contextual information in VWM. We discuss these findings within the context of this special issue, in particular whether research performed within the slots-versus-resources debate and research on the effects of contextual information might focus on two different storage systems within VWM.

Global scene layout modulates contextual learning in change detection

Change in the visual scene often goes unnoticed – a phenomenon referred to as “change blindness.” This study examined whether the hierarchical structure, i.e., the global–local layout of a scene can influence performance in a one-shot change detection paradigm. To this end, natural scenes of a laid breakfast table were presented, and observers were asked to locate the onset of a new local object. Importantly, the global structure of the scene was manipulated by varying the relations among objects in the scene layouts. The very same items were either presented as global-congruent (typical) layouts or as global-incongruent (random) arrangements. Change blindness was less severe for congruent than for incongruent displays, and this congruency benefit increased with the duration of the experiment. These findings show that global layouts are learned, supporting detection of local changes with enhanced efficiency. However, performance was not affected by scene congruency in a subsequent control experiment that required observers to localize a static discontinuity (i.e., an object that was missing from the repeated layouts). Our results thus show that learning of the global layout is particularly linked to the local objects. Taken together, our results reveal an effect of “global precedence” in natural scenes. We suggest that relational properties within the hierarchy of a natural scene are governed, in particular, by global image analysis, reducing change blindness for local objects through scene learning.

Differential functioning of retrieval/comparison processing in detection of the presence and absence of change

Background

The phenomenon of change blindness may reflect the failure to detect the presence of change or the absence of change. Although performing the latter is considered more difficult than the former, the differential functioning of retrieval/comparison processing that leads to differences between the detection of the presence and the absence of change has not been clarified. This study aimed to fill this research gap by comparing performance in the detection of the presence and the absence of a change in one item among a set of items.

Methodology/Principal Findings

Twenty subjects performed two types of change detection tasks, the first task was detection of one changed item among a set of unchanged items (detection of the presence of a change) and the other was the detection of one unchanged item among a set of changed items (detection of the absence of a change). The ANOVA results for the percentage of correct responses and signal detection measurement of A’ values regarding change detection and the pattern of the results indicate that the subjects found (1) detection of the presence of change less difficult than detection of the absence of change (2), rejection of the presence of change less difficult than acceptance of the presence of change, and (3) rejection of the absence of change as difficult as acceptance of the absence of change.

Conclusions/Significance

Retrieval/comparison processing for the detection of the presence of change differs from that for the absence of change, likely because the retrieval/comparison process appears aimed at determining whether an item has changed but not whether an item appears the same as it had previously. This conclusion suggests the existence of an identification process that recognizes each item as the same as that observed previously that exists apart from the mechanism underlying retrieval/comparison processing.

Strength of retinotopic representation of visual memories is modulated by strategy

Human visual cortex shows retinotopic organization during both perception and attention, but whether this remains true for visual short-term memory (VSTM) is uncertain. In 2 functional magnetic resonance imaging experiments, we separated retinotopic activation during perception, attention, and VSTM maintenance. The 2 experiments differed in whether spatial encoding of the VSTM stimuli and prospective attention to the locations of the remembered items was encouraged or discouraged. Using multivoxel pattern analysis to extract a measure of spatial coding in early visual cortex, we saw sensory and attentional retinotopic coding in both experiments. However, significant spatial coding during memory maintenance was only seen where a spatial strategy was encouraged. Furthermore, individual differences in attentional spatial coding predicted performance in both experiments, while individual differences in maintenance spatial coding predicted performance in neither. We conclude that retinotopic coding in the early visual cortex during VSTM maintenance is not obligatory, that attentional processes during stimulus perception modulate memory performance, and that different attentional strategies are used depending on the task in hand.

Attention to attributes and objects in working memory

It has been debated on the basis of change-detection procedures whether visual working memory is limited by the number of objects, task-relevant attributes within those objects, or bindings between attributes. This debate, however, has been hampered by several limitations, including the use of conditions that vary between studies and the absence of appropriate mathematical models to estimate the number of items in working memory in different stimulus conditions. We reexamined working memory limits in 2 experiments with a wide array of conditions involving color and shape attributes, relying on a set of new models to fit various stimulus situations. In Experiment 2, a new procedure allowed identical retrieval conditions across different conditions of attention at encoding. The results show that multiple attributes compete for attention, but that retaining the binding between attributes is accomplished only by retaining the attributes themselves. We propose a theoretical account in which a fixed object capacity limit contains within it the possibility of the incomplete retention of object attributes, depending on the direction of attention.

Lifespan age differences in working memory: a two-component framework

We suggest that working memory (WM) performance can be conceptualized as the interplay of low-level feature binding processes and top-down control, relating to posterior and frontal brain regions and their interaction in a distributed neural network. We propose that due to age-differential trajectories of posterior and frontal brain regions top-down control processes are not fully mature until young adulthood and show marked decline with advancing age, whereas binding processes are relatively mature in children, but show senescent decline in older adults. A review of the literature spanning from middle childhood to old age shows that binding and top-down control processes undergo profound changes across the lifespan. We illustrate commonalities and dissimilarities between children, younger adults, and older adults reflecting the change in the two components' relative contribution to visual WM performance across the lifespan using results from our own lab. We conclude that an integrated account of visual WM lifespan changes combining research from behavioral neuroscience and cognitive psychology of child development as well as aging research opens avenues to advance our understanding of cognition in general.

Models of verbal working memory capacity: what does it take to make them work?

Theories of working memory (WM) capacity limits will be more useful when we know what aspects of performance are governed by the limits and what aspects are governed by other memory mechanisms. Whereas considerable progress has been made on models of WM capacity limits for visual arrays of separate objects, less progress has been made in understanding verbal materials, especially when words are mentally combined to form multiword units or chunks. Toward a more comprehensive theory of capacity limits, we examined models of forced-choice recognition of words within printed lists, using materials designed to produce multiword chunks in memory (e.g., leather brief case). Several simple models were tested against data from a variety of list lengths and potential chunk sizes, with test conditions that only imperfectly elicited the interword associations. According to the most successful model, participants retained about 3 chunks on average in a capacity-limited region of WM, with some chunks being only subsets of the presented associative information (e.g., leather brief case retained with leather as one chunk and brief case as another). The addition to the model of an activated long-term memory component unlimited in capacity was needed. A fixed-capacity limit appears critical to account for immediate verbal recognition and other forms of WM. We advance a model-based approach that allows capacity to be assessed despite other important processing contributions. Starting with a psychological-process model of WM capacity developed to understand visual arrays, we arrive at a more unified and complete model.

Configural representations in spatial working memory: modulation by perceptual segregation and voluntary attention

In what form are multiple spatial locations represented in working memory? The present study revealed that people often maintain the configural properties (interitem relationships) of visuospatial stimuli even when this information is explicitly task-irrelevant. However, the results also indicated that the voluntary allocation of selective attention prior to stimulus presentation, as well as feature-based perceptual segregation of relevant from irrelevant stimuli, can eliminate the influence of stimulus configuration on location-change detection performance. In contrast, voluntary attention cued to the relevant target location following presentation of the stimulus array failed to attenuate these influences. Thus, whereas voluntary selective attention can isolate or prevent the encoding of irrelevant stimulus locations and configural properties, people, perhaps due to limitations in attentional resources, reliably fail to isolate or suppress configural representations that have been encoded into working memory.