A Metacognitive Perspective of Visual Working Memory With Rich Complex Objects

Comparing memory capacity across stimuli requires maximally dissimilar foils: Using deep convolutional neural networks to understand visual working memory capacity for real-world objects

The capacity of visual working and visual long-term memory plays a critical role in theories of cognitive architecture and the relationship between memory and other cognitive systems. Here, we argue that before asking the question of how capacity varies across different stimuli or what the upper bound of capacity is for a given memory system, it is necessary to establish a methodology that allows a fair comparison between distinct stimulus sets and conditions. One of the most important factors determining performance in a memory task is target/foil dissimilarity. We argue that only by maximizing the dissimilarity of the target and foil in each stimulus set can we provide a fair basis for memory comparisons between stimuli. In the current work we focus on a way to pick such foils objectively for complex, meaningful real-world objects by using deep convolutional neural networks, and we validate this using both memory tests and similarity metrics. Using this method, we then provide evidence that there is a greater capacity for real-world objects relative to simple colors in visual working memory; critically, we also show that this difference can be reduced or eliminated when non-comparable foils are used, potentially explaining why previous work has not always found such a difference. Our study thus demonstrates that working memory capacity depends on the type of information that is remembered and that assessing capacity depends critically on foil dissimilarity, especially when comparing memory performance and other cognitive systems across different stimulus sets.

Retrospective judgments of confidence in a complex span task

Although the study of metamemory monitoring originated in predictions for simple span tasks, the study of metacognition for working memory (WM) has been somewhat neglected in comparison with long-term memory. We aimed to fill this gap by exploring the ability to self-assess WM operations. Thirty-four participants performed 16 series of complex span tasks and rated their confidence in a verbal recall paradigm. We manipulated the cognitive load based on the TBRS model in order to analyze the role of attentional resources on both WM and metacognitive evaluations. As expected, we found that recall is affected by cognitive load and we found standard serial position effects. Interestingly, metacognitive evaluations followed the same pattern, and measures of metacognitive sensitivity suggest that participants are able to make item-by-item retrospective judgments reflective of their performance. We discuss how these results contribute to our understanding of metacognitive access to newly-formed WM contents.

Disentangling Processing and Storage Accounts of Working Memory Development in Childhood

Researchers have been asking the question of what drives the development of working memory (WM) during childhood for decades. This question is particularly challenging because so many aspects of cognition develop with age that it is difficult to disentangle them and find out which factors are causal or fundamental. In this review, we first prepare to discuss this issue by inquiring whether increases in storage, processing, or both are the fundamental driving factor(s) of the age-related increase in WM capability in childhood. We contend that by experimentally manipulating either factor and observing changes in the other, it is possible to learn about causal roles in WM development. We discuss research on school-aged children that seems to suggest, by means of such an approach, that the growth of storage is causal for some phases or steps in WM tasks, but that the growth of processing is causal for other steps. In our theoretical proposal, storage capacity of the focus of attention determines earlier steps of information processing by constraining the selective encoding of information into WM, whereas processing dependent on the focus of attention determines later steps, like the detection of patterns that can simplify the effective memory load and adoption of a proactive stance of maintenance in dual-task settings. Future directions for research are discussed.

Metacognition in working memory: Confidence judgments during an n-back task

Metacognition in working memory (WM) has received less attention than episodic memory, and few studies have investigated confidence judgements while carrying out a verbal WM task. The present study investigated whether individuals are aware of their own level of performance while carrying out an ongoing verbal WM task, and whether judgments of confidence are sensitive to factors that determine WM performance. A verbal n-back task was adapted to obtain confidence judgments on a trial-by-trial basis. Memory load and lure interference were manipulated. Results showed that metacognition judgments were affected by memory load and levels of interference just as performance accuracy. Even when judgments were sensitive to memory factors, participants were overconfident and generally showed poor metacognitive accuracy at discriminating between erroneous and accurate responses. Results are discussed in terms of possible cues contributing to metacognitive judgements during an ongoing WM task and reasons for WM metacognitive accuracy.

Using spontaneous eye blink-related brain activity to investigate cognitive load during mobile map-assisted navigation

The continuous assessment of pedestrians' cognitive load during a naturalistic mobile map-assisted navigation task is challenging because of limited experimental control over stimulus presentation, human-map-interactions, and other participant responses. To overcome this challenge, the present study takes advantage of navigators' spontaneous eye blinks during navigation to serve as event markers in continuously recorded electroencephalography (EEG) data to assess cognitive load in a mobile map-assisted navigation task. We examined if and how displaying different numbers of landmarks (3 vs. 5 vs. 7) on mobile maps along a given route would influence navigators' cognitive load during navigation in virtual urban environments. Cognitive load was assessed by the peak amplitudes of the blink-related fronto-central N2 and parieto-occipital P3. Our results show increased parieto-occipital P3 amplitude indicating higher cognitive load in the 7-landmark condition, compared to showing 3 or 5 landmarks. Our prior research already demonstrated that participants acquire more spatial knowledge in the 5- and 7-landmark conditions compared to the 3-landmark condition. Together with the current study, we find that showing 5 landmarks, compared to 3 or 7 landmarks, improved spatial learning without overtaxing cognitive load during navigation in different urban environments. Our findings also indicate a possible cognitive load spillover effect during map-assisted wayfinding whereby cognitive load during map viewing might have affected cognitive load during goal-directed locomotion in the environment or vice versa. Our research demonstrates that users' cognitive load and spatial learning should be considered together when designing the display of future navigation aids and that navigators' eye blinks can serve as useful event makers to parse continuous human brain dynamics reflecting cognitive load in naturalistic settings.

Tracking attentional states: Assessing the relationship between sustained and selective focused attention in visual working memory

Attention has multiple influences on visual working memory (VWM). Fluctuations in sustained attention predict VWM performance. Furthermore, focusing selective attention in VWM by retro-cuing the to-be-tested item during maintenance boosts retrieval. So far, we lack knowledge how the ability to focus selective attention relates to the state of sustained attention during the VWM trial. Here, we combined a retro-cue task and a self-rated attention protocol to test whether focusing selective attention via retro-cues: (1) mitigates spontaneous attention fluctuations, in which case retro-cues should be more helpful under low levels of self-rated attention; (2) depends on an optimal state of sustained attention, in which case retro-cue benefits should be largest under high levels of self-rated attention; or (3) is independent of sustained attention, in which case retro-cue benefits and self-rated attention effects should be additive. Our data supported the additive hypothesis. Across four experiments, self-rated attention levels predicted continuous reproduction of colors. Retro-cue trials produced better recall and higher rated attention. Critically, retro-cues improved recall to a similar extent across all levels of self-rated attention. This indicates that attention has multi-faceted and independent contributions to VWM.

The nationality benefit: Long-term memory associations enhance visual working memory for color-shape conjunctions

Visual working memory (VWM) is typically found to be severely limited in capacity, but this limitation may be ameliorated by providing familiar objects that are associated with knowledge stored in long-term memory. However, comparing meaningful and meaningless stimuli usually entails a confound, because different types of objects also tend to vary in terms of their inherent perceptual complexity. The current study therefore aimed to dissociate stimulus complexity from object meaning in VWM. To this end, identical stimuli - namely, simple color-shape conjunctions - were presented, which either resembled meaningful configurations ("real" European flags), or which were rearranged to form perceptually identical but meaningless ("fake") flags. The results revealed complexity estimates for "real" and "fake" flags to be higher than for unicolor baseline stimuli. However, VWM capacity for real flags was comparable to the unicolor baseline stimuli (and substantially higher than for fake flags). This shows that relatively complex, yet meaningful "real" flags reveal a VWM capacity that is comparable to rather simple, unicolored memory items. Moreover, this "nationality" benefit was related to individual flag recognition performance, thus showing that VWM depends on object knowledge.

Uncertainty is maintained and used in working memory

What are the contents of working memory? In both behavioral and neural computational models, a working memory representation is typically described by a single number, namely, a point estimate of a stimulus. Here, we asked if people also maintain the uncertainty associated with a memory and if people use this uncertainty in subsequent decisions. We collected data in a two-condition orientation change detection task; while both conditions measured whether people used memory uncertainty, only one required maintaining it. For each condition, we compared an optimal Bayesian observer model, in which the observer uses an accurate representation of uncertainty in their decision, to one in which the observer does not. We find that this “Use Uncertainty” model fits better for all participants in both conditions. In the first condition, this result suggests that people use uncertainty optimally in a working memory task when that uncertainty information is available at the time of decision, confirming earlier results. Critically, the results of the second condition suggest that this uncertainty information was maintained in working memory. We test model variants and find that our conclusions do not depend on our assumptions about the observer's encoding process, inference process, or decision rule. Our results provide evidence that people have uncertainty that reflects their memory precision on an item-specific level, maintain this information over a working memory delay, and use it implicitly in a way consistent with an optimal observer. These results challenge existing computational models of working memory to update their frameworks to represent uncertainty.