Introspective judgments predict the precision and likelihood of successful maintenance of visual working memory

Evidence for Optimal Integration of Visual Feature Representations across Saccades

We explore the visual world through saccadic eye movements, but saccades also present a challenge to visual processing by shifting externally stable objects from one retinal location to another. The brain could solve this problem in two ways: by overwriting preceding input and starting afresh with each fixation or by maintaining a representation of presaccadic visual features in working memory and updating it with new information from the remapped location. Crucially, when multiple objects are present in a scene the planning of eye movements profoundly affects the precision of their working memory representations, transferring limited memory resources from fixation toward the saccade target. Here we show that when humans make saccades, it results in an update of not just the precision of representations but also their contents. When multiple item colors are shifted imperceptibly during a saccade the perceived colors are found to fall between presaccadic and postsaccadic values, with the weight given to each input varying continuously with item location, and fixed relative to saccade parameters. Increasing sensory uncertainty, by adding color noise, biases updating toward the more reliable input, which is consistent with an optimal integration of presaccadic working memory with a postsaccadic updating signal. We recover this update signal and show it to be tightly focused on the vicinity of the saccade target. These results reveal how the nervous system accumulates detailed visual information from multiple views of the same object or scene.

SIGNIFICANCE STATEMENT

This study examines the consequences of saccadic eye movements for the internal representation of visual objects. A saccade shifts the image of a stable visual object from one part of the retina to another. We show that visual representations are built up over these different views of the same object, by combining information obtained before and after each saccade. The weights given to presaccadic and postsaccadic information are determined by the relative reliability of each input. This provides evidence that the visual system combines inputs over time in a statistically optimal way.

Detection of the number of changes in a display in working memory

Here we examine a new task to assess working memory for visual arrays in which the participant must judge how many items changed from a studied array to a test array. As a clue to processing, on some trials in the first 2 experiments, participants carried out a metamemory judgment in which they were to decide how many items were in working memory. Trial-to-trial fluctuations in these working memory storage judgments correlated with performance fluctuations within an individual, indicating a need to include trial-to-trial variation within capacity models (through either capacity fluctuation or some other attention parameter). Mathematical modeling of the results achieved a good fit to a complex pattern of results, suggesting that working memory capacity limits can apply even to judgments that involve an entire array rather than just a single item that may have changed, thus providing the expected conscious access to at least some of the contents of working memory.

How is working memory content consciously experienced? The 'conscious copy' model of WM introspection

We address the issue of how visual information stored in working memory (WM) is introspected. In other words, how do we become aware of WM content in order to consciously examine or manipulate it? Influential models of WM have suggested that WM representations are either conscious by definition, or directly accessible for conscious inspection. We propose that WM introspection does not operate on the actual memory trace but rather requires a new representation to be created for the conscious domain. This conscious representation exists in addition and in parallel to the actual memory representation. The existence of such a separate representation is revealed by and reflected in the qualitatively different functional characteristics between the actual memory trace and its conscious experience, and their distinct interactions within external visual input. Our model differs from state-based models in that WM introspection does not involve a change in the state of WM content, but rather involves the creation of a new, second representation existing in parallel to the original memory trace.

Expertise for upright faces improves the precision but not the capacity of visual working memory

Considerable research has focused on how basic visual features are maintained in working memory, but little is currently known about the precision or capacity of visual working memory for complex objects. How precisely can an object be remembered, and to what extent might familiarity or perceptual expertise contribute to working memory performance? To address these questions, we developed a set of computer-generated face stimuli that varied continuously along the dimensions of age and gender, and we probed participants' memories using a method-of-adjustment reporting procedure. This paradigm allowed us to separately estimate the precision and capacity of working memory for individual faces, on the basis of the assumptions of a discrete capacity model, and to assess the impact of face inversion on memory performance. We found that observers could maintain up to four to five items on average, with equally good memory capacity for upright and upside-down faces. In contrast, memory precision was significantly impaired by face inversion at every set size tested. Our results demonstrate that the precision of visual working memory for a complex stimulus is not strictly fixed but, instead, can be modified by learning and experience. We find that perceptual expertise for upright faces leads to significant improvements in visual precision, without modifying the capacity of working memory.

Factorial comparison of working memory models

Three questions have been prominent in the study of visual working memory limitations: (a) What is the nature of mnemonic precision (e.g., quantized or continuous)? (b) How many items are remembered? (c) To what extent do spatial binding errors account for working memory failures? Modeling studies have typically focused on comparing possible answers to a single one of these questions, even though the result of such a comparison might depend on the assumed answers to both others. Here, we consider every possible combination of previously proposed answers to the individual questions. Each model is then a point in a 3-factor model space containing a total of 32 models, of which only 6 have been tested previously. We compare all models on data from 10 delayed-estimation experiments from 6 laboratories (for a total of 164 subjects and 131,452 trials). Consistently across experiments, we find that (a) mnemonic precision is not quantized but continuous and not equal but variable across items and trials; (b) the number of remembered items is likely to be variable across trials, with a mean of 6.4 in the best model (median across subjects); (c) spatial binding errors occur but explain only a small fraction of responses (16.5% at set size 8 in the best model). We find strong evidence against all 6 documented models. Our results demonstrate the value of factorial model comparison in working memory.

Reappraising the relationship between working memory and conscious awareness

Classically, the operation of working memory (WM) has been strongly coupled with conscious states; it is thought that WM operates on conscious input and that we are conscious of the contents and operations of WM. Here, we re-evaluate the relationship between WM and conscious awareness in light of current data and question the views that awareness is mandatory for the operation of WM and that WM contents are necessarily linked to experiential states that are consciously accessible for perceptual report. We propose a novel framework for the relationship between WM and conscious awareness.

Accuracy and confidence of visual short-term memory do not go hand-in-hand: behavioral and neural dissociations

Currently influential models of working memory posit that memory content is highly accessible to conscious inspection. These models predict that metacognition of memory performance should go hand-in-hand with the accuracy of the underlying memory representation. To test this view, we investigated how visual information presented during the maintenance period affects VSTM accuracy and confidence. We used a delayed cue-target orientation discrimination task in which participants were asked to hold in memory a grating, and during the maintenance period a second memory cue could be presented. VSTM accuracy of the first memory cue was impaired when the orientation of the second memory cue was sufficiently different. However, participants' response confidence was reduced whenever the second memory cue was presented; thus VSTM accuracy and confidence were dissociated. In a second experiment, we applied transcranial direct current stimulation (tDCS) over the right dorsolateral prefrontal cortex (DLPFC) to investigate the causal role of this region in VSTM metacognition. Relative to the sham condition, anodal tDCS induced a general reduction in confidence ratings but did not affect VSTM accuracy. Overall, these results indicate that our metacognition of memory performance is influenced by factors other than the accuracy of the underlying memory representation.

Noise in neural populations accounts for errors in working memory

Errors in short-term memory increase with the quantity of information stored, limiting the complexity of cognition and behavior. In visual memory, attempts to account for errors in terms of allocation of a limited pool of working memory resources have met with some success, but the biological basis for this cognitive architecture is unclear. An alternative perspective attributes recall errors to noise in tuned populations of neurons that encode stimulus features in spiking activity. I show that errors associated with decreasing signal strength in probabilistically spiking neurons reproduce the pattern of failures in human recall under increasing memory load. In particular, deviations from the normal distribution that are characteristic of working memory errors and have been attributed previously to guesses or variability in precision are shown to arise as a natural consequence of decoding populations of tuned neurons. Observers possess fine control over memory representations and prioritize accurate storage of behaviorally relevant information, at a cost to lower priority stimuli. I show that changing the input drive to neurons encoding a prioritized stimulus biases population activity in a manner that reproduces this empirical tradeoff in memory precision. In a task in which predictive cues indicate stimuli most probable for test, human observers use the cues in an optimal manner to maximize performance, within the constraints imposed by neural noise.

Changing concepts of working memory

Working memory is widely considered to be limited in capacity, holding a fixed, small number of items, such as Miller's 'magical number' seven or Cowan's four. It has recently been proposed that working memory might better be conceptualized as a limited resource that is distributed flexibly among all items to be maintained in memory. According to this view, the quality rather than the quantity of working memory representations determines performance. Here we consider behavioral and emerging neural evidence for this proposal.

Serotonergic modulation of spatial working memory: predictions from a computational network model

Serotonin (5-HT) receptors of types 1A and 2A are strongly expressed in prefrontal cortex (PFC) neurons, an area associated with cognitive function. Hence, 5-HT could be effective in modulating prefrontal-dependent cognitive functions, such as spatial working memory (SWM). However, a direct association between 5-HT and SWM has proved elusive in psycho-pharmacological studies. Recently, a computational network model of the PFC microcircuit was used to explore the relationship between 5-HT and SWM (Cano-Colino et al., 2013). This study found that both excessive and insufficient 5-HT levels lead to impaired SWM performance in the network, and it concluded that analyzing behavioral responses based on confidence reports could facilitate the experimental identification of SWM behavioral effects of 5-HT neuromodulation. Such analyses may have confounds based on our limited understanding of metacognitive processes. Here, we extend these results by deriving three additional predictions from the model that do not rely on confidence reports. Firstly, only excessive levels of 5-HT should result in SWM deficits that increase with delay duration. Secondly, excessive 5-HT baseline concentration makes the network vulnerable to distractors at distances that were robust to distraction in control conditions, while the network still ignores distractors efficiently for low 5-HT levels that impair SWM. Finally, 5-HT modulates neuronal memory fields in neurophysiological experiments: Neurons should be better tuned to the cued stimulus than to the behavioral report for excessive 5-HT levels, while the reverse should happen for low 5-HT concentrations. In all our simulations agonists of 5-HT_1A receptors and antagonists of 5-HT_2A receptors produced behavioral and physiological effects in line with global 5-HT level increases. Our model makes specific predictions to be tested experimentally and advance our understanding of the neural basis of SWM and its neuromodulation by 5-HT receptors.

Serotonin regulates performance nonmonotonically in a spatial working memory network

The prefrontal cortex (PFC) contains a dense network of serotonergic [serotonin, 5-hydroxytryptamine (5-HT)] axons, and endogenous 5-HT markedly modulates PFC neuronal function via several postsynaptic receptors. The therapeutic action of atypical antipsychotic drugs, acting mainly via 5-HT receptors, also suggests a role for serotonergic neurotransmission in cognitive functions. However, psychopharmacological studies have failed to find a consistent relationship between serotonergic transmission and cognitive functions of the PFC, including spatial working memory (SWM). Here, we built a computational network model to investigate 5-HT modulation of SWM in the PFC. We found that 5-HT modulates network's SWM performance nonmonotonically via 5-HT1A and 5-HT2A receptors, following an inverted U-shape. This relationship may contribute to blur the effects of serotonergic agents in previous SWM group-based behavioral studies. Our simulations also showed that errors occurring at low and high 5-HT concentrations are due to different network dynamics instabilities, suggesting that these 2 conditions can be distinguished experimentally based on their distinct dependency on experimental variables. We inferred specific predictions regarding the expected behavioral effects of serotonergic agents in 2 classic working-memory tasks. Our results underscore the relevance of identifying different error types in SWM tasks in order to reveal the association between neuromodulatory systems and SWM.