Evidence of gradual loss of precision for simple features and complex objects in visual working memory

Dissecting the components of error in analogue report tasks

Over the last two decades, the analogue report task has become a standard method for measuring the fidelity of visual representations across research domains including perception, attention, and memory. Despite its widespread use, there has been no methodical investigation of the different task parameters that might contribute to response variability. To address this gap, we conducted two experiments manipulating components of a typical analogue report test of memory for colour hue. We found that human response errors were independently affected by changes in storage and maintenance requirements of the task, demonstrated by a strong effect of set size even in the absence of a memory delay. In contrast, response variability remained unaffected by physical size of the colour wheel, implying negligible contribution of motor noise to task performance, or by its chroma radius, highlighting non-uniformity of the standard colour space. Comparing analogue report to a matched forced-choice task, we found variation in adjustment criterion made a limited contribution to analogue report variability, becoming meaningful only with low representational noise. Our findings validate the analogue report task as a robust measure of representational fidelity for most purposes, while also quantifying non-representational sources of noise that would limit its reliability in specialized settings.

Representation and computation in visual working memory

The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have advanced our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event. New experimental tools have revealed that observers possess richer information about the uncertainty in their memories and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visuospatial WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research on human WM from a computational perspective, including the neural mechanisms that support it.

A Neural Decision Signal during Internal Sampling from Working Memory in Humans

How humans transform sensory information into decisions that steer purposeful behavior is a central question in psychology and neuroscience that is traditionally investigated during the sampling of external environmental signals. The decision-making framework of gradual information sampling toward a decision has also been proposed to apply when sampling internal sensory evidence from working memory. However, neural evidence for this proposal remains scarce. Here we show (using scalp EEG in male and female human volunteers) that sampling internal visual representations from working memory elicits a scalp EEG potential associated with gradual evidence accumulation-the central parietal positivity. Consistent with an evolving decision process, we show how this signal (1) scales with the time participants require to reach a decision about the cued memory content and (2) is amplified when having to decide among multiple contents in working memory. These results bring the electrophysiology of decision-making into the domain of working memory and suggest that variability in memory-guided behavior may be driven (at least in part) by variations in the sampling of our inner mental contents.

A dynamic neural resource model bridges sensory and working memory

Probing memory of a complex visual image within a few hundred milliseconds after its disappearance reveals significantly greater fidelity of recall than if the probe is delayed by as little as a second. Classically interpreted, the former taps into a detailed but rapidly decaying visual sensory or 'iconic' memory (IM), while the latter relies on capacity-limited but comparatively stable visual working memory (VWM). While iconic decay and VWM capacity have been extensively studied independently, currently no single framework quantitatively accounts for the dynamics of memory fidelity over these time scales. Here, we extend a stationary neural population model of VWM with a temporal dimension, incorporating rapid sensory-driven accumulation of activity encoding each visual feature in memory, and a slower accumulation of internal error that causes memorized features to randomly drift over time. Instead of facilitating read-out from an independent sensory store, an early cue benefits recall by lifting the effective limit on VWM signal strength imposed when multiple items compete for representation, allowing memory for the cued item to be supplemented with information from the decaying sensory trace. Empirical measurements of human recall dynamics validate these predictions while excluding alternative model architectures. A key conclusion is that differences in capacity classically thought to distinguish IM and VWM are in fact contingent upon a single resource-limited WM store.

Working memory performance is tied to stimulus complexity

Working memory is the cognitive capability to maintain and process information over short periods. Behavioral and computational studies have shown that visual information is associated with working memory performance. However, the underlying neural correlates remain unknown. To identify how visual information affects working memory performance, we conducted behavioral experiments in pigeons (Columba livia) and single unit recordings in the avian prefrontal analog, the nidopallium caudolaterale (NCL). Complex pictures featuring luminance, spatial and color information, were associated with higher working memory performance compared to uniform gray pictures in conjunction with distinct neural coding patterns. For complex pictures, we found a multiplexed neuronal code displaying visual and value-related features that switched to a representation of the upcoming choice during a delay period. When processing gray stimuli, NCL neurons did not multiplex and exclusively represented the choice already during stimulus presentation and throughout the delay period. The prolonged representation possibly resulted in a decay of the memory trace ultimately leading to a decrease in performance. In conclusion, we found that high stimulus complexity is associated with neuronal multiplexing of the working memory representation possibly allowing a facilitated read-out of the neural code resulting in enhancement of working memory performance.

Reference induces biases in late visual processing

How we perceive a visual stimulus can be influenced by its surrounding context. For example, the presence of a reference skews the perception of a similar feature in a stimulus, a phenomenon called reference repulsion. Ongoing research so far remains inconclusive regarding the stage of visual information processing where such repulsion occurs. We examined the influence of a reference on late visual processing. We measured the repulsion effect caused by an orientation reference presented after an orientation ensemble stimulus. The participants' reported orientations were significantly biased away from the post-stimulus reference, displaying typical characteristics of reference repulsion. Moreover, explicit discrimination choices between the reference and the stimulus influenced the magnitudes of repulsion effects, which can be explained by an encoding-decoding model that differentiates the re-weighting of sensory representations in implicit and explicit processes. These results support the notion that reference repulsion may arise at a late decision-related stage of visual processing, where different sensory decoding strategies are employed depending on the specific task.

Attractor dynamics with activity-dependent plasticity capture human working memory across time scales

Most cognitive functions require the brain to maintain immediately preceding stimuli in working memory. Here, using a human working memory task with multiple delays, we test the hypothesis that working memories are stored in a discrete set of stable neuronal activity configurations called attractors. We show that while discrete attractor dynamics can approximate working memory on a single time scale, they fail to generalize across multiple timescales. This failure occurs because at longer delay intervals the responses contain more information about the stimuli than can be stored in a discrete attractor model. We present a modeling approach that combines discrete attractor dynamics with activity-dependent plasticity. This model successfully generalizes across all timescales and correctly predicts intertrial interactions. Thus, our findings suggest that discrete attractor dynamics are insufficient to model working memory and that activity-dependent plasticity improves durability of information storage in attractor systems.

Assessing the interaction between working memory and perception through time

Content maintained in visual working memory changes concurrent visual processing, suggesting that visual working memory may recruit an overlapping neural representation with visual perception. However, it remains unclear whether visual working memory representations persist as a sensory code through time, or are recoded later into an abstract code. Here, we directly contrasted a temporal decay + visual code account and a temporal decay + abstract code account within the temporal dynamics of the interaction between working memory and perception. By manipulating the ISI (inter-stimulus interval) between working memory encoding and a perceptual discrimination task, we found that task-relevant and therefore actively maintained perceptual information parametrically altered participants' ability to discriminate perceptual stimuli even 4 s after encoding, whereas task-irrelevant information caused only an acutely transient effect. While continuously present, the size of this shift in discrimination thresholds gradually decreased over time. Concomitantly, the size of the bias in working memory reports increased over time. The opposing directions of threshold and bias effects are consistent with the local maintenance of information in perceptual areas, explained by a temporal decay + visual code account. As the maintained representation decays over time, its ability to alter incoming perceptual signals decreases (reduced threshold effects) while its likelihood of being impacted by those same signals increases (increased bias effects). Altogether, these results suggest that the readout of working memory relies on a sensory representation at a cost of increased interference by ongoing perception.

The effect of memory load on object reconstruction: Insights from an online mouse-tracking task

Why can't we remember everything that we experience? Previous work in the domain of object memory has suggested that our ability to resolve interference between relevant and irrelevant object features may limit how much we can remember at any given moment. Here, we developed an online mouse-tracking task to study how memory load influences object reconstruction, testing participants synchronously over virtual conference calls. We first tested up to 18 participants concurrently, replicating memory findings from a condition where participants were tested individually. Next, we examined how memory load influenced mouse trajectories as participants reconstructed target objects. We found interference between the contents of working memory and what was perceived during object reconstruction, an effect that interacted with visual similarity and memory load. Furthermore, we found interference from previously studied but currently irrelevant objects, providing evidence of object-to-location binding errors. At the greatest memory load, participants were nearly three times more likely to move their mouse cursor over previously studied nontarget objects, an effect observed primarily during object reconstruction rather than in the period before the final response. As evidence of the dynamic interplay between working memory and perception, these results show that object reconstruction behavior may be altered by (i) interference between what is represented in mind and what is currently being viewed, and (ii) interference from previously studied but currently irrelevant information. Finally, we discuss how mouse tracking can provide a rich characterization of participant behavior at millisecond temporal resolution, enormously increasing power in cognitive psychology experiments.

Long-term high-altitude exposure influences task-related representations in visual working memory

Objective

Human working memory is impaired when individuals are exposed to high altitudes, however, whether the capacity of visual working memory is affected remains unclear. This study combined a lateralized change detection task and event-related potentials analysis to explore changes in visual working memory capacity among individuals who emigrated from a low-altitude environment to Tibet (a high-altitude environment).

Materials and methods

Thirty-five college students were recruited from Tibet University as the high-altitude (HA) group, and thirty-six low-altitude (LA) students were enrolled from South China Normal University (sea level) as the LA group. We measured participants' contralateral delay activity (CDA) under different memory loads.

Results

ERP component analysis showed that both the HA and LA groups reached an asymptote at memory load four. However, the contralateral and ipsilateral activity of the HA and LA groups shows different patterns. The results showed a significantly larger contralateral activity for the LA group than for the HA group at memory load one (p = 0.04, Cohen's d = 0.52) and load three (p = 0.02, Cohen's d = 0.61). Additionally, we found marginally larger contralateral activity at memory load four for the LA group (p = 0.06, Cohen's d = 0.47), but not at memory load two (p = 0.10) or load five (p = 0.12). No significant differences were observed for ipsilateral activity. In addition, we observed that the HA group performed larger ipsilateral activity than contralateral activity under each memory load, compared with the LA group.

Conclusion

These findings demonstrated that the attentional resource of long-term HA exposure is more captured by task-irrelevant information, potentially due to impaired inhibitory control, which makes it difficult for them to exclude the interference of task-irrelevant information.

Memory decay enhances central bias in time perception

Temporal expectations are essential for appropriately interacting with the environment, but they can be biased. This tendency, called central bias, places higher weights on expected rather than actual duration distributions when perceiving incoming sensory stimuli. In particular, the central bias is strengthened in order to decrease total response error when incoming sensory stimuli are unclear. In the present study, we investigated whether the central bias was enhanced via memory decay. For this, we used a delayed reproduction task, manipulating retention periods by introducing delays between the sample interval and the reproduction phase (0.4, 2, 4 s in Experiment 1; 0.4, 2, 8 s in Experiments 2 and 3). Through three experiments, we found the gradual strengthening of the central bias as a function of the retention period (i.e., short-term memory decay). This suggests that the integration of temporal expectation, generated from past trials and stored sensory stimuli, in a current trial occurs in the reproduction phase in the delayed reproduction task.

An adaptive perspective on visual working memory distortions

When holding multiple items in visual working memory, representations of individual items are often attracted to, or repelled from, each other. While this is empirically well-established, existing frameworks do not account for both types of distortions, which appear to be in opposition. Here, we demonstrate that both types of memory distortion may confer functional benefits under different circumstances. When there are many items to remember and subjects are near their capacity to accurately remember each item individually, memories for each item become more similar (attraction). However, when remembering smaller sets of highly similar but discernible items, memory for each item becomes more distinct (repulsion), possibly to support better discrimination. Importantly, this repulsion grows stronger with longer delays, suggesting that it dynamically evolves in memory and is not just a differentiation process that occurs during encoding. Furthermore, both attraction and repulsion occur even in tasks designed to mitigate response bias concerns, suggesting they are genuine changes in memory representations. Together, these results are in line with the theory that attraction biases act to stabilize memory signals by capitalizing on information about an entire group of items, whereas repulsion biases reflect a tradeoff between maintaining accurate but distinct representations. Both biases suggest that human memory systems may sacrifice veridical representations in favor of representations that better support specific behavioral goals. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Working Memory Content Is Distorted by Its Use in Perceptual Comparisons

Visual information around us is rarely static. To perform a task in such a dynamic environment, we often have to compare current visual input with our working memory (WM) representation of the immediate past. However, little is known about what happens to a WM representation when it is compared with perceptual input. To test this, we asked young adults (N = 170 total in three experiments) to compare a new visual input with a WM representation prior to reporting the WM representation. We found that the perceptual comparison biased the WM report, especially when the input was subjectively similar to the WM representation. Furthermore, using computational modeling and individual-differences analyses, we found that this similarity-induced memory bias was driven by representational integration, rather than incidental confusion, between the WM representation and subjectively similar input. Together, our findings highlight a novel source of WM distortion and suggest a general mechanism that determines how WM interacts with new visual input.

Magnitude and sources of proactive interference in visual memory

Proactive interference - the disruptive effect of old memories on new learning - is a long-established forgetting mechanism, yet there are doubts about its impact on visual working memory and uncertainty about the kinds of information that cause proactive interference. The present study aimed to assess these issues in three experiments using a modified recent probes task. Participants encoded four target images on each trial and determined whether a probe matched one of those targets. In Experiment 1, probes matching targets from trial N-1 or N-3 damaged responding in relation to a novel probe. Proactive interference was also produced by probes differing in state to a previously experienced target. This was further assessed in Experiments 2 and 3. Here, probes differing in colour to a previous target, or matching the general target category only, produced little proactive interference. Conversely, probes directly matching a prior target, or differing in state information, hindered task performance. This study found robust proactive interference in visual working memory that could endure over multiple trials, but it was also produced by stimuli closely resembling an old target. This challenges the notion that proactive interference is produced by an exact representation of a previously encoded image.

Temporal dynamics of implicit memory underlying serial dependence

Serial dependence is the effect in which the immediately preceding trial influences participants' responses to the current stimulus. But for how long does this bias last in the absence of interference from other stimuli? Here, we had 20 healthy young adult participants (12 women) perform a coincident timing task using different inter-trial intervals to characterize the serial dependence effect as the time between trials increases. Our results show that serial dependence abruptly decreases from 0.1 s to 1 s inter-trial interval, but it remains pronounced after that for up to 8 s. In addition, participants' response variability slightly decreases over longer intervals. We discuss these results in light of recent models suggesting that serial dependence might rely on a short-term memory trace kept through changes in synaptic weights, which might explain its long duration and apparent stability over time.

Errors in visuospatial working memory across space and time

Visuospatial working memory (VSWM) involves cortical regions along the dorsal visual pathway, which are topographically organized with respect to the visual space. However, it remains unclear how such functional organization may constrain VSWM behavior across space and time. Here, we systematically mapped VSWM performance across the 2-dimensional (2D) space in various retention intervals in human subjects using the memory-guided and visually guided saccade tasks in two experiments. Relative to visually guided saccades, memory-guided saccades showed significant increases in unsystematic errors, or response variability, with increasing target eccentricity (3°-13° of visual angle). Unsystematic errors also increased with increasing delay (1.5-3 s, Experiment 1; 0.5-5 s, Experiment 2), while there was little or no interaction between delay and eccentricity. Continuous bump attractor modeling suggested neurophysiological and functional organization factors in the increasing unsystematic errors in VSWM across space and time. These findings indicate that: (1) VSWM representation may be limited by the functional topology of the visual pathway for the 2D space; (2) Unsystematic errors may reflect accumulated noise from memory maintenance while systematic errors may originate from non-mnemonic processes such as noisy sensorimotor transformation; (3) There may be independent mechanisms supporting the spatial and temporal processing of VSWM.

Attention fluctuations impact ongoing maintenance of information in working memory

Working memory maintains information in a readily accessible state and has been shown to degrade as the length of the retention interval increases. Previous research has suggested that this decline is attributable to changes in precision as well as sudden loss of item representations. Here, by measuring trial-to-trial variations in performance, we examined an orthogonal distinction between the maximum number of items that an individual can store, and the probability of achieving that maximum. Across two experiments, we replicated the finding that performance declines after long (10 s) retention intervals, as well as past observations that forgetting was due to probabilistic dropping of individual items rather than all-or-none losses of the stored memories. Critically, longer retention intervals did not reduce the maximum amount of information that could be stored in working memory. Instead, lower attentional control accounted for a decreased probability of maintaining the maximum number of items in working memory. Thus, longer retention intervals impact working memory storage via fluctuations in attentional control that lower the probability of achieving a stable maximum storage capacity.

Delay-period activity in frontal, parietal, and occipital cortex tracks noise and biases in visual working memory

Working memory is imprecise, and these imprecisions can be explained by the combined influences of random diffusive error and systematic drift toward a set of stable states ("attractors"). However, the neural correlates of diffusion and drift remain unknown. Here, we investigated how delay-period activity in frontal and parietal cortex, which is known to correlate with the decline in behavioral memory precision observed with increasing memory load, might relate to diffusion and drift. We analyzed data from an existing experiment in which subjects performed delayed recall for line orientation, at different loads, during functional magnetic resonance imaging (fMRI) scanning. To quantify the influence of drift and diffusion, we modeled subjects' behavior using a discrete attractor model and calculated within-subject correlation between frontal and parietal delay-period activity and whole-trial estimates of drift and diffusion. We found that although increases in frontal and parietal activity were associated with increases in both diffusion and drift, diffusion explained the most variance in frontal and parietal delay-period activity. In comparison, a subsequent whole-brain regression analysis showed that drift, rather than diffusion, explained the most variance in delay-period activity in lateral occipital cortex. These results are consistent with a model of the differential recruitment of general frontoparietal mechanisms in response to diffusive noise and of stimulus-specific biases in occipital cortex.

Individual differences in infancy research: Letting the baby stand out from the crowd

Within the developmental literature, there is an often unspoken tension between studies that aim to capture broad scale, fairly universal nomothetic traits, and studies that focus on mechanisms and trajectories that are idiographic and bounded to some extent by systematic individual differences. The suitability of these approaches varies as a function of the specific research interests at hand. Although the approaches are interdependent, they have often proceeded as parallel research traditions. The current review notes some of the historical and empirical bases for this divide and suggests that each tradition would benefit from incorporating both methodological approaches to iteratively examine universal (nomothetic) phenomena and the individual differences (idiographic) factors that lead to variation in development. This work may help isolate underlying causal mechanisms, better understand current functioning, and predict long-term developmental consequences. In doing so, we also highlight empirical and structural issues that need to be addressed to support this integration.

Theory of neural coding predicts an upper bound on estimates of memory variability

Observers reproducing elementary visual features from memory after a short delay produce errors consistent with the encoding-decoding properties of neural populations. While inspired by electrophysiological observations of sensory neurons in cortex, the population coding account of these errors is based on a mathematical idealization of neural response functions that abstracts away most of the heterogeneity and complexity of real neuronal populations. Here we examine a more physiologically grounded model based on the tuning of a large set of neurons recorded in macaque V1 and show that key predictions of the idealized model are preserved. Both models predict long-tailed distributions of error when memory resources are taxed, as observed empirically in behavioral experiments and commonly approximated with a mixture of normal and uniform error components. Specifically, for an idealized homogeneous neural population, the width of the fitted normal distribution cannot exceed the average tuning width of the component neurons, and this also holds to a good approximation for more biologically realistic populations. Examining eight published studies of orientation recall, we find a consistent pattern of results suggestive of a median tuning width of approximately 20°, which compares well with neurophysiological observations. The finding that estimates of variability obtained by the normal-plus-uniform mixture method are bounded from above leads us to reevaluate previous studies that interpreted a saturation in width of the normal component as evidence for fundamental limits on the precision of perception, working memory, and long-term memory.

Drifting codes within a stable coding scheme for working memory

Working memory (WM) is important to maintain information over short time periods to provide some stability in a constantly changing environment. However, brain activity is inherently dynamic, raising a challenge for maintaining stable mental states. To investigate the relationship between WM stability and neural dynamics, we used electroencephalography to measure the neural response to impulse stimuli during a WM delay. Multivariate pattern analysis revealed representations were both stable and dynamic: there was a clear difference in neural states between time-specific impulse responses, reflecting dynamic changes, yet the coding scheme for memorised orientations was stable. This suggests that a stable subcomponent in WM enables stable maintenance within a dynamic system. A stable coding scheme simplifies readout for WM-guided behaviour, whereas the low-dimensional dynamic component could provide additional temporal information. Despite having a stable subspace, WM is clearly not perfect-memory performance still degrades over time. Indeed, we find that even within the stable coding scheme, memories drift during maintenance. When averaged across trials, such drift contributes to the width of the error distribution.

Brain Processing of Complex Geometric Forms in a Visual Memory Task Increases P2 Amplitude

We study the cognitive processing of visual working memory in three different conditions of memory load and configuration change. Altering this features has been shown to alter the brain's processing in memory tasks. Most studies dealing with this issue have used the verbal-phonological modality. We use complex geometric polygons to assess visual working memory in a modified change detection task. Three different types of backgrounds were used to manipulate memory loading and 18 complex geometric polygons to manipulate stimuli configuration. The goal of our study was to test whether the memory load and configuration affect the correct-recall ratios. We expected that increasing visual items loading and changing configuration of items would induce differences in working memory performance. Brain activity related to the task was assessed through event-related potentials (ERP), during the test phase of each trial. Our results showed that visual items loading and changing of item configuration affect working memory on test phase on ERP component P2, but does not affect performance. However frontal related ERP component-P3-was minimally affected by visual memory loading or configuration changing, supporting that working memory is related to a filtering processing in posterior brain regions.

The visual nonverbal memory trace is fragile when actively maintained, but endures passively for tens of seconds

Despite attempts at active maintenance in the focus of attention, the fragile nature of the visual nonverbal memory trace may be revealed when the retention interval between target memoranda and probed recall on a trial is extended. In contrast, a passively maintained or unattended visual memory trace may be revealed as persisting proactive interference extending across quite extended intervals between trials in a recent probes task. The present study, comprising five experiments, used this task to explore the persistence of such a passive visual memory trace over time. Participants viewed some target visual items (for example, abstract colored patterns) followed by a variable retention interval and a probe item. The task was to report whether the probe matched one of the targets or not. A decaying active memory trace was indicated by poorer performance as the memory retention interval was extended on a trial. However, when the probe was a member of the target set from the preceding trial, task performance was poorer than a comparison novel probe, demonstrating proactive interference. Manipulations of the intertrial interval revealed that the temporal persistence of the passive memory trace of an old target was impressive, and proactive interference was largely resilient to a simple ‘cued forgetting’ manipulation. These data support the proposed two-process memory conception (active–passive memory) contrasting fragile active memory traces decaying over a few seconds with robust passive traces extending to tens of seconds.

Scale-invariant Characteristics of Forgetting: Toward a Unifying Account of Hippocampal Forgetting across Short and Long Timescales

After over 100 years of relative silence in the cognitive literature, recent advances in the study of the neural underpinnings of memory-specifically, the hippocampus-have led to a resurgence of interest in the topic of forgetting. This review draws a theoretically driven picture of the effects of time on forgetting of hippocampus-dependent memories. We review evidence indicating that time-dependent forgetting across short and long timescales is reflected in progressive degradation of hippocampal-dependent relational information. This evidence provides an important extension to a growing body of research accumulated in recent years, showing that-in contrast to the once prevailing view that the hippocampus is exclusively involved in memory and forgetting over long timescales-the role of the hippocampus also extends to memory and forgetting over short timescales. Thus, we maintain that similar rules govern not only remembering but also forgetting of hippocampus-dependent information over short and long timescales.

Error-correcting dynamics in visual working memory

Working memory is critical to cognition, decoupling behavior from the immediate world. Yet, it is imperfect; internal noise introduces errors into memory representations. Such errors have been shown to accumulate over time and increase with the number of items simultaneously held in working memory. Here, we show that discrete attractor dynamics mitigate the impact of noise on working memory. These dynamics pull memories towards a few stable representations in mnemonic space, inducing a bias in memory representations but reducing the effect of random diffusion. Model-based and model-free analyses of human and monkey behavior show that discrete attractor dynamics account for the distribution, bias, and precision of working memory reports. Furthermore, attractor dynamics are adaptive. They increase in strength as noise increases with memory load and experiments in humans show these dynamics adapt to the statistics of the environment, such that memories drift towards contextually-predicted values. Together, our results suggest attractor dynamics mitigate errors in working memory by counteracting noise and integrating contextual information into memories.

Neural representations in working memory are susceptible to internal noise, which scales with memory load. Here, the authors show that attractor dynamics mitigate the influence of internal noise by pulling memories towards a few stable representations.

Coexisting representations of sensory and mnemonic information in human visual cortex

Traversing sensory environments requires keeping relevant information in mind while simultaneously processing new inputs. Visual information is kept in working memory via feature selective responses in early visual cortex, but recent work had suggested that new sensory inputs obligatorily wipe out this information. Here we show region-wide multiplexing abilities in classic sensory areas, with population-level response patterns in early visual cortex representing the contents of working memory alongside new sensory inputs. In a second experiment, we show that when people get distracted, this leads to both disruptions of mnemonic information in early visual cortex and decrements in behavioral recall. Representations in the intraparietal sulcus reflect actively remembered information encoded in a transformed format, but not task-irrelevant sensory inputs. Together these results suggest that early visual areas play a key role in supporting high resolution working memory representations that can serve as a template for comparing incoming sensory information.

Not-so-working Memory: Drift in Functional Magnetic Resonance Imaging Pattern Representations during Maintenance Predicts Errors in a Visual Working Memory Task

Working memory (WM) is critical to many aspects of cognition, but it frequently fails. Much WM research has focused on capacity limits, but even for single, simple features, the fidelity of individual representations is limited. Why is this? One possibility is that, because of neural noise and interference, neural representations do not remain stable across a WM delay, nor do they simply decay, but instead, they may "drift" over time to a new, less accurate state. We tested this hypothesis in a functional magnetic resonance imaging study of a match/nonmatch WM recognition task for a single item with a single critical feature: orientation. We developed a novel pattern-based index of "representational drift" to characterize ongoing changes in brain activity patterns throughout the WM maintenance period, and we were successfully able to predict performance on the match/nonmatch recognition task using this representational drift index. Specifically, in trials where the target and probe stimuli matched, participants incorrectly reported more nonmatches when their activity patterns drifted away from the target. In trials where the target and probe did not match, participants incorrectly reported more matches when their activity patterns drifted toward the probe. On the basis of these results, we contend that neural noise does not cause WM errors merely by degrading representations and increasing random guessing; instead, one means by which noise introduces errors is by pushing WM representations away from the target and toward other meaningful (yet incorrect) configurations. Thus, we demonstrate that behaviorally meaningful drift within representation space can be indexed by neuroimaging.

A Flexible Model of Working Memory

Working memory is fundamental to cognition, allowing one to hold information "in mind." A defining characteristic of working memory is its flexibility: we can hold anything in mind. However, typical models of working memory rely on finely tuned, content-specific attractors to persistently maintain neural activity and therefore do not allow for the flexibility observed in behavior. Here, we present a flexible model of working memory that maintains representations through random recurrent connections between two layers of neurons: a structured "sensory" layer and a randomly connected, unstructured layer. As the interactions are untuned with respect to the content being stored, the network maintains any arbitrary input. However, in our model, this flexibility comes at a cost: the random connections overlap, leading to interference between representations and limiting the memory capacity of the network. Additionally, our model captures several other key behavioral and neurophysiological characteristics of working memory.

Separating memoranda in depth increases visual working memory performance

Visual working memory is the mechanism supporting the continued maintenance of information after sensory inputs are removed. Although the capacity of visual working memory is limited, memoranda that are spaced farther apart on a 2-D display are easier to remember, potentially because neural representations are more distinct within retinotopically organized areas of visual cortex during memory encoding, maintenance, or retrieval. The impact on memory of spatial separability in depth is less clear, even though depth information is essential to guiding interactions with objects in the environment. On one account, separating memoranda in depth may facilitate performance if interference between items is reduced. However, depth information must be inferred indirectly from the 2-D retinal image, and less is known about how visual cortex represents depth. Thus, an alternative possibility is that separation in depth does not attenuate between-items interference; it may even impair performance, as attention must be distributed across a larger volume of 3-D space. We tested these alternatives using a stereo display while participants remembered the colors of stimuli presented either near or far in the 2-D plane or in depth. Increasing separation in-plane and in depth both enhanced performance. Furthermore, participants who were better able to utilize stereo depth cues showed larger benefits when memoranda were separated in depth, particularly for large memory arrays. The observation that spatial separation in the inferred 3-D structure of the environment improves memory performance, as is the case in 2-D environments, suggests that separating memoranda in depth might reduce neural competition by utilizing cortically separable resources.